Adults with autism spectrum disorder (ASD) have significant difficulties obtaining and maintaining employment after high school. Results of surveys vary widely but suggest that only about 10 to 50% of individuals with ASD have paying jobs (e.g., Nord, Stancliffe, Nye-Lengerman, & Hewitt, 2016; Shattuck et al., 2012; Taylor & Seltzer, 2011). As a result, many of these individuals depend on other adults for support (Howlin, Goode, Hutton, & Rutter, 2004). The limited post-secondary vocational services available to individuals with ASD are one likely barrier to successful employment. However, some research findings suggest that access to vocational services does not improve outcomes for this population (e.g., Burgess & Cimera, 2014; Taylor & Mailick, 2014).

Few studies have directly examined reasons why adults with ASD struggle with employment or have evaluated interventions to improve their vocational skills (for reviews, see Bennett & Dukes, 2013; Palmen, Didden, & Lang, 2012; Roth, Gillis, & DiGennaro Reed, 2014; Seaman & Cannella-Malone, 2016; Taylor et al., 2012). Studies on the employment difficulties of individuals with intellectual disabilities and surveys of individuals with ASD about their job experiences suggest that interpersonal skills, particularly those involving interactions with coworkers and supervisors, are critical to job success (Baldwin, Costley, & Warren, 2014; Cheney & Foss, 1984; Greenspan & Shoultz, 1981; Herbert & Ishikawa, 1991; Hurlbutt & Chalmers, 2004; Müller, Schuler, Burton, & Yates, 2003). Effective functioning on the job requires a variety of social skills, including how to respond appropriately to corrective feedback and how to ask for help when needed.

Difficulties with social skills is a diagnostic characteristic of ASD and, as such, should be given special consideration when providing services to improve employment outcomes for this population. Individuals with ASD may have difficulties initiating or responding to social interactions, taking the perspective of others, understanding sarcasm or humor, and employing social norms successfully (e.g., using tact; Carter, Davis, Klin, & Volkmar, 2005). They also may misinterpret others’ nonverbal forms of communication, such as gestures and facial expressions (e.g., signs of boredom or frustration). These types of social skills likely are needed to respond successfully when encountering situations that commonly arise in the workplace. For example, employees may need to request help from others when they run out of necessary materials, receive unclear instructions or feedback, or have finished their work. Although a small but growing number of studies have focused on interventions to improve social skills of adults with ASD, none has focused exclusively on those related to interactions on the job (e.g., Alexander, Dummer, Smeltzer, & Denton, 2011; Davis, Boon, Cihak, & Fore, 2010; Dotson, Leaf, Sheldon, & Sherman, 2010; Laugeson, Gantman, Kapp, Orenski, & Ellingsen, 2015).

Research in this area would benefit from further study of typical job-related social skills deficits in individuals with ASD. Identifying the characteristics and consistency of these deficits via assessment also would help inform the development and selection of individualized interventions. Although a number of vocational skill assessments are available, the majority rely solely on indirect measures (i.e., verbal report via questionnaires and rating scales) to gather information on job-related skills (Bullis & Foss, 1986; Foss, Cheney, & Bullis, 1986; Carter & Wehby, 2003; Gal, Meir, & Katz, 2013). Indirect assessments should be combined with more objective assessments that involve direct observation of behavior under a variety of work-related conditions to identify skill deficits and develop behavioral interventions to address them. However, this type of assessment might be challenging to conduct if the individual with ASD is not employed or does not have access to work experiences. Even if an individual is placed in a job, the work site may not provide frequent opportunities to observe all critical social skills (e.g., if the individual is rarely given difficult tasks or vague instructions), requiring clinicians to spend a substantial amount of time observing at the site. Direct observation of individuals at their places of employment also may be intrusive or difficult to conduct for other reasons (e.g., some job sites restrict nonemployees from entering certain areas). Assessments completed in a clinic setting under conditions that resemble a naturalistic job site may be a viable solution for conducting efficient, objective assessments of job-related social skills.

The purposes of this study were to evaluate a clinic-based assessment of job-related social skills and to summarize the results for a small group of adults with ASD who had a history of difficulties obtaining or maintaining employment.

Method

Participants and Setting

Participants were eight individuals, aged 16 to 32 years, who were referred by the state vocational rehabilitation agency for an assessment of job-related social skills due to prior difficulties obtaining or maintaining employment, as reported by caregivers or vocational counselors. These were the first eight individuals diagnosed with an autism spectrum disorder (ASD) to complete the assessment after we had developed and pilot-tested the procedures. No other inclusionary or exclusionary criteria were used to select participants for this study. All of the participants provided documentation showing that they had been diagnosed with ASD by appropriate professionals prior to the study. None was employed and two were attending public high school at the time of the study (Daryl and Glen). One participant (John) was enrolled in college courses at the time of the study, and one participant (Carol) had previously taken college courses. All participants were receiving or had received special education services in school. Five of the participants (Rick, John, Daryl, Carol, and Hershel) had prior volunteer or work experience. Rick had volunteered at a soup kitchen, grocery store, and retail store; Hershel had volunteered at a thrift shop; John had volunteered in a soup kitchen, animal shelter, and thrift shop; Daryl had volunteered at a grocery store and retail store; and Carol had been employed at a local theme park (temporary summer job) and at a department store as a cashier. Hershel had applied for a paid position at the thrift shop after volunteering for several months but was told that he would not be hired, so he quit the volunteer position. Carol was fired from the department store, reportedly for engaging in inappropriate interactions with customers. The remaining three participants had no prior work, volunteer, or vocational training experience. The participants’ caregivers provided any available documentation on the participants’ diagnoses, individual education plans, and intellectual assessments. Table 1 displays demographic and assessment information for each participant.

Table 1 Participant information
Full size table

The assessment was conducted in therapy rooms at a university-based clinic. Each room had a one-way observation window, a desk, chairs, and a camera installed in the upper corner of the room. The designated “office work room” was 9 m by 3.5 m and contained a desktop computer, a printer, a small table, chairs, a small filing cabinet, and other materials necessary to conduct the sessions (e.g., office supplies, stapler, shredder). The designated “stocking room” was 11 m by 5 m and contained a 81-cm by 145-cm cabinet with six shelves, two small tables, chairs, and a variety of games and toys or grocery items (e.g., canned goods, pasta boxes) placed throughout the room (on floors and tables). The designated “break room” was 12 m by 5 m and contained a couch, chairs, table, decorative plants, and wall clock. The designated “supervisor’s office” contained a desk and two office chairs. All sessions were video recorded for data collection purposes. As part of the informed consent process, participants were told that we were evaluating their job skills and that all sessions would be video recorded.

Response Measurement and Reliability

The dependent variables targeted in this study were based on the results of studies that evaluated on-the-job behavior of employees with developmental disabilities (Butterworth & Strauch, 1994; Cheney & Foss, 1984; Greenspan & Shoultz, 1981), surveys of employers (e.g., Foss & Peterson, 1981; Ju, Zhang, & Pacha, 2012; McConaughy, Stowitschek, Salzberg, & Peatross, 1989; Salzberg, Agran, & Lignugaris-Kraft, 1986), commercially available assessment and curriculum guides (e.g., Montague & Lund, 2009; Partington & Mueller, 2015), and anecdotal information gathered from participants referred for our services regarding the reasons that they had been unsuccessful in keeping a job. We selected a variety of job-related social skills that included (a) making confirming statements when given a task, (b) asking for help with a task, (c) asking for help with missing or more materials, (d) responding to corrective feedback, and (e) notifying the supervisor of task completion. In addition, we collected data on two other behaviors that seem critical to job success (on-task and inappropriate behavior). Trained observers collected data using paper and pencil for all target behaviors. The measurement system was designed such that we could evaluate session-by-session data on performance. Such a system would reveal any changes in responding across the course of the assessment and provide the practitioner with a pretreatment baseline.

Confirming statements were defined as repeating parts of an instruction delivered by the supervisor, even if in a question form, to indicate that the participant understood the task (e.g., “You want me to sort by color.” “I should stuff all of these envelopes, right?” “Okay. Staple”). Data were collected by marking the occurrence or nonoccurrence of statements per opportunity. An opportunity for a confirming statement was recorded when the supervisor delivered an instruction to complete a task. No more than one opportunity to make a confirming statement occurred in most sessions; however, additional opportunities arose when the supervisor gave participants additional instructions in response to requests for help and when the participant completed a task earlier than anticipated, and the supervisor gave the participant more work to complete in that session. The data were converted to a percentage of opportunities by dividing the number of occurrences by the total number of opportunities and multiplying by 100.

Asking for help with a task was defined as any statements or questions that referred to an inability to complete the task or the need for more information to complete the task. Furthermore, we scored the accuracy of the following steps and converted the data to percentage of steps performed correctly: (a) asking for help within 1 min of off-task behavior or after no more than 5 min of problem solving without success, (b) knocking on the supervisor’s door, (c) waiting for the supervisor to invite entry, and (d) delivering a specific statement or question indicating that the participant needed help (e.g., “Sort by what?” “How do you want me to do that?” “I don’t understand” “I’m not good at _____”). The number of correct steps was divided by the total number of possible steps and multiplied by 100 to obtain a percentage for each session. If a participant did not go to the supervisor’s office to ask for help (i.e., the participant never asked for help, asked for help before the supervisor left the room, or waited until the supervisor re-entered the room before asking for help), steps (b) and (c) were scored as “no opportunity” and were not included in the calculation. If a participant did go to the supervisor’s office but did not knock on the door before entering, step (c) was scored as no opportunity and was not included in the calculation. No more than one opportunity to ask for help typically occurred in a single work session; however, additional opportunities arose when the participant needed help on different components of a task (e.g., help logging into a computer, followed by help in printing a document completed as part of the task).

Asking for materials was defined as statements indicating that the participant had run out of materials or was missing necessary materials. Furthermore, we scored the accuracy of the following steps and expressed the data as percentage of steps completed correctly per session: (a) asking for materials within 1 min of off-task behavior or after no more than 5 min of problem solving without success, (b) knocking on the supervisor’s door, (c) waiting for the supervisor to invite entry, and (d) making a clear and specific statement about the need for materials (e.g., “I need more napkins.” “Where is the vacuum?” “I ran out of staples”). Participants received partial “credit” (i.e., .5 instead of 1) for step (d) if the statement was nonspecific (e.g., “I need help.”). The response was scored as incorrect if the statement was unrelated to the task or failed to indicate that the participant needed assistance (e.g., “I’m done.”). As noted earlier, if a participant did not go to the supervisor’s office to ask for help, steps (b) and (c) were scored as no opportunity and were not included in the calculation. If a participant did go to the supervisor’s office but did not knock on the door before entering, step (c) was scored as no opportunity and was not included in the calculation.

Responding accurately to corrective feedback was defined as (a) acknowledging the supervisor’s presence (e.g., facing towards the supervisor); (b) delivering a statement indicating that the mistake would be corrected (e.g., “Okay, I will sort by color.”) without making inappropriate comments or facial expressions; and (c) correcting the mistake without inappropriate comments, complaints, or facial expressions. Each step was counted as correct or incorrect per opportunity, and the data were expressed as percentage of correct steps per session.

Correctly notifying the supervisor of task completion was defined as (a) notifying the supervisor of completion within 1 min of task completion, (b) knocking on the supervisor’s door, (c) waiting for the supervisor to invite entry, and (d) delivering a statement indicating that the task was complete (e.g., “I’m done.”). Each step was counted as correct or incorrect per opportunity and converted into a percentage of steps completed correctly per session. As noted earlier, if a participant did not go to the supervisor’s office, steps (b) and (c) were scored as no opportunity and were not included in the calculation. If a participant did go to the supervisor’s office but did not knock on the door before entering, step (c) was scored as no opportunity and was not included in the calculation.

On-task behavior was defined as orienting towards and manipulating materials in a manner needed to complete the task or attempting to complete the task (i.e., problem solving). A participant also was scored as on task if he or she checked the accuracy of a completed item no more than twice for the same item. A participant was considered off-task contingent on the absence of on-task behavior for at least 3 s or after 5 min of problem solving without success. Observers scored on-task behavior using 10-s whole-interval recording. An opportunity for on-task behavior was only scored if the participant was in the room, had task materials available, and had not yet completed the task and after the supervisor had left the room. The opportunity ended when the supervisor entered the room. The total number of 10-s intervals on-task was divided by the total number of 10-s intervals of opportunities and multiplied by 100 to generate the percentage of on-task behavior for each session.

Inappropriate behavior was defined as complaining (e.g., “I don’t like this.” “Why do I have to do this?” “This is too much work.”), asking inappropriate questions (e.g., “Do you have autism?” “What is your disability?”), cursing, talking out loud to oneself, and engaging in stereotypic behavior (e.g., repetitive movements or vocal responses that were unrelated to the context). Inappropriate behavior was recorded using 10-s partial-interval recording. Data on inappropriate behavior were converted into a percentage by dividing the total number of 10-s intervals with inappropriate behavior by the total session time and multiplying by 100.

Interobserver agreement (IOA) was calculated by having an independent observer score 25% of total opportunities for each dependent variable for each participant. To calculate interval-by-interval IOA for on-task and inappropriate behavior, the total number of intervals in which both observers were in exact agreement was divided by the number of intervals agreed plus the number of intervals disagreed, multiplied by 100 to get a percentage of exact IOA for each session. To calculate IOA for the dependent variables with multiple components (e.g., asking for help, notifying supervisor of task completion, responding to task feedback), the number of components with agreement was divided by the total number of agreement plus disagreements and multiplied by 100 to calculate the percentage of agreement for each session. To calculate IOA for confirming statements, the number of opportunities with agreement was divided by the total number of opportunities and multiplied by 100 to get a percentage of opportunities with agreement.

Mean agreement scores across all participants were 88.6% (range, 75.8 to 100%) for on-task behavior, 91% for inappropriate behavior (range, 86 to 100%), 87.5% (range, 75 to 100%) for asking for help with a task, 85% (range, 75 to 100%) for asking for help with materials, 93% (range, 82 to 100%) for responding to task feedback, 92.5% (range, 86 to 100%) for notification of task completion, and 88.5% (range, 82 to 100%) for confirming statements. Mean IOA data for individual participants are available from the first author upon request.

Procedure

Prior to the assessment, the experimenter met with the participant and his or her caregivers to identify work tasks that likely were or were not in the participant’s repertoire, inappropriate behavior and the condition(s) under which it occurred, and job preferences and goals. Copies of the participant’s individual education plans and other psychological reports and assessments were obtained if available. The experimenter gave the participant a general description of the assessment (i.e., “We will be giving you a number of work tasks to see how you do.”) and instructed the participant to dress and behave as though he or she was at an actual job site. All participants participated in a similar set of conditions, designed to assess and evaluate job-related social skills, along with other work-related skills (e.g., on-task behavior, task accuracy). The assessment was conducted across 3 days, typically separated by 1 week, for approximately 2 h each day, with the exception of the assessment for John. John’s assessment was conducted across 2 days, for approximately 3 h each day. Each day, we conducted approximately five to seven sessions, most lasting no less than 8 min and no more than 15 min. If a session lasted less than 8 min, data on on-task behavior or inappropriate behavior were not included in the analysis.

A female experimenter served as the “supervisor” for all conditions and gave the participants instructions and feedback throughout the assessment. After delivering the task instruction, the supervisor always said “Come find me if you need anything” or “I’ll be in my office if you need anything.” The supervisor showed the participants the location of the supervisor’s office (located around the corner and within 90 m of the workroom) prior to each day’s sessions. A few work sessions were conducted in a different workroom, and the supervisor was located in a waiting area within 45 m of this workroom. This location also was shown to the participant prior to the session.

Participants were alone in the workroom during each session, with some exceptions (see further description in the following), so that we could evaluate their performance when expected to work without the supervisor present. Conditions were arranged so that we had an opportunity to record all dependent measures at least three times during each assessment. For conditions that required the participant to seek the supervisor in her office (e.g., to indicate the task was finished, to ask for help), the supervisor asked “Why didn’t you come get me?” the first time that the participant failed to do so on each assessment day, regardless of whether the participant engaged in the correct statement after the supervisor entered the room. This latter question was included to ensure that a failure to seek out the supervisor was not solely a function of the unfamiliar and somewhat contrived nature of the setting.

Opportunities to record some dependent measures occurred in one or more conditions (see further explanation in the following). All participants were exposed to the following conditions: (a) clear instructions, (b) vague instructions, (c) missing materials, and (d) task not in repertoire. Some participants also were exposed to a condition during which the supervisor gave the participant multiple-step instructions (e.g., cleaning a room, completing unrelated office tasks). All assessments began with the clear instruction condition described in the following section. The order of the remaining conditions was randomly determined. In some cases, more than one condition was combined within a single work session (vague instructions and missing materials). Some participants were exposed to additional conditions based on information obtained during the preassessment interview or as part of brief evaluations of variables that might influence performance (see further description in the following). Tasks selected for the assessment included folding clothes, rolling silverware into fabric napkins, stapling paper, shredding paper, cutting out shapes, punching holes in paper, stocking items on shelves, alphabetizing books, cleaning a room, printing documents, composing and sending e-mail messages, creating a Power Point presentation, creating an Excel graph, sorting items (e.g., foam letters, paper, silverware), folding letters, and stuffing envelopes.

Clear Instructions

The supervisor gave the participant a task that the participant should have been able to complete accurately if given clear instructions, as indicated by information obtained prior to the assessment. The supervisor gave the participant the work materials, clearly described how to complete the task correctly, and modeled a correct response. The purpose of this condition was to assess on-task behavior, confirming statements, and notification of task completion under an “ideal” work condition. In addition, if the participant made any errors on the task, this would provide an opportunity for the participant to respond to corrective feedback. Once the participant completed the task, the supervisor waited at least 1 min to determine if the participant would notify the supervisor of completion. If the participant failed to do so, the supervisor entered the room, asked, “How are you doing?” and waited 5 s for the participant to respond. Regardless of the participant’s response, the supervisor indicated that it was time to start the next task.

Vague Instructions

The supervisor selected a task that the participant should have been able to complete accurately if given clear instructions; however, in the absence of specific instructions and/or a model, the participant could complete the task in more than one way (e.g., folding clothes, sorting items with multiple dimensions). The supervisor gave the participant all necessary materials to complete the task, along with very general instructions (e.g., “Sort.” “Fold the clothes.”) and without a model. The purpose of this condition was to assess whether the participant would request help with the task (e.g., “Sort by what?” “How do you want me to do this?”), and, if not, how the participant would respond to corrective feedback. If the participant did not ask for help before the supervisor checked his or her work, the supervisor provided corrective feedback (e.g., “That’s wrong.”), along with clear instructions (e.g., “I wanted you to sort by color, not by letter.”) and a model of the correct response. The session then continued for an additional 10 min (scored as a separate post-feedback session) to determine whether the participant would fix the errors or engage in inappropriate behavior (e.g., talking out loud angrily) after receiving feedback.

Missing Materials

The supervisor selected a task that was in the participant’s repertoire and delivered clear instructions. However, the supervisor did not provide a sufficient amount or all of the materials needed to complete the task. For example, the supervisor instructed the participant to staple 15 pieces of paper, but the stapler contained just ten staples; the supervisor instructed the participant to clean a room, including vacuuming, but a vacuum was not present in the room. The purpose of this condition was to assess whether the participant would ask the supervisor for help and/or for more materials. We also assessed on-task behavior during this condition, as well as the participant’s response to corrective feedback, in some cases, if a task was completed incorrectly. If the participant did not approach the supervisor within 1 min of off-task behavior or 5 min of problem solving unsuccessfully, the supervisor entered the room, asked, “How are you doing?” and waited 5 s for the participant to respond. If the participant failed to respond or responded incorrectly, the supervisor indicated that it was time to start the next task. If the participant responded correctly at any time during the session, the supervisor provided the missing materials.

Task Not in Repertoire

The supervisor selected a task that the participant would not be able to complete accurately, even with clear instructions, based on information received prior to the assessment (e.g., the participant did not have requisite skills to alphabetize books, create an excel graph, count money). The supervisor provided the necessary materials and clear instructions but did not model a correct response. The purpose of this condition was to assess whether the participant would ask the supervisor for help. We also assessed on-task behavior. If the participant did not approach the supervisor within 1 min of off-task behavior or after 5 min of problem solving unsuccessfully, the supervisor walked into the workroom, asked, “How’s it going?” and waited 5 s for the participant’s response. If the participant did not ask for help, the supervisor indicated that it was time to start the next task. If the participant asked for help at any time during the session, the supervisor provided the necessary help.

Multiple-Step Instructions

The supervisor instructed the participants to complete a sequence of tasks (e.g., staple paper, sort items, and stuff letters into envelopes). The instructions were given vocally only or combined with a written list. The supervisor did not model task completion. In some conditions, the supervisor told the participant to complete the tasks in a specified order. In other conditions, the supervisor told the participant to complete each task at a specific time, regardless of whether he or she had completed the prior task on the list. This latter condition always was conducted via a written list. The purpose of this condition was to evaluate the participants’ task performance when given multiple tasks and time monitoring requirements. In some cases, it provided opportunities to evaluate the participant’s response to feedback if the supervisor delivered feedback due to errors in the task completion.

Other

Peer Presence/Redirection

A research assistant pretending to be a peer was present in the workroom, typically during the clear instruction condition. The supervisor gave the peer either the same task as the participant or a different task, and they worked at the same table. The purpose of this condition was to evaluate the effects of a coworker’s presence on the participants’ on-task and inappropriate behavior. The peer reciprocated all conversation initiated by the participant and, in some cases (“redirection” condition), asked the participant to help him or her finish the task so that we could determine if the participant would interrupt his or her own work to help the peer.

Supervisor Presence

The supervisor was present in the workroom but pretended to be busy (e.g., looking through papers, sorting through materials for a different task). The supervisor did not initiate any conversation with the participant but responded briefly if the participant asked any questions or made any comments. The purpose of this condition was to determine if the supervisor’s presence impacted levels of on-task behavior or inappropriate behavior.

Time Pressure

The supervisor informed the participant that she needed a particular task to be completed “right away” and asked the participant to work “as quickly as possible.” In some cases, she also mentioned that she needed it completed within 5 min. This condition was always combined with clear instructions only. The purpose of this condition was to determine if this type of brief instruction would impact levels of on-task behavior or inappropriate behavior.

Results

Detailed results for two representative participants, John and Adam, are shown in Figs. 1, 2, 3, and 4. Data for the remaining participants are summarized in Tables 2, 3, and 4. Data on the primary job-related social skills are shown in Fig. 1 for John. As shown in the top left panel, John rarely made confirming statements when given instructions to complete a task (M = 26% of total opportunities and sessions). The four remaining panels on the left show session-by-session data on the percentage of correct responses when opportunities arose to ask for help with the task or materials, to notify the supervisor of task completion, and to respond to corrective feedback. The corresponding panels on the right show the mean percentage of correct responses for each targeted component of the social skills across all opportunities. John’s performance was quite variable when he encountered opportunities to ask for help with a task (i.e., when given vague instructions or a task that was not in his repertoire; second panels). When he did ask for help, which occurred on about 70% of the opportunities, John almost always asked for help immediately (before the supervisor left the room), and he made a clear statement on the majority of these opportunities. Because John never left the workroom when he asked for help with a task (he either asked immediately or waited until the supervisor returned), we had no opportunities to score the components of knocking on the door and waiting to enter. On the other hand, John consistently completed nearly all of the targeted skill components when he was missing materials or equipment or when he needed more materials (M = 91%; third panels). He always asked for help within 1 min off of-task behavior or 5 min of problem solving, knocked on the supervisor’s door, and waited to be invited entry. He made a clear statement about the problem on about 70% of opportunities. John also consistently notified the supervisor of task completion when he finished a task (M = 96%; fourth panels). On only one occasion, John did not seek the supervisor within 1 min of task completion, but he always knocked on the supervisor’s door, waited to be invited entry, and made a clear statement that he was finished with the task. In contrast, John engaged in moderate to low percentages of the targeted skill components when given corrective feedback (M = 54%). Although he always corrected his mistake, he oriented towards the supervisor while receiving the feedback on just 50% of opportunities and indicated that he would correct his mistake on just 50% of opportunities.

Fig. 1
figure 1

Mean percentage of opportunities with confirming statements, percentage of steps completed correctly when asking for help with a task or materials, and percentage of steps completed correctly when responding to corrective feedback during relevant assessment sessions for John (left panels). Mean percentage of opportunities with correct responses for each step when asking for help with a task or materials and when responding to corrective feedback during the assessment for John (right panels)

Full size image
Fig. 2
figure 2

Percentage of intervals with on-task behavior during each work session (top left panel), mean percentage of intervals with on-task behavior during each condition (top right panel), percentage of intervals with on-task behavior in the presence versus absence of a peer (bottom left panel), and percentage of on-task behavior and responses per minute with and without a time pressure statement (bottom right panel) for John

Full size image
Fig. 3
figure 3

Mean percentage of opportunities with confirming statements, percentage of steps completed correctly when asking for help with a task or materials, and percentage of steps completed correctly when responding to corrective feedback during relevant assessment sessions for Adam (left panels). Mean percentage of opportunities with correct responses for each step when asking for help with a task or materials and when responding to corrective feedback during the assessment for Adam (right panels)

Full size image
Fig. 4
figure 4

Percentage of intervals with on-task behavior during each work session (left panel) and mean percentage of intervals with on-task behavior during each condition (right panel) for Adam

Full size image
Table 2 Mean percentage of opportunities with correct responses for each step when asking for help with task or materials for the remaining participants during the assessment
Full size table
Table 3 Mean percentage of opportunities with correct responses for each step when responding to feedback and notifying supervisor of task completion for the remaining participants during the assessment
Full size table
Table 4 Mean percentage of intervals with on-task and inappropriate behavior and mean percentage of opportunities with confirming statements for the remaining participants during the assessment
Full size table

Figure 2 shows John’s results for the targeted work-related skills. On-task behavior (upper left panel) was fairly high during most sessions, averaging about 86% of intervals overall. On-task behavior was lower when given vague instructions (M = 70%; upper right panel). The middle left panel shows the levels of on-task behavior with a task that was in John’s repertoire (folding clothes) when a peer attempted to redirect John from his work by requesting his help and when John was working on the same task but in the absence of a peer. John’s on-task behavior decreased substantially in the peer redirection condition because he agreed to help the peer complete her work. This occurred even after the supervisor told him not to do so at the end of the first session of this condition. Because we also noticed that John worked inconsistently and very slowly on certain tasks, we evaluated the effects of a time pressure statement by having the supervisor tell John at the end of a session that she needed the work completed within the next 5 min and that he should work as quickly as possible on the same task in the next session. The time pressure statement was evaluated with two tasks that were in John’s repertoire (i.e., an envelope-stuffing task in the first two sessions and alphabetizing books in the second two sessions; see right middle panel). As shown in the figure, John’s on-task behavior increased to nearly 100% when given the time pressure statement and was noticeably lower on those tasks prior to receiving those statements. In addition to on-task behavior, we examined differences in the rate of task completion during these sessions. John completed those tasks more quickly when given the time pressure statement (M = .75 rpm) than when he was not given the statement (M = .2 rpm).

Results for the primary job-related social skills for Adam are shown in Fig. 3. Relative to John, Adam was more likely to make confirming statements when receiving instructions. Although he made confirming statements in just 36% of total opportunities, he engaged in this behavior during 50% of the sessions (top left panel). However, Adam’s performance was consistently poor when encountering opportunities to ask for help with a task or with materials. He did not complete any of the targeted components correctly during five of the six sessions when the supervisor gave him vague instructions or asked him to complete a task that was not in his repertoire (second left panel). In one session, he asked the supervisor to help him log on to the computer. Thus, overall, he sought help from the supervisor and made a correct statement about the problem in just 17% of opportunities (second right panel). Because Adam either never asked for help or (on one occasion) asked for help before the supervisor left the room, we had no opportunities to score the components of knocking on the door and waiting to enter. Adam engaged in low to moderate percentages of correct component steps when he needed help with more materials (M = 32%; third left panel). He asked for help within 1 min of off-task behavior or 5 min of problem solving unsuccessfully on 33% of opportunities and delivered a clear statement about the problem on 44% of opportunities (third right panel). When he did seek the supervisor in her office, he never knocked on the door. Adam’s performance was somewhat better when he finished with a task, performing an average of 54% of the component steps correctly (fourth left panel). He notified the supervisor of task completion within 1 min on 80% of opportunities and always made a clear statement indicating that he was done with the task (fourth right panel). However, he never knocked on the door when seeking the supervisor, which lowered his overall percentage of components completed accurately. Adam responded to task feedback with an average of about 70% of steps completed correctly (bottom left panel). In contrast to John, he acknowledged the supervisor on 80% of opportunities and delivered a correct statement on 70% of opportunities (bottom right panel). However, unlike John, Adam only corrected his work as instructed on about 30% of opportunities. He failed to follow any of the steps in one session during which he got feedback twice from the supervisor on a cleaning task. On-task behavior was quite variable (see Fig. 4), averaging about 75% of intervals across conditions, with much lower levels when given tasks that were not in his repertoire (M = 41.5%).

Results for the remaining six participants are summarized in Tables 2, 3, and 4, expressed as means across opportunities and sessions. The primary job-related social skills are shown in Tables 2 and 3, with the exception of confirming statements, which are displayed in Table 4. Similar to the findings for John and Adam, the remaining participants were generally less likely to ask for help with a task in a timely manner (M = 44%; range, 21 to 86%) than to ask for help with materials in a timely manner (M = 77%; range, 50 to 100%) or to notify the supervisor of task completion in a timely manner (M = 74%; range, 50 to 100%). When the participants did seek the supervisor in her office, performance regarding knocking and waiting to enter was quite variable among the participants. The percentage of opportunities in which the participants engaged in a correct statement when asking for help with a task or materials or notifying the supervisor of task completion also varied among the participants (M = 46%; range, 21 to 86% [task]; M = 60%; range, 33 to 100% [materials]; M = 76%; range, 50 to 100% [task completion]). However, with the exception of Hershel, the participants’ use of correct statements was higher when informing the supervisor of task completion than when asking for help with the task or materials. All of the participants except Hershel and Gabe acknowledged the supervisor every time (or nearly every time) that she provided corrective feedback. Only Rick provided a correct statement following feedback on every opportunity. The remainder of the participants engaged in a correct statement during a fairly low percentage of opportunities (M = 15%, range, 0 to 25%). Similarly, only Rick corrected his work as instructed on every opportunity. However, the remaining participants did so for a substantial proportion of the opportunities (M = 63%, range, 0 to 92%) with the exception of Hershel.

Like John and Adam, the majority of participants responded to a low or moderate percentage of new task instructions with a full confirming statement, as shown in Table 4 (M = 34%; range, 8 to 75%). With the exception of Hershel, the participants also remained on task for a high percentage of intervals (M = 80; range, 57 to 98%) and engaged in low levels of inappropriate behavior (M = .07%; range, 0 to 25%). Inappropriate behavior observed during the assessment included motor stereotypy (Hershel), inappropriate questions (e.g., “Do you have autism?” Carol), and other types of inappropriate vocalizations (e.g., mumbling, complaining, swearing; Daryl, Glenn, Gabe). We conducted additional analyses of variables that might impact on-task behavior for Hershel, Daryl, and Gabe because they engaged in the lowest levels of on-task behavior and/or the highest levels of inappropriate behavior. The mere presence of the supervisor in the workroom was associated with an increase in on-task behavior (from a mean of 57% to a mean of 84%) and a decrease in stereotypic behavior (from a mean of 25% to a mean of 0.8%) for Hershel during a controlled evaluation of this variable. On the other hand, this variable did not appear to impact the level of Daryl’s on-task behavior. For Gabe, who engaged in high levels of on-task behavior but moderate levels of inappropriate verbalizations while working (self-talk that included swearing, mumbling, and echolalia after the supervisor left the room; e.g., “Sort? Why the heck do I sort these?”), we evaluated the presence of a peer in the workroom. The mere presence of the peer was associated with a decrease in inappropriate vocalizations (from a mean of 15% to a mean of 1%) with no change in his high levels of on-task behavior. We also evaluated the effects of peer presence on on-task behavior for Carol, due to reports from her parents that she might get distracted by socializing with others on the job. Although Carol did engage in conversation with the peer, her levels of on-task behavior remained high.

Discussion

Results of this assessment contribute to our knowledge by providing data on the work-related social skills of adolescents and adults with ASD and by describing a promising approach for assessing them objectively in a clinic-based setting. Behavior analytic approaches to assessment emphasize the use of direct observation and measurement of skills due to potential problems with reliability and validity of indirect assessment. As noted previously, however, the behavior analyst might encounter numerous barriers to direct observation of social skills on actual job sites. One main purpose of this project was to evaluate the use of a clinic-based assessment. Our results suggest that it is useful for identifying potential targets for individuals with ASD who have trouble obtaining and maintaining employment. Most of the participants showed difficulties with one or more job-related social skills that likely would be important for success in competitive employment. However, further research is needed to determine if the social skills that the participants demonstrated (or failed to demonstrate) during the assessment would be similar to those demonstrated on actual job sites.

One potential limitation of this study was the somewhat contrived nature of the work sessions and use of a clinic setting. Although we attempted to arrange the environment so that it resembled a work setting as much as possible, the participants were aware that we were evaluating their skills in some way. Reactivity effects thus may have impacted the outcomes. No such effects occurred to our knowledge, and the session-by-session data on responding also revealed few changes in performance that would be indicative of initial reactivity. However, it should be noted that our arrangement—use of rooms with video monitoring and one-way observation windows—may limit its practicality for behavior analysts who do not have access to these facilities. Further research should evaluate this assessment in settings without dedicated session rooms or video recording equipment.

Questions also remain about the social validity of our targeted social skills and goals. Collecting normative data on job-related social skills in the workplace (e.g., how typical employees respond to corrective feedback), comparing the social skills of employees who are considered successful versus unsuccessful, and soliciting employers’ opinions of the targeted skills would provide useful information to guide further refinements of this assessment. Additional targets (e.g., eye contact when interacting with the supervisor; offers to help; problem solving when the supervisor is unavailable) and work conditions also might be evaluated in further research on this assessment approach. For example, information provided by vocational counselors and clients indicates that employees may encounter other problematic situations on the job not captured by this assessment, such as broken or malfunctioning equipment, conflicting instructions from different supervisors, and vague or inconsistent feedback from supervisors.

Finally, we did not collect data on the experimenter’s procedural integrity when conducting the assessment or include treatment data for the participants who completed the assessment. Several of the participants in this study received intervention services targeting skill deficits identified via the assessment, and we are currently conducted a more controlled evaluation of this intervention package with additional participants. Based on our initial assessment and intervention outcomes, we are able to provide the following recommendations for practitioners.

First, we recommend that practitioners conduct a controlled assessment of job-related social skills in either clinic or vocational (if available) settings as part of the initial planning process for clients seeking employment services. Practitioners should include the “standard” evocative situations described here, along with individualized situations based on the client’s desired vocation (e.g., providing customer service) and reports or observations of problematic situations (e.g., becoming distracted by coworkers), as illustrated in this paper. For example, if a client’s vocational goal is to work as a cashier in a store, the practitioner could set up a simulated sales situation in which the practitioner pretends to be a customer and determines if the client engages in the social skills important for this position (e.g., greets the customer, asks if he or she found everything needed, thanks the customer). If caregivers or former employers report that the client had difficulty when a customer complained, the practitioner could set up a simulated customer service interaction that incorporates this evocative situation.

When combined with assessments of job-specific skills (i.e., skills needed to perform the assigned tasks or responsibilities of a desired job with limited assistance), results should provide guidance regarding an individual’s readiness for competitive employment and the level of support and intervention services that would be needed prior to or following placement in competitive employment. For example, practitioners of clients like Rick, who performed well on nearly all of our measures of job-related social skills, might provide short-term training on a few of the client’s less proficient skills while the client actively searches for or starts competitive employment. Alternatively, clients like Hershel would likely benefit from more intensive intervention prior to placement in competitive employment. Practitioners should prioritize skills that seem highly important to success on the job, such as asking for help when needed and responding to feedback appropriately.

Second, practitioners might employ this assessment to evaluate the outcomes of training programs (e.g., internships) by conducting the assessment prior to the training experience and then again at the conclusion of the experience or at periodic intervals (e.g., every 3 months) while the client is participating in on-going training. This approach would provide an objective way to monitor and assess the outcomes of programs intended to improve job-related social skills.

Finally, we recommend that practitioners employ behavior skill training (BST), consisting of instructions, modeling, role play, practice with feedback, and reinforcement, to teach the relevant social skills to clients. Studies have demonstrated the effectiveness of BST for teaching skills to individuals with ASD (e.g., Leaf et al., 2012; Palmen et al., 2012) and for teaching job-related skills to individuals with other disabilities (Gear, Bobzien, Judge, & Raver, 2011; Whang, Fawcett, & Mathews, 1984). Our preliminary clinical work further suggests that stimulus prompts (e.g., text prompts) may help promote generalization of the skills from the training location to the job site. Practitioners also should consult reviews of research on interventions for individuals with ASD when designing programs to target job-related social skills (e.g., Camargo et al., 2014; Palmen et al., 2012; Seaman & Cannella-Malone, 2016).

In summary, behavior analysts serving youth and adults with ASD and related disorders may find this type of assessment beneficial for evaluating job-related social skills via direct observation measures. These data could guide vocational preparation and programming for clients and their caregivers by identifying the client’s strengths and potential areas in need of improvement. In light of the poor employment outcomes for individuals with ASD, the field would greatly benefit from more research and practice focusing on skills critical to success on the job.