Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

“Pan, who and what art thou?” he cried huskily.

“I’m youth, I’m joy,” Peter answered at a venture, “I’m a little bird that has broken out of the egg.” – J. M. Barrie from Peter Pan (Barrie, 1928)

Adolescence is a period of transition marked by both joys and trials. As adolescents grow and parents relinquish control, their once-sheltered world opens up and they are like birds breaking free of their shells and seeing the bright skies. However, the skies are not without clouds, the world is full of temptations, and young people face intense challenges (Wagner, 1996). Adolescents’ increased access to adult privileges, such as autonomy, status, and material resources, comes into conflict with their cognitive, psychological, physiological, and social changes (Sentse, Dijkstra, Lindenberg, Ormel, & Veenstra, 2010). As a result, some youngsters may not know where to go and stray from a healthy path, without understanding the dangers therein. Research suggests that at the beginning of adolescence, youngsters are vulnerable to the engagement of risk or problem behaviors, such as youth violence, substance abuse, and high-risk sexual behavior (Agnew, 2003; Biglan, Brennan, Foster, & Holder, 2004). To tackle adolescent developmental problems, a growing number of prevention and positive youth development programs have been designed specifically for budding adults. The confidence in these interventions is based on empirical evidence from studies conducted with rigorous methodologies. However, a review of the literature shows that while there is an abundance of related programs in the West, very few programs exist in the Chinese culture, with the possible exception of the Project P.A.T.H.S. (Catalano et al., 2012).

Over the last two decades, great attention has been devoted to the development of school-based prevention and youth development programs targeted at adolescent developmental problems, particularly in the Western context. Significant increases in the number and breadth of evidence-based programs are apparent (Gottfredson & Wilson, 2003; Payne & Eckert, 2010). Given this surge of programs, private and public funders, practitioners, and recipients require greater accountability and do so by asking the following questions: “What evidence-based programs work?”, “What is the essence of good prevention programs?” (Hardcastle, Blake, & Hagger, 2012). Focused on answering the first question, traditional evaluation studies on adolescent prevention programs have been based primarily on objective outcome evaluation (e.g., Shek & Yu, 2012) with program implementation (i.e., what a program consists of in its deliverance) overlooked. Dane and Schneider (1998) examined mental health prevention studies published between 1980 and 1994 and found that only 24 % of the studies had described steps about documenting program implementation. Similarly, Durlak (1997) reported that less than 5 % of over 1,200 published prevention studies had investigated the effects of implementation on outcomes, and Domitrovich and Greenberg (2000) found merely 13 of 32 reviewed mental health prevention programs which conducted analyses relating implementation to outcomes.

In recent years, researchers have attempted to place more emphasis on the assessment of program implementation (Dane & Schneider, 1998; Durlak & DuPre, 2008). Implementation is multifaceted and can be measured in numerous ways. These include fidelity (the extent to which the program is implemented in correspondence to its original intended design), dosage (how much the program has been delivered), quality (how well the components are delivered), participant responsiveness (attentiveness of program recipients), program differentiation (uniqueness of the program as distinguished from its counterparts), monitoring (observing the nature and amount of service received by participants), program reach (involvement rate and representativeness of service recipients), and adaptation (alterations made during implementation as compared to the original program). These eight dimensions vary across programs and each of them has been demonstrated to influence program outcomes.

Among the eight factors proposed by Durlak and DuPre (2008), program dosage was highlighted as one that deserves attention. Dosage refers to the quantity or amount of the program delivered. It can be operationalized by the number of sessions, session lengths, spacing of sessions, or the overall duration of the program. The principle of sufficient dosage highlights the need for participants to be exposed sufficiently to the intervention for them to have the desired effect (Nation et al., 2003). An extensive review of 162 studies found that only 6 studies reported dosage effects on program outcome. It was reiterated that more research into this issue is warranted (Dane & Schneider, 1998). But how much is sufficient? Is more necessarily better? The answers to these questions remain thorny, as the relation between dosage and program effectiveness is inconclusive. Some empirical studies suggest a positive dose-effect relationship. For instance, in their evaluation of two school-based drug prevention programs targeted at elementary and middle school students, Ferrer-Wreder and her colleagues (2010) found that a higher program dosage level resulted in an improvement of students’ assertive skills, anxiety management skills, and a decrease in drug use intentions. Similarly, in Valentine, Gottlieb, Keel, Griffith, and Ruthhazer’s (1998) evaluation of a substance abuse prevention program, the benefits of increased program exposure on social coping and school performance, especially for high-risk secondary school students, were also identified. Charlebois, Brendgen, Vitaro, Normandeau, and Boudreau (2004) examined the effects of dosage on the post-intervention academic performance and behaviors of disruptive young boys who attended a 3-year school-based prevention program focused on improving their social skills. Findings from the study revealed a positive dosage effect – participants who attended more program sessions were rated as more likable by their peers, and their post-intervention academic performance was also enhanced. Different reasons have been accounted for the observed positive dosage effect. First, the opportunity for repetitive practicing of skills increases with the number of sessions participants attend. Setting routines for repeated practice facilitates the nurturance of habits and repetition may lead to the automaticity of certain skills (Duke, Simmons, & Cash, 2009). Second, students are often encouraged by program implementers to attempt and persevere to complete assigned tasks during program sessions (Charlebois et al., 2004). As program sessions increase, the amount of tasks participants are able to accomplish are likely to increase accordingly. As a result, students may gain a sense of achievement that serves as an intrinsic reward. This sense of efficacy may, in turn, act as a motivation for positive behaviors. Besides acquisition of skills, it can be argued that some sessions of a program can help the program participants to develop positive attitudes and values.

On the other hand, there are studies which do not support the relationship between dosage and program effectiveness. For example, a review of 130 secondary prevention mental health interventions targeted at children and adolescents with presenting internalized or externalized problem behaviors, adjustment problems, poor peer relations, and low levels of academic performance showed that the effectiveness of programs did not depend on dosage (Durlak & Wells, 1998). Given these inconsistent findings, more research on the dose-effect relationship would be useful for future applications of prevention and positive youth development programs on how much exposure is needed to achieve certain intervention goals.

Shek and Sun (2012) reported subjective outcome evaluation findings based on the perspective of the participants of nine datasets collected from 2005 to 2009 (N = 206,313 program participants). The overall profile showed that the participants generally had positive perceptions of the program, implementers, and benefits of the program. Adopting the same dataset, we focused on examining whether students’ perceptions of the program content, implementers, or effectiveness would differ between those who participated in the 10-h core program and the 20-h full program in this study.

Methods

Participants and Procedures

From 2005 to 2009, a total of 713 schools (i.e., Secondary 1 level, 330 schools; Secondary 2 level, 250 schools; and Secondary 3 level, 133 schools) joined the 10-h program, and 614 schools (Secondary 1 level, 339 schools; Secondary 2 level, 193 schools; and Secondary 3 level, 82 schools) joined the 20-h program. Altogether, 223,101 students participated in the Tier 1 Program across the 5 years (Table 1). Upon completion of the Tier 1 Program, students were invited to respond to the Subjective Outcome Evaluation Form for Students (Form A) developed by the first author. For the 10-h program, 111,696 questionnaires were completed, and for the 20-h program, 94,640 questionnaires were completed. A 92.48 % overall response rate was achieved.

Table 1 Description of data characteristic from 2005 to 2009
Full size table

Data collection was conducted during the last session of the program (Shek & Sun, 2012). Form A (Ma & Shek, 2010; Shek & Ma, 2007; Shek & Sun, 2008) was used to measure program participants’ perceptions of the Tier 1 Program. Broadly speaking, this evaluation form consisted of several sections: (a) participants’ perceptions of the program (10 items), (b) participants’ perceptions of the implementers (10 items), (c) participants’ perceptions of the effectiveness of the program (16 items), (d) the extent to which the participants would recommend the program to other people with similar needs or join the program again (2 items), (e) overall satisfaction with the program (1 item), (f) things that the participants learned from the program and appreciated most (open-ended questions), and (g) opinions about the implementers and areas that require improvement (open-ended questions).

Results

Quantitative findings based on the closed-ended questions are presented in this chapter. Reliability analysis with the schools as the unit of analyses showed that Form A was internally consistent (Table 2): 10 items related to the program (α = .98), 10 items related to the implementers (α = .99), 16 items related to the benefits (α = 1.00), and the overall 36 items measuring program effectiveness (α = .99). First, participants generally had positive perceptions of the program in both the 10-h and the 20-h programs (Table 3). For instance, participants in both types of program perceived that the objectives of the curriculum were clear (10-h program, 83.32 %, and 20-h program, 83.64 %), the teaching activities were well planned (10-h program, 81.43 %, and 20-h program 81.78 %), and that there was much interaction among students (10-h program, 81.44 %, and 20-h program, 81.57 %). Second, a high proportion of the participants had positive evaluation of the implementers’ performance in both dosages of implementation (Table 4). Particularly, the participants thought that the implementers were highly involved (10-h program, 88.20 %, and 20-h program, 88.98 %), were ready to help them when needed (10-h program: 87.76 % and 20-h program: 88.49 %), and encouraged them to participate in activities (10-h program, 87.73 %, and 20-h program, 88.52 %). Third, as shown in Table 5, participants in both dosages of implementation perceived that the program promoted their development in different domains, including their moral competence (10-h program, 84.22 %, and 20-h program, 84.12 %), competence in making sensible and wise choices (10-h program, 82.86 %, and 20-h program, 83.13 %), ability to resist harmful influences (10-h program, 82.67 %, and 20-h program, 82.56 %), and overall development (10-h program, 83.18 %, and 20-h program, 83.30 %).

Table 2 Means, standard deviations, Cronbach’s alphas, and mean of inter-item correlations among the variables by dosage of implementation
Full size table
Table 3 Summary of the students’ perception toward the program
Full size table
Table 4 Summary of the students’ perception toward the performance of program implementers
Full size table
Table 5 Summary of the students’ perception toward the program effectiveness
Full size table

To examine the differences in the subjective outcome measures (i.e., program content, program implementers, and program effectiveness) across the two dosages of implementation, a series of one-way ANOVAs were conducted with the respective outcome measures as dependent variables and program dosage (i.e., 10 h vs. 20 h) as the independent variable. Results revealed that participants’ view toward the program implementers differed significantly between the 10- and 20-h programs, F(1, 1,322) = 9.53, p < .01. Specifically, participants had a more positive view (M = 4.62, SD = .30) toward the program implementers in the 20-h program compared to those in the 10-h program (M = 4.57, SD = .31). However, no significant differences were found in participants’ view toward program content and program effectiveness. Regardless of whether respondents participated in the 10- or 20-h program, their perceptions toward the program and its effectiveness were similar.

Discussion

Generally speaking, the present study showed that the program was well received by participants in both the 10- and 20-h programs. For instance, in terms of program content, the majority of the respondents stated that the objectives of the curriculum were clear and well-designed, and the activities were carefully planned. Furthermore, students were highly interested in and enthusiastic about the program, as they reported that they participated actively during lessons. In terms of program effectiveness, students across both dosages of implementation indicated that the program has helped to strengthen their competencies across a wide array of domains: cognitively, psychologically, and socially. Specifically, regardless of whether they participated in the 10- or 20-h program, upon completion of all units, students stated that their cognitive competence was strengthened and that they were more able to make wise and sensible choices. In addition, their self-confidence was reinforced, and they reported that they would be more resilient in future encounters of adverse conditions. Socially, the program has helped to increase participants’ sense of compassion and care about others and the community while successfully promoting students’ sense of civic responsibility. The findings are generally consistent with those reported in Shek and Sun (2012).

An interesting finding from the results is the significant difference in students’ perceptions of the program implementers across the two levels of program dosage. There are several explanations for this finding. First, students who participated in the 20-h program had the opportunity to spend more time with the program implementers over the span of the school term, as opposed to those who participated in the 10-h program. The above observation may be explained in terms of mere exposure effect (Gurung & Burns, 2011) which is applicable to the educational setting. According to Zajonc (1968), mere exposure effect refers to the fact that the more an individual sees someone, the more he/she will like that person. Moreland and Topolinski (2010) explained that as someone becomes more familiar with another person, he/she will perceive that person as being more similar to oneself and will simultaneously feel that the person is more likeable.

Second, as self-reflection and sharing are emphasized in the Project P.A.T.H.S., such elements would be stronger in programs with higher dosage. With more self-reflection and sharing, the program participants may like the implementers more. A comprehensive study involving the observations of over 800 classrooms across the United States found that levels of student engagement and positive interaction among peers were higher and more frequently observed in classrooms that were rated high on emotional and instructional support. Instructors in these classrooms were also responsive to students and promoted positive social interactions. Students and instructors under such circumstances engaged in high-quality conversational exchanges (National Institute of Child Health and Human Development and Early Child Care Research Network, 2002). Van de Grift (2007) also found that a safe and stimulating climate was positively related to student engagement. Indeed, students need to feel that they are in an environment that is safe for sharing, one that they will not be judged by others, particularly, the program implementers.

Third, adopting the perspective of attachment theories (Bowlby, 1969), programs with high dosage may help to strengthen the relationship between a student and a teacher which is analogous to the relationship between a child and his/her caregiver. It is argued that secure and reciprocal attachments are important for students to engage in their relationships with teachers and to develop a healthy self-concept and a sense of well-being (Cornelius-White, 2007). Besides, a secure and reciprocal relationship between program implementers and participants is characterized by positive interdependency, similar to the notion of cooperative learning which would contribute to the healthy development of program participants. These attributes would then result in more positive perceptions of the program implementers.

The above attachment perspective is also found in the contemporary learner-centered approach. The learner-centered approach demands that students be responsible for setting their own learning objectives and be intrinsically motivated toward achieving these objectives. Students are expected to be more engaged in the learning process (Duncan & Buskirk-Cohen, 2011). Cambourne (2002) proposed that learners are more likely to be engaged in demonstrations or activities that are free from anxiety. In addition, learners are also more engaged with activities that are conducted by someone they like, respect, admire, and trust. Likewise, we believe that in order for youngsters to be engaged in the activities and demonstrations pertinent to the development of the positive youth constructs and to be able to gain the most out of each designed unit, it is vital that learners (i.e., program participants, in our case) have a positive perception of the program implementers. It is undeniable that respect, admire, and trust are elements that take time to build in every relationship, and that between program implementers and participants is of no exception.

Finally, the dosage effect on participants’ perception of their program implementers can also be attributed to the increased opportunity for teachers and social workers to disclose themselves in the 20-h program as opposed to the 10-h program. Self-disclosure refers to “a teacher’s sharing of personal and professional information about himself or herself in a believable way” (Goldstein & Benassi, 1994, p. 212) and the revelation of teachers’ past experiences, stories, genuine feelings, and thoughts. When self-disclosure is utilized appropriately, whereby the teacher shares and demonstrates the attributes of empathy, genuineness, and respect, a safe relationship between teachers and students can then be fostered (Lau & Shek, 2010). A large part of the curriculum in the Project P.A.T.H.S. emphasizes self-disclosure of implementers to encourage sharing in the classroom. Program implementers in the 20-h program undoubtedly have more opportunities for self-disclosure when delivering the program elements, as compared to those in the 10-h program.

There are several limitations of the current study that should be noted when interpreting the results. First, in our present study, dosage was operationalized as the total number of program hours in which adolescents participated in the program. Yet, it is noteworthy that there are other methods of operationalization for dosage (e.g., session spacing, the length of each session). For instance, research on memory and learning has demonstrated the differential effects of spaced versus massed learning on memory (Ebbinghaus, 1964). Therefore, it is possible that the spacing of program sessions may influence program effectiveness. In order to gain a more comprehensive picture of the effects of dosage on program implementation, it would thus be useful for future studies to consider operationalizing dosage in alternative manners. Second, the use of self-report measure from the single perspective of students reveals only one side of the story. The process of program implementation involves both program implementers and participants; hence, the inclusion of program implementers’ views on the effect of dosage on program effectiveness would certainly illuminate the issue.

We believe that the question researchers ask should not merely be whether more is better. To implement a lengthy prevention or positive youth development program that is loosely developed would be meaningless. It may sound cliché, but the quality over quantity rule applies also to the evaluation of program effectiveness. Dosage is only one of the variables among the constellation of program implementation factors that must be considered. To conclude, a qualitative comment provided by one of the teachers who participated in the Project P.A.T.H.S. program implementers training workshop sums up this notion nicely,

“… Among the program implementers training workshops that I have attended in the past three years, I have gained the most from the workshop this year, because I now understand that adolescents’ developmental journey is one that we build collaboratively. Teaching a class of one hour is not an easy task, but you [the program trainers] have given us 20 hours of training in three days’ time. We go on to teach our students for another 20 hours; I believe it [this teaching] goes as 20.20.20… and continues on. I hope that our children will continue to grow and develop. I will not think about the outcomes at this moment, but in my opinion, this Program [the Project P.A.T.H.S.] is indispensable.”