Keywords

Introduction

The chapter has two objectives: First, to describe a logic model or template for evaluating a community demonstration project, and second, to sketch one application of this logic model to a pilot project aimed at increasing civic and economic engagement in a sample of 18–29-year-olds through social networking and problem solving in an educational environment.

Community demonstration projects are accompanied by a host of methodological complexities. These include: selecting a sample, which may or may not have a comparison group; introducing an intervention to facilitate the development of participants potentially coming from vastly different social backgrounds and histories; taking measurements during the time period of the project in an attempt to tap into significant dimensions related to the project’s goals; or interpreting results in order to correct features for the present or future projects. Yet community demonstration projects are, perhaps, the richest source of information on facilitating positive change in segments in society, particularly in communities where significant social problems exist. As such, it is worth the risk and the challenge to try to understand important events and changes in those events, while they are happening in real-time.

The Template for Constructing and Evaluating a Social Demonstration Project

There are many types of logic models, as a brief exploration of Google entries for “Logic Models” will attest. What we offer here is a template that intentionally focuses on five basic features so as to be applicable to many situations for which community demonstration projects are being designed. Figure 1 presents an overview of these five features, and the text below explains the terms and interrelationships among them.

Fig. 1
figure 1

A Logic Model: Five Steps in the Evaluation of Community Demonstration Projects

Full size image

Let’s consider each of these interrelated steps. We began, first, with the identification of some social problem or capability that provides the goal(s) of the project, and raise questions about related issues (such as the value context in which the project operates, ethical constraints, assumptions that structure actions for a given cultural and physical environment, etc.). Objectives are operationally defined components of goals, and are then subdivided as necessary to specify the particular targets of the intervention. Evaluation determines the baseline nature of these targets.

Second, there is a series of activities that begins the actual demonstration project. These include determining what the project will do with its available resources (staff, client strengths and limitations, and the activities composing the intervention, other contextual matters that may facilitate or interfere with the progress of the project) with regard to the identified targets. These activities—also referred to as inputs—are measured in a standardized way, in order to characterize the specific details of the intervention for others.

Third, these activities are monitored over time , for corrective feedback, as needed. Evaluation also uses standardized tools and methods of analysis of the data, as far as possible, to provide the research community with clear communications among qualified users.

Fourth, outcomes of the intervention are assessed, and factored back into the objectives and goals of the project, to judge the nature of the changes in participants. Outcomes may be short term and pertinent to the specific participants, or long term and pertinent to the organizations and community in which the intervention took place. Short term outcomes may be used rapidly to provide information about participant actions, and aid in staying on track toward the project’s objectives. Long-term outcomes may be used with careful consideration to modify the future projects from this pilot study, and to provide feedback to cooperating organizations, and the public at large.

Fifth, evaluation continues throughout the life of the project to provide information to decision makers. For ongoing practice, this feedback offers whatever corrective in services are needed. For longer term project outcomes, the formal feedback (reports to funders, papers for fellow scientists, and other kinds of presentations to potential users of this information in new settings, etc.) provide the basic information of science on which our common knowledge base continues to grow.

Given the nature of many community demonstration projects, it is difficult to obtain a large population of participants in a classical experimental design, randomly assigned to experimental and control groups. Consequently, it is often the case that some form of time series analysis is performed to describe changes in the participants. While this leads to suggestive correlations and rich case descriptions, without randomization, it is difficult to make causal statements.

Next, we will present some further discussion of these five elements, which we hope will alert potential users to some issues and problems, as well as positive highlights of community demonstration projects.

A. Problems/Potentials leading to Goals, Objectives, and Targets

Goals are defined by the project initiators—ideally, involving as many interested and relevant stakeholders as possible—community agency staff, researchers, practitioners, and members of the public who may be affected by the project. Project goals will center on initiators’ observations of problems in need of correction, or potentials in need of fulfillment. Both avenues are important as there should be equal concern in developing and promoting strengths as well as resolving or diminishing deficits. From among the project initiators, a smaller group of interested and available parties—hereafter referred to as the Project Team—emerges. Outside specialists may be employed in a consultant capacity when specific skills are needed. Using problems as an extended example, the project team recognizes that addressing problems usually involves large-scale goals—for example “reducing poverty among citizens of x community.” Such a goal must be parsed into measurable statements of the objectives of what this project seeks to accomplish—for example, “to provide 100 new jobs requiring advanced skills and providing adequate compensation.” The particular targets involved in achieving a given objective such as this, may include such defined actions as: specific training for needed advanced skills; identifying firms or non-profit agencies that will offer a training program; securing adequate compensation that may be provided in stages as the employee gains sufficient skills for the job; and counseling and progress monitoring for project participants as a means of providing support for them as newcomers in this new employment setting. It is toward these targets that specific project interventions are directed. For example, a training program is constructed by project staff in conjunction with community resources, in which needed skills are taught. The participating firms or non-profit agencies may help construct the curricula for particular needed skills, ensuring a close fit between needs and training experiences.

The Project Team explores assumptions about the context of project goals. For example, persons living in poverty may be stereotyped as being unassertive on their own behalf. Such negative value judgments may make it difficult to see a project participant’s strengths that are present among limitations that are often a result of systemic inequality and thus beyond his or her individual control. Participants may have self-defeating self-concepts derived from dominant society’s stereotypical assumptions and long experiences with prejudice and discrimination. Staff of the project should be aware of the cultural mores and how these hamper progress toward program goals, and actively incorporate methods to counteract problematic stereotypes.

Other broad cultural assumptions may benefit the goals of the project, such as belief in the American Dream—where hard work results in rewards enabling a reasonably good life—so that offering some path toward that American Dream belief may motivate potential participants to explore the project as something that could benefit them. Wherever possible, we strive to work with positives and participant strengths to achieve goals and objectives, rather than trying to address participant limitations as such.

B. Project activities constitute the intervention

In a classical research model, after the experimental and control groups are chosen at random from a larger population and thus assumed to be equivalent on relevant dimensions, the experimental intervention is given only to the former, and denied to the latter. Both groups are then periodically observed to measure differences logically caused by the intervention, since both groups were “equivalent” to begin with. It is difficult to obtain such conditions in community demonstration research, and project teams often fall back on more qualitative or approximate methods.

One such approximate method involves a time series design in which participants are used as their own controls, by being measured before the intervention begins, and then again at several intervals during the intervention to observe relevant changes. This continues until some end point when the participant has reached a desired and stable condition or until the project is completed or terminated. This approach is also referred to as single-system design because we can evaluate a single individual’s or a single group’s progress over time. In the former case, for example, we can monitor and evaluate an individual’s changes in attitude toward the community as a long-term location for employment and raising a family. In the latter case, we can observe changes in the dynamics of a group as it goes through a collective problem solving experience, such as average level of cooperation, or developmental progress such as group members’ perceptions of other participants’ leadership skills, etc.

The intervention for each research/evaluation model may involve the same kinds of activities, but will differ by size depending on the number of participants who are involved. Regardless of the nature of the intervention, it should be carefully defined to allow for replication in other projects, as needed. A guiding activity formula necessary for all research/evaluation can be stated as “Who does what to whom under what conditions and to what degree.” In this formulation, the basic characteristics of the “who” must be determined in terms of the educational credentials and level of specific training required of project implementation staff to deliver the project’s unique interventions. Also to be determined is the degree of oversight and support assigned to these staff members, etc.

The “what” is more complex. It refers to the entire set of activities that constitutes an intervention regarding a specific target. It may be useful to distinguish the structural or environmental aspect of the intervention from its content aspect, which refers to both the nature and extent of the subject matter upon which the intervention focuses. Structurally, the intervention has to take place in some physical space—although electronic interventions are becoming more common and information may be delivered in a private location in a public setting or in a classroom setting. Describing these details is important if another researcher wants to replicate this project in any meaningful way.

The content of the intervention is often considered the most important component, as it should be when the nature of that content is specified as well. This includes the type and extent of information the participant was given, the materials or resources supplied, and the other supports offered to facilitate the participant using the intervention in his or her own life situation. Outlining specific program components potentially allows for the approximate replication of what occurred in the delivery of the project’s intervention. Approximation is highlighted here as it is more likely in the social sciences than in the physical or medical sciences that exact conditions may not be entirely replicable.

The “to whom” is another necessary aspect of intervention and evaluation. Minimally, this includes the target population’s age, gender, socioeconomic status, ethnicity, and other relevant psychosocial factors that might possibly influence the outcome. There is no hard and fast science of selecting these characteristics, and as such, it allows for some creativity in scale construction.

Monitoring over time

Repeated monitoring over the life of a project is usually more characteristic of time series designs than in classical research because the Project Team wants real time feedback so as to make corrective actions as needed. For monitoring to occur, it has to fit comfortably within the context of the whole project, rather than being a burdensome task imposed on participants on a repeated basis. For example, non-reactive measures—such as counts of attendance at classes, observations of verbal contributions in group setting, etc.,—can be made. On announced occasions, more reactive measures can also be taken—for example: participants’ votes on “most useful project suggestion;” or “members who seem highly trustworthy in the group.” Standard scale measures can also be implemented such as “program satisfaction” or “group morale.”

Measurement tools for monitoring need to be as carefully constructed as the tools used to measure the presenting problem or challenge, and the specific interventions taken in their regard. Some tools are objective in nature: counts of how many behaviors of a certain type are observed in the participant within a specified timeframe; measures of the size or content of the forces and structures operating for or against achieving program goals; etc. Other measures are subjective, as no one but the participant can describe internal thoughts, feelings, and psychological states. How descriptors of these subjective factors are obtained, however, represents the art of the evaluator. Asking a question with particular words, tone, or non-verbal cues may influence how the question is answered. Wherever possible, the evaluator seeks to use non-reactive measures, but we rarely know how a given wording will affect a given client. Social science at its best is a fragile art form.

Outcome evaluation and Reports

Upon completion of every project , it is necessary to produce a final evaluation of the outcomes. Scientific ethics requires fair impartial reporting of the facts—whether they support the effectiveness of the intervention or not – and reasonable interpretations of them, so as to build an empirical basis for further study and improvement toward the long-term goal. Failure, that is, reporting no change in the baseline condition or even deterioration from that baseline, is as important as reporting success. Both provide guidance for future project teams to determine what they might use in their own new studies. However, in the real world of promotion in academic environments where published papers are required, scientific ethics faces serious challenges since there is often only a bias towards publication of positive results. Let the reader beware.

There are several elements of conducting a demonstration project and reporting on its results that need amplifying. First, researchers recognize that whenever an intervention is introduced, it may require a training period for participants to become accustomed to the new elements in their everyday world. During the training period, there may not be immediate change, and the intervention may actually introduce barriers in habitual behavior that lead to decreased functioning for participants. As such, it may be necessary in a given project to recognize this training period during which the intervention becomes routinized in the participant’s life, and its full effects may be measured with this consideration in mind.

Second, client involvement in the whole range of program details is becoming more common. In one instance, after there appears to be positive change in the practitioner-dominated intervention, it may be useful to have a maintenance phase in which the machinery of intervention is placed in the hands of the participant—to the extent possible—so that he or she is temporarily responsible for sustaining the intervention in his or her life—as he or she would do at the end of the project. In the maintenance phase, however, data are collected as before, and the effect of having the client run the entire program for him or herself is still monitored. Should the results continue to be positive during this phase, they provide a second indicator of a potentially successful intervention, in as realistic a setting as possible. If the participant acknowledges this second stage of success, then the practitioner might have a stronger faith that the client is really able to live his or her life under his or her own steam.

A third element in research/evaluation is the follow-up , a post-intervention reconnection with the participant, using some abbreviated forms of the prior measurements, to determine the intervention’s sustainability in the client’s life. In time series studies, which often take place within a narrower time frame than classical research and in a more local setting, evaluators may reconnect with participants more often and demonstrate the viability of the client-controlled intervention, now a stable part of his or her normal life. Moreover, there is always the possibility to invite the participant who is not doing well back to receive further help, whereas research subjects usually receive no direct help from their participation in large-scale studies.

The final element, report writing consists of various kinds of reports, which are extremely vital in a number of ways. First, reports are written in language suitable for the diverse range of audiences, and serve to inform funders that their confidence in the research was well-founded. Second, scientists and practitioners recognize the need to build a collective scientific basis for basic understanding and for taking similar actions in related areas. Documentation for these efforts is made in reports, from standardized formulas to persuasive case studies. Third, but not least, these kinds of documents may filter down to the general public through various media sources, informing public opinion and dismantling harmful stereotypes in order to facilitate advocacy and action, while supporting decision makers in making strides to building a more constructive democracy.

E. Decision making based on reasonable research and evaluation

Decisions are made, based on a wide variety of factors , from the purely political, through the cultural (“what will the affected population accept by way of a new intervention?”), and the personal (“how do I interpret the data, regardless of what the experts say?”). All of these biases (and others) may be positive in the sense of a larger picture of leadership in a complex society where pushes and pulls come from many directions, but they may also be starkly negative (where the individual decision maker is profited, while the larger society is not).

It is possible to predict some of the potential effects of a large-scale decision on the many stakeholders. On the other hand, there are sometimes unexpected consequences. We speculate that the larger the project, the greater the likelihood of such unexpected and potentially unwelcomed consequences. Otherwise respected politicians may frustrate their own supporters whose view focuses on the limited issue, rather than weighing that decision against other competing ones. On the other hand, grass roots supporters may be closer to the realities of a decision than their leaders, which simply verify how complicated modern society is. Decisions are likely to be based on what we know, what we think we know, and how we weigh these elements against what we do not know. However, in a working democracy, we elect leaders, or people emerge as self-selected leaders to make decisions based on some sense of shared values, and we hope for the best, but prepare for the next election just in case, “Stay calm and carry on.”

The complexity of life in modern society can benefit from the inclusion of objective, empirically-based information by decision makers as part of the myriad factors that shape large-scale decisions in government and the for-profit and non-profit sectors, and constitutes democracy in action. Not to be excluded from this discussion is the project’s impact on the participants, whose lives are the focus directly affected by the project team’s efforts. By directly asking participants about their perspectives on the project’s implementation, confidence in the efficacy of an intervention can be informed and strengthened by the actual lived experience of those to whom it is targeted (Bloom & Britner, 2012). Client-centered evaluation may thus be an important corrective element and supportive measure for ethics in the social sciences, since participants are not only the people most affected by the given project, but are highly likely to be similar to those who may be provided the intervention in the future. There are few scientific terms to reflect this kind of information and the decisions made, in part, on them. What participants experience—positively or negatively—during an intervention is vital to any meaningful replication of the project, along with the formal report of “scientific” results.

Serve Here Connecticut Project (SHCT ): Practical Evaluation Strategies

This section of the chapter focuses on a discussion of program impact of the first year of the Serve Here CT Project, and a presentation of a preliminary, unpiloted research tool that was developed by the program’s evaluation subcommittee. Social capital was the driving concept, since its development involved many people who had to be able to “live” with the definition in doing their own specific chapters. This process of reiteration was directed by LaShaune Johnson, and will not be repeated here. The working definition listed below, informed the development of a questionnaire that might facilitate an assessment of program impact (2015–2016).

Social capital refers to the connections among individuals such that, over

  • time, a social network is created in which people come to expect mutual

  • support and trust. This leads to (a) potential increases in each individual’s

  • physical and social-emotional well-being, as well as, (b) potential increases

  • in civic engagement and employment in the community of which they are a part, both contributing to a healthier and more effectively functioning society(Johnson, 2016, in this volume).

How would a team of evaluators approach this definition as their guidelines for constructing a questionnaire to measure the impact of an intervention attainment of social capital in this project?

The term “social capital ” is an abstract theoretical term, which may be operationalized as (1) the connections among individuals, such that (2) over time, (3) a social network is created. The form of

  1. 1.

    Connection among individuals is not specified but it is assumed that the connections will emerge during the course of the project. As defined by the project staff, this means participation in 30 formal classroom meetings in the project year, which will be devoted to a common learning experience, as defined in part by the several chapters of this book, as well as some small group experiential learning projects—located in one or several of the participating non-profit organizations—that will emerge from participant interactions among themselves and the sponsoring organizations. We expect some ordinary group dynamics to occur, bringing the participants closer together in social and recreational ways. Participants’ classroom experiences will not be as passive students, but as members of an active learning community in which contributions from each participant will become part of the curriculum as well. The small group projects will emerge as participants individually develop possible projects and the group reaches a consensus as to which are most feasible at this time. If several projects are mutually selected, then participants will volunteer to be members of one such group and continue to develop the idea in small group meetings.

  2. 2.

    The phrase, “over time,” recognizes the likelihood that connections require interactions among participants, especially on shared ventures, such as the classroom learning and small group projects. This timing element is built into the project in several ways. First, the questionnaire given at the beginning of the project will be repeated over the year’s time. Annual follow-ups will be provided for each of the subsequent 4 years. This will require, secondly, the construction of questions that are sensitive to potential changes in respondents over time. Third, the form of analysis will follow the time series design for individual change. Measurement will focus on a number of questions and scales and the analysis will be at the individual participant level. This design uses each individual as his or her own control person, and compares changes in responses to similar questions over time.

  3. 3.

    The term “social network” is an important concept in the social capital literature. It has an honorable lineage, dating from Emile Durkheim and Georg Simmel in the late nineteenth and early twentieth centuries, through Peter Blau’s social exchange theory and Theodore Newcomb’s balance theory at mid-century, and it has continued to generate new perspectives into the twenty-first century. Definitions range from the technical—as in graph theory and social network analysis, stemming from Jacob Moreno’s early sociometric analyses of social groupings—to the more accessible sociological theories. It will be expedient to use one of the latter sociological perspectives in defining this term for the Serve Here CT Project:

A social network is one kind of social structure comprised of the interactions between and among actors, and which has the possibility of encouraging mutually beneficial outcomes for its members by means of the connections each has both within and outside the group. Over time, these interactions provide the basis of mutual trust and cooperation toward social and/or economic goals. Networks can vary as to the strength of ties among members. Social networks are also described as complex, emergent units, with few necessary structures common to all other networks, so that patterns used to describe them are relatively fluid.

Once a social network is formed—or is in the process of being formed—the process of interactions and positive reinforcements for such actions leads to the social structure itself. It should be noted that there is no final state of formation for this relatively loose kind of group. This social structure, or actualized social network, stems from the mutual benefit and trust that comes with the fulfillment of early stages of beneficial and trustworthy actions, and grows stronger with subsequent fulfillment of such actions.

The working definition of social capital then suggests that this social network will lead to two potential outcomes: First, an increase of participants’ physical and social-emotional well-being, and second, an increase in civic engagement and employment in the community of which they are a part. The potential increases in well-being, civic engagement and employment are seen as contributing to a healthier and more effectively functioning society. This statement provides some evaluation guidelines that we outline below:

Measures of participants’ physical and social-emotional well-being are not well developed in the scientific literature. We interpret these issues to be addressed with both familiar measures of physical health and some questions about how this state of physical health is related to the content and activities of the Project. In terms of social-emotional well-being, we interpret this issue to be a subjective assessment by participants on their life satisfaction in general, and some specific questions of possible changes in well-being related to their participation in the project with other young adults.

We note that there are positive and negative versions of each of these two outcomes, which offer another way to measure their development. In addition to the positive version of well-being, there are risk factors that stand to interfere with it, such as drug use, being involved in automobile accidents, depression, among others. We will illustrate these general and specific questions below.

Connected to these physical and social-emotional well-being items is another set of concerns that are potentially measureable, namely, how this social network contributes to a healthier and more effectively functioning society. It is not likely that such a small sample working for so short a period of time would have an appreciable effect on the health of society, but there may be a number of ways in which participants, individually and collectively, might help to make their communities function more effectively for fellow citizens. Each of the following examples of ways in which participants might function in their community may be measured over time: volunteering in tutoring children, assisting the disabled, helping the elderly in a variety of ways; serving in an array of community organizations on behalf of people, the environment, animals; being involved in advocacy for those less able to deal with local government and other organizations; voting and paying taxes, as compared to these activities before participation in the Project; making plans for advanced education (and/or reducing existing college debt), etc.

Next, we present our joint application form-and-evaluation form , termed A/E 2015–2016, which all applicants receive as the first step in the process of joining the Project. People completing the application questions will describe themselves and their relevant experiences in work or volunteer capacities. Based on these answers, cooperating non-profit organizations will invite selected persons to come in for interviews as they would ordinarily conduct them, and make their final determination of which applicants are most suitable for their available positions. Upon their selection, these persons (now employees) will be termed Project “participants,” and will be one study group in the Project.

Persons not accepted by the non-profits will be a second study group, termed “applicants.” They may reapply in another Project year. Applicants will serve as a control group for the participants enrolled in the first year of the project.

The second part of the application form contains the evaluation items, which are integrated into the application form, and which will be repeatedly administered at various times over the course of the 1 year of the Project. These items were drawn from various research studies, and will be described below. Other items and scales may be used in future Project years.

The general hypothesis for this study is that participants in the 30-week classroom learning experience, and the small group project at a participating organization, along with individual work experiences at these organizations, will build stronger “social capital,” both for their own benefit and for the benefit of the organization at which they work, as well as for the Project group of participants and the larger community. “Stronger social capital” will be operationalized by measuring the degree of social networking over time . In particular, we will consider (1) whether and how participants become helpful to each other over time in occupational career paths; (2) whether participants become effective workers in their organizations; (3) whether participants signal their intentions to obtain advanced education and/or reduce their existing college debt; (4) and whether participants signal that they intend to stay in the community (and state) as their long-term residence.

Given the nature of the small participant group and the manner of their selection, we will devote major attention to individual patterns of change over time . If any general patterns emerge, these will be reported in the form of hypotheses for future study in the later stages of this project. We will also present detailed case studies of participants that will describe different paths to the project, the experience of participating in the project, and project consequences

Overview of Application/Evaluation A/E 2015–2016 Questions

What follows is a brief overview of the questions that we plan to use in the quantitative evaluation that was discussed in the previous section. The actual questionnaire is appended to the end of this chapter. Many of the questions in the survey, especially those related to community satisfaction/belongingness and participation and were adapted from the Social Capital Community Benchmark Survey conducted by the Saguaro Seminar of Harvard University; this survey involved a national sample of 3,000 respondents and representative samples in 40 communities nationwide across 29 states (Social Capital Community Benchmark Survey, 2000). Psychosocial questions were adapted from specific scales as noted below.

Community Satisfaction/Belongingness : There is a group of 10 community questions (questions 4–13) that explore to what extent the participant’s community (self-defined) meets his/her needs, whether the participant feels like a member of that community, and whether people in that community can be trusted and are fair, etc. We hypothesize that over time, people in the Serve Here CT Project will gain more favorable attitudes towards their community.

Community Activity Participation : There is another group of 15 questions (questions 14–28) that asks participants about a variety of activities they might have been involved in, over the past 12 months. These include whether participants worked or volunteered in a community project; donated blood; voted; attended public meeting; attended cultural groups, support groups, or religious services; attended school or college, etc. All of these items show some kind of connection with the community, and presumably can change toward becoming more socially involved. We hypothesize that over time, people in the Serve Here CT project will spend more time on community activities listed in this section.

Another cluster of 6 questions (questions 29–34) refers to public affairs. These include items on voting, trusting the national or local governments, and what political label they would use to describe themselves. We will observe any changes in responses to these questions during the year of participation in the Project. Whatever the participant’s political leaning, we hypothesize that there will be greater interest in public affairs after involvement in the project.

Another cluster of 9 questions (questions 35–43)) asks about the challenges participants have faced in the last few years. These include questions on debt, college or otherwise; involvement in an auto accident; breakup of a close relationship; feeling depressed on the way life is going; experienced the death of a friend or family member; having health issues; or being a caregiver for others. Participants are asked about the specific challenge and whether the challenge is “still limiting.” We will be looking for whether these challenges have or have not been resolved. We hypothesize that participants will endorse fewer problems as “still limiting” after involvement in the project.

A group of 9 questions (questions 44–52) deal with participants’ feelings about their ability to accomplish things, to adjust in the face of challenge, and feelings of overall optimism. These include items like “I can solve difficult problems if I try hard enough,” “I can do about anything if I set my mind to it,” and “I have great faith in the future.” Some of these questions (items 44-47) were taken from the General Perceived Self-Efficacy Scale (Schwarzer & Jerusalem, 1995); several items (items 48-50) were taken from the Pearlin Mastery Scale (Pearlin & Schooler, 1978). Item 51 was taken from the Positivity Scale and item 52 was adapted from the same measure (Capara et al., 2012). We hypothesize that levels of agreement with these questions will increase as a result of participation in the program.

The measure ends with two questions about overall happiness as well as general health (questions 53 and 54). Again, we hypothesize that overall levels of happiness and health will improve as a result of participation in the program.

Our overall plan is to administer this questionnaire at admission to the program (or upon application) and then at 6 months, and finally, after completion of the first project year. Those who applied but were not eligible but not admitted (due to space limitations or other reasons), will be requested to provide two follow-up interviews during the same period. Analyses will be conducted using two group, repeated measures ANOVA to look at changes on individual questions well as on possible subscales that might be constructed from groups of questions. Group by time interactions (with a greater rate of improvement observed in the intervention group) will be examined. More sophisticated mixed model regression procedures will be employed in order to statistically control for possible differences between participants (e.g., gender) that might account for the variance in the changes as well as for the a possible number of follow-up interviews across cohorts. When analyses are concluded, individual case studies will then be drafted. In addition during the first year of the project, we will conduct semi-structured interviews with key informants and project stakeholders, including project staff, employers and policymakers in the State of Connecticut. We will also compile data on key benchmarks: educational attainment, employment status, and residential location (whether or not the participant remained in Connecticut)) within 1 year after the program completion.

Within 2 years after the beginning of the project (a year after the completion of the first cohort) a report will be generated and a summary of the results of the analyses as well as the key informant interviews will be presented to youth participants, policy makers, employer participants and other key change agents in the State of Connecticut. As experiences with the Project and its evaluation accumulate, the Project staff will be considering ways to improve the delivery of services and its measurement in the next stages of its incarnation. Therefore, this chapter is offered as a progress report as we begin the actual project.