Introduction

The role of evidence-based knowledge and research in informing immigration and settlement policy is an important but under-examined area of inquiry. Traditionally, research and its applications have been centralized within government bureaucracies, so the contributions of bodies outside government policy circles are less well known. Specifically, the transfer (knowledge transfer (KT)) and mobilization (knowledge mobilization (KM)) of evidence gleaned from social science research is an important source of knowledge from which policy and programming can both benefit. This paper seeks to explore one especially neglected dimension of this terrain, the role of non-governmental organizations (NGOs) in evidence/research-centred KT/KM. The issue is explored through a consideration of the case of CERIS—The Ontario Metropolis Centre.

CERIS was part of a unique initiative by the Government of Canada launched in 1995 to enhance evidence-based research about immigration that would inform state policy.Footnote 1 Recognizing the need for ongoing knowledge exchange, five Canadian research centres,Footnote 2 of which CERIS was one, were established at the behest of the federal government to promote policy-relevant research about immigration and settlement issues. The centres administered through the Social Sciences and Humanities Research Council (SSHRC) with major funding from SSHRC and Citizenship and Immigration Canada (CIC), and smaller amounts from other federal departments and provincial governments, made up the domestic core of the Canadian Metropolis Project, which also had an international component. The founding mandate of the Canadian Metropolis ProjectFootnote 3 was “to improve policies for managing migration and cultural diversity in major cities by:

  • Enhancing academic research capacity

  • Focusing academic research on critical policy questions, options and delivery mechanisms

  • Developing effective ways to use research in decision making” (Horizons 2002)

Subsequently, the project was also mandated to undertake pan-Canadian research and examine immigrants’ impacts on regional development outside big Canadian cities (Biles 2012).

The centres had three characteristics that were rare at the time of their development. Although they were mandated to promote policy-relevant research, the SSHRC administration safeguarded the independence of academic researchers who were free to propose research topics and methods and who controlled the use of their research findings, unlike contracted researchers. To ensure the relevance of the research and assist with the dissemination and use of research findings by federal government departments, the Metropolis Secretariat, charged with liaising with all centres, was established in Ottawa. Locally, each centre was also required to establish partnerships with representatives from local NGOs that are service providers and advocates for newcomer populations. Since each centre was to concentrate on issues in its own region, community partners provided critical familiarity with local immigration and settlement issues and crucial contacts to facilitate the research, its dissemination and its use.

The third and final phase of Metropolis Canada emphasized knowledge exchange; the focus was the creation, sharing and use of policy-relevant research about immigration and settlement issues in each region, by the immigrant-serving sector, representatives from all three levels of government and academic researchers. In Ontario, CERIS—The Ontario Metropolis Centre set out to foster knowledge that was of high quality and relevant to multiple stakeholders (Rummens 2013; Anisef et al. 2007). The contributions of this innovative partnership have been evaluated previously, mainly from the perspectives of policymakers from the federal and provincial governments (Shields and Evans 2008, 2012; Biles 2012; Biles et al. 2011). In this article, we concentrate on the role of NGOs in knowledge exchange.

Building partnerships between government and non-governmental actors has become increasingly important to modern governance (Young 2013; Levin 2008). While the roles of community-based organizations and the private sector in partnering with government for such things as service delivery are well examined, research partnerships between the state, academics and non-governmental actors have been neglected in academic research. Investigating this example allows us to better understand the wide range of relationships in which the state and civil society are now engaged. It may also signal government’s innovative attempts to strengthen areas of weakness, in particular with respect to enhancing policy capacity (Baskoy et al. 2011), and an expanded role for universities in building community partnerships (Martin 2010; Nichols et al. 2013). For their part, NGO partners saw the potential to gain greater voice in government policy circles and a chance to strengthen their research capacity. From its inception, the mandate of CERIS that emphasized the involvement of NGO and government partners in governance, research and knowledge dissemination responded to growing interest in enhancing the connections between policy and practice and research and evidence (Levin 2008, 4).

Our evaluation begins from the premise that knowledge/evidence has the potential to influence policy decisions.Footnote 4 In fact numerous evidence-based initiatives have been developed to promote more research-centred approaches to public policy. These include the Coalition for Evidence-Based Policy in the United States, the Centre for Evidence and Policy in the UK and, within the federal Canadian government, Policy Horizons Canada, formally known as the Policy Research Initiative (Young 2013, 6). The rapid growth of bodies engaged in evidence-centred public policy interventions is associated with the weakening of the state’s own internal policy capacity as a result of many years of government restraint which curbed the resources of state policy units (Baskoy et al. 2011; Evans and Shields 2014; Aucoin and Bakvis 2005). There is a strong motivation for the civil service to tap into external sources of policy-relevant evidence to fill policy gaps.

The proliferation of policy think tanks and research institutes, moreover, speaks to the desire on the part of many interests to use ‘evidence’ to inform and influence policy-making. Of course, governments must also be open to listening to evidence-based research with regard to how they make policy. In the contemporary period, there is a certain duality within government regarding what the dominant influences over policy determination are. As the former head of the Canadian public service, Mel Cappe has observed some governments “have privileged ideology and doctrine over evidence” and their interest in evidence rests in the direction of ‘policy-based evidence’, i.e. only the use of evidence that supports their predetermined ideologically based decisions (2013, xi). Other governments have been more ideologically neutral and have sought out evidence with the goal of determining what works. This approach, advocated in the UK by the Blair Government, informed the federal government’s initiation of the Metropolis Project in 1995–1996 (Shields and Evans 2012). The goal was to make reliable evidence a more important part of the debates and discussions about immigration and settlement policy (Levin 2013, 59). To achieve this goal, knowledge must be related to current policy issues and hold the possibility of identifying implementable policy alternatives. It is virtually impossible to draw a straight-line link between research and policy decisions (Bunker 1978; Campbell et al. 2007; Levin 2008, 8). Rather, ongoing knowledge exchange in which all participants contribute to the identification of research questions, their investigation, the interpretation and presentation of research findings and their dissemination is most likely to result in relevant knowledge and its actual utilization in policy-making (Armstrong and Alsop 2010).

Evidence is, of course, not neutral. In government, research and its conclusions are filtered through a political lens. Academics bring their own perspectives that shape the research. NGOs also have a distinct perspective that stems from their mandate to address the needs of the communities that they serve. For NGOs, evidence-producing endeavours are useful if they ensure the betterment of their clients by identifying unmet needs and gaps in services and by providing authoritative evidence that will strengthen funding applications. Research on immigration and settlement can help NGOs assess the effects of policy on immigrant communities and recognize policy areas in need of improvement. NGOs also have detailed and up-to-date information about the needs, perspectives and feelings of newcomers that is very valuable for academic researchers and government policymakers. As a result of their mandates and their close relationships with immigrants, NGOs engage in the “symbolic use of evidence” (Levin 2013, 59), meaning that community and practitioner perspectives inform their use of evidence. In knowledge mobilization and transfer, NGOs bring both perspectives into policy-engaged discourse (Laforest 2013, 162).

Our assessment is partial in two respects. We focus on the participation of NGOs, recognizing that each stakeholder in CERIS—The Ontario Metropolis Centre, particularly the federal government, had its own goals for the partnership. As well, this preliminary evaluation focuses on only one of many activities at CERIS—The Ontario Metropolis Centre, the Community Research Symposia that took place between 2008 and 2012. Through this assessment, we seek to evaluate the circumstances under which NGOs can participate in effective knowledge exchange that meets their own research needs in terms of programming and advocacy and contributes to progressive, evidence-based policy-making.

Knowledge Transfer and Mobilization

The goals of KT and KM—which were highly innovative outside the area of health nearly a decade ago—were not discussed and analysed initially in the Metropolis Project. The early focus was on creating so-called policy-relevant research that could be fed to policy officials in government and disseminated at conferences and through working papers and other user-friendly publications. Both KT and KM are concerned with the problem of how to bring relevant and useable knowledge/evidence to end users so they are able to make better informed decisions and improve policy, programing and practice. The goal was to move “knowledge into action” and to bridge the “knowledge-to-action gap” (Graham 2006, 13–14).

Conceptually there is a challenge in that the literature fails to provide a consensus on the meaning of the terms KT and KM. In fact, a multiplicity of terms has emerged around knowledge creation and transfer with a large degree of overlap in their meanings (Levin 2008, 11). In its simplest form, KM can be thought of as the stage at which knowledge is created and gathered together (mobilized) before its transfer to end users. KT is related more directly to putting the knowledge into a readily accessible form and then finding the channels to transfer it.

Over time, it has become clear that KT and KM are dynamic practices that involve multidimensional, complex and long-term interactions rather than “a one directional and linear move from research to practice”. Too many models of KT/KM give “dominance to the work of researchers, with everything else being organized around the production of research” (Levin 2008, 11, 13). In reality, KT involves “efforts to understand and strengthen the relationship between research and practice …‘Mobilization’ indicates that this work requires specific effort, over time, working with others” (Levin 2013, 45).

From its inception, the Metropolis Project recognized the importance of building knowledge networks and facilitating knowledge exchange through interaction. Phipps and Shapson have noted that such multiplayer interaction creates ‘social innovation’ recognizing that knowledge useful for “developing sustainable solutions” to problems is best developed in collaborations (2009, 212). In fact, “… the use of research is fundamentally a social and organizational process. Whether people are interested in, pay attention to, and make use of research evidence depends much more on their organizational setting and social relations” (Levin 2013, 49–50). Policy analysts and decision-makers in government are far more likely to give attention to research and evidence that come from known social contacts (Levin 2013, 50).

One limitation in terms of how the federal government conceived of the Metropolis research partnership was their emphasis on the creation of academic knowledge and an under-appreciation of NGOs’ roles in knowledge creation. This was a gap which the academic and community partners in CERIS worked hard to address. For example, community partners were always included as central members in governance structures at CERIS and in adjudication committees that recommended the proposals to be funded with CERIS research grants. All CERIS-funded research also required meaningful partnership between academics and community.

Evidence-Based Policy-making

Public policy is about the “broad understanding, priorities, goals, and objectives that a government entity will possess with respect to a given field of human activity and governmental interest; … [it] refers to the set of understandings respecting what should be the ends of governmental actions …” (Johnson 2006, 63). How good public policy actually comes to be made is, however, complex and a contested subject (Howlett and Craft 2013). The common, but mistaken, assumption often made by those distanced from the political system is that policy-making is a thoroughly rational science where research/evidence is applied to solve policy problems in a neutral and technocratic manner (Goldberg 2006). The reality of the policy process is that it is “highly political, volatile, conjunctual, and ‘irrational’ in the traditional academic sense of scientific inquiry” (Richmond 2006). Policy-making is not a science but a practice and consequently evidence comes to be “mediated through political processes” (Levin 2013, 62).

While public policy-making will never truly be based exclusively on evidence and technocratic rationality, nor should it be in a democratic society, the idea behind having policy decision-making more informed by research evidence is that this will make policy more accountable and responsive rather than ideologically driven (Yanow 2007, 299; Howlett et al. 2009, 181). The goal behind evidence-based policy-making is the “attempt to enhance the possibility of policy success by improving the amount and type [/quality] of information processed in public policy decision-making” (Howlett et al. 2009, 157) and to make public policy both more rational and pragmatic (Solesbury 2001). There has been widespread movement in government and policy circles towards evidence-informed policy approaches to decision-making (Pawson 2006). There is also concern that due to government downsizing, the state bureaucracy is losing its policy capacity (Baskoy et al. 2011), hence the interest in making greater use of policy-relevant research partnerships (Shields and Evans 2012). Guided by a policy framework where research evidence has the possibility of having greater influence over decision-making, both academics and NGOs have strong motivation to become involved in partnerships like CERIS.

The Three Communities Involved in KT/KM

In KT/KM, it is helpful to consider the nature and the role of the three central communities engaged in the policy process. These communities are (1) the policy decision-makers (senior government politicians and top-level civil servants who actually make the final policy decisions); academics, NGOs, policy institutes, think tanks, media and the like, who are involved in the making and shaping of policy-relevant knowledge/evidence and perspectives; and (3) knowledge brokers (mid-level policy analysts and advisors in government policy units) (Cohn 2006).

Knowledge brokers provide a bridge between policy decision-makers and academics and NGO research partners in CERIS. The knowledge brokers are the policy professionals who translate and disseminate research evidence within government through the creation of briefing notes, background papers, presentations and policy consultations with senior management and political officials in ministries. Their work brings knowledge producers together with knowledge consumers (Bogenschneider and Corbett 2010, 2). The key to successful KT/KM is not just about “producing more knowledge, but also of improving both the desire and capacity for its use as well as the mediating processes” (Levin 2008, 8). The challenges of getting heard in senior levels of government are formidable but knowledge brokers facilitate this process. Hence, establishing sustained, regular and institutionalized communication and exchange between academic and community research partners and knowledge brokers greatly increases the possibilities that evidence from research will actually have the chance to influence policy. As CERIS partners, NGOs and academics gained greater access to government. The strengthened relationships between NGOs and academics allowed them to share their government contacts and relationships with each other—a triangular effect as opposed to one-way access between government and each of the other stakeholders. As a result, various levels of government—federal, provincial and municipal—enhanced their connections into the community and the university sectors.

The Role of Community-Based Organizations in KM/KT

NGOs involved in settlement and immigration issues were eager to partner with Toronto universities to build CERIS. The value of community organizations’ involvement in research is often attributed to their central role within the model of Canadian settlement services outside the Province of Quebec in which NGOs offer a community-centred delivery system financed by government. Viewed internationally as a best practice model (Richmond and Shields 2005), the system ensures NGOs have deep roots within the newcomer communities directly affected by government programmes and policies. NGOs are also involved in advocacy and educational initiatives with newcomer populations, putting them in a unique position to assess the effectiveness of policy and programming from the perspectives of service impact and broader community interests.

NGOs also are involved to various degrees in research which they often use in policy advocacy. In action research projects and reports, submissions to government, community newsletters, project evaluation documents and the like, a large volume of community-derived evidence/research is produced. This rapidly expanding ‘grey literature’ (sometimes referred to as ‘fugitive literature’) is widely acknowledged as making a valuable contribution to policy-relevant research even though it is rarely catalogued or centrally collected. Community research partnerships like CERIS help to bring this material more clearly into view and facilitate its uptake, often by posting documents and their bibliographic information to the World Wide Web.

Finally, NGOs are, as Shields and Evans note, “physically, socially and emotionally close to the communities they serve, and there are generally high levels of trust between them. … One of the strategic advantages of academic and government [partnerships with NGOs] is unprecedented access to grassroots populations for in-depth study. The ‘lived experience’ of these populations can be accessed in a manner that was rarely possible before” (2012, 262).

Collaboration in Action: the Community Research Symposia

In their third phase, the Metropolis research centres committed to promote knowledge exchange with all of their partners: non-governmental organizations, the three levels of government, and academic affiliates (Anisef et al. 2007). Each centre strengthened its relationships and facilitated knowledge exchange with NGO and government partners. Among the many strategies for deepening partnerships, public events that ranged from informal, brown bag lunchtime seminars in British Columbia, the Prairies and Quebec to public lectures and round tables at the Atlantic Metropolis Centre informed partners about each other’s research priorities and findings. At CERIS—The Ontario Metropolis Centre, NGO and government representatives took the lead in organizing four symposia to highlight pressing immigration and settlement issues and to publicize their own research contributions. The first symposium occurred in June, 2008, and the last took place in February, 2012. With attendance always reaching capacity wherever each symposium took place,Footnote 5 the events were greeted with almost unanimous approval by participants.Footnote 6 A steering committee that included representatives from the three founding community partners of CERIS; the Ontario Council of Agencies Serving Immigrants (OCASI), the United Way of Toronto and Social Planning Toronto, municipal and provincial government representatives, a CERIS director and the CERIS coordinator planned and organized each symposium. Here, we evaluate how these activities strengthened relationships among academic, NGO and government affiliates of CERIS and enhanced knowledge exchange. We also use the symposia to reveal the limitations of these relationships for effecting policy change.

The four symposia had diverse topics that arose from the evolving interests of the NGOs. The first symposium entitled “Settlement Without Borders” was designed to promote new scholarship and dialogues about immigration, settlement and refugee issues in Ontario and Canada. Taking up this challenge, the second symposium in February, 2010, was one of the first public events in Toronto to examine in detail the growing numbers of temporary migrants in Canada. Entitled “Permanently Temporary: Temporary Foreign Workers and Canada’s Changing Attitude to Citizenship and Immigration,” speakers described the increasing diversity of temporary workers and the issues of settlement, access to services and citizenship that they face. This was a ground-breaking discussion at the time that focused on many of the issues captured in recent headlines (Gross 2014; Smolkin 2013; Canadian Press 2013). Proposed changes in federal funding for settlement services in Ontario (Pagliaro and Mahoney 2010) led the organizers in a new direction in 2011 when the symposium examined the challenges of evaluating the impacts of the immigrant-serving sector on successful settlement and integration from the perspectives of funders and the sector itself. Local innovative approaches to evaluation that aimed to address stakeholders’ diverse concerns were emphasized. Responding to funders’ growing emphasis on achieving outcomes for each dollar of funding (Evans and Shields 2010), various ways to measure the outcomes of settlement services were discussed in the fourth symposium entitled “The Impact of the Sector Serving Immigrants: A Critical Introduction.” The merits of secondary data such as the census, agencies’ own administrative data and survey data for assessing outcomes were presented and debated.

The symposia had four distinctive characteristics that facilitated effective knowledge exchange. First, the topics were relevant to NGOs and all levels of government, responding to changing immigration policy and the evolving policy and funding context for immigrant-serving agencies in Ontario. For each symposium, NGO and government partners identified issues that had received little public attention at the time. The timeliness of each symposium is readily apparent from the topics that included temporary workers and the challenges of evaluating the sector’s outcomes (Try and Radnor 2007; Evans and Shields 2010). Municipal officials participated in every symposium while provincial and federal government representatives made presentations at the last two.

While the symposia offered opportunities to highlight the research and knowledge of NGOs, they also lent more authority to their research. The first symposium was invaluable in ensuring that academic affiliates appreciated the research being undertaken by NGOs and that they gained confidence that their research would be taken seriously by academics. By highlighting the depth, breadth and quality of research being undertaken by NGOs, the first symposium laid the groundwork for the later events for which many presenters were identified and invited by representatives of the NGO and government sectors. NGO and government partners were also instrumental in building audiences for the symposia. They publicized the symposia and encouraged attendance by their own stakeholders. Social Planning Toronto, the United Way of Toronto and OCASI broadcast information about each event to all of their stakeholders, while government representatives on the organizing committee ensured that information about each event was distributed to policy analysts across the federal, provincial and municipal governments.

To expand the audiences for the symposia and to enrich knowledge exchange, the steering committee undertook innovative dissemination activities. Diverse audiences including policymakers from the municipal, provincial and federal governments, agency workers outside the Greater Toronto Area and researchers at universities across the country received summaries of the presentations from the last three symposia. Intended for busy policymakers and service providers who do not have the time to read lengthy research reports and academic articles, the summaries encapsulated the findings and significance of each presentation in two pages. They were written in accessible language with as few technical terms as possible. Preparing the summaries provided an opportunity to introduce the principles and practice of knowledge exchange to graduate students who were trained to write them in accessible language. The summaries were also posted on the CERIS website where they are still available.Footnote 7

The symposia were important opportunities for service providers, academic researchers and government funders to meet and share their views. In this respect, the involvement of CERIS was crucial. Linked officially to the federal government’s Metropolis Secretariat and charged with establishing relationships with non-governmental organizations, as well as provincial and municipal government officials (Anisef et al. 2007), CERIS had extensive experience creating a space where all stakeholders could share their views respectfully and thoughtfully in safe environments where participants could speak freely outside the confines of their official positions. The value of this space for dialogue was evident at the fourth symposium where federal policymakers announced plans to develop a new administrative data system and welcomed comments on their proposal. This symposium that was held immediately before the 14th National Metropolis Conference was an ideal venue for the announcement since its low registration fees allowed local NGOs who were affected directly by the announcement to attend.

Evaluation: Achieving Successful Knowledge Exchange

The Community Research Symposia, like the National Metropolis Conferences, were an exceptionally successful example of knowledge exchange (Shields and Evans 2008). They created timely knowledge about issues relevant to policymakers in all three levels of government and the NGOs that serve immigrants. The symposia also made information accessible to diverse groups and individuals through ongoing collaboration. They also engaged diverse stakeholders, enhancing their communication with each other, and encouraging forthright exchanges of different views. Despite these successes, the Community Research Symposia also illustrate the challenges inherent in developing ongoing knowledge exchange with NGOs.

The community sector has very limited resources and capacity to devote to research. By contrast, many government departments have dedicated research and policy analysis units, and of course, a core mandate of the university is research for which each is comparatively well resourced. For community organizations, research is most often done ‘out of the side of the desk’ (Evans and Wellstead 2013) as NGOs are compelled to multitask and do ever more with less due to serious funding limitations (Evans and Shields 2014). The human and other resources that NGO partners have devoted to Metropolis are scarce and valuable assets. In the current context of austerity and deepening public service restructuring, the resource issues become even more challenging, raising questions regarding the long-term sustainability of NGO contributions to such partnerships. NGOs are at a considerable disadvantage in relation to other actors in the research partnership. Structurally, they are positioned as junior partners. The funding of CERIS reinforced this inequality since SSHRC rules required funds to flow through universities and lead academics with only minor funds channeled directly to NGOs.Footnote 8 This power differential was felt keenly by the NGOs (Legault et al. 2006).

Another factor is that while NGOs have put considerable effort into CERIS, the payback in terms of actual influence on public policy remains very difficult to assess since actual policy-making remains a hidden ‘black box’ activity (Young 2013). This leaves the question as to whether there was sufficient policy impact for the investment.

On the positive side, NGOs have an interest in raising the public profile of immigration and settlement issues and research. In fact, this is seen by the community as one of the important accomplishments of the Metropolis initiative (Legault et al. 2006, 2). The experiences and observations from the community concerning contemporary immigration and settlement issues were largely validated by CERIS and other Metropolis research (Legault et al. 2006, 5). The research findings legitimated many of the concerns long expressed by NGOs and confirmed the value of their work with newcomer populations.

Research on immigration and settlement is often useful to NGOs for grant and service contract applications, as well as for community education and advocacy. Partnering with academics in research projects can also be beneficial in terms of knowledge generation, resources (as limited as these may be) brought to the community through research grants and the ability through conferences, symposia and forums, for example, to build connections with governments, academics and other NGOs. Partnering with academics on research allows NGOs to augment their limited research capacity and advantageously position and enhance their voice through association with scholarly standards of evidence. NGO advocacy is too often discounted because they are viewed by government as ‘special interest’ organizations. Revealingly, one CERIS-based community member noted how embracing an “evidence-based advocacy” approach amplified the NGO voice, a process greatly facilitated through the NGO-academic partnership (Shields 2013, 24, 23).

Conclusions

In assessing the CERIS collaboration with community organizations, we should bear in mind the complex nature of the policy process. In terms of the production of policy-relevant research and the related activities of KT/KM, this experiment appears to have produced significant value for all partners. New perspectives on policy were introduced as well as new sources of research information. With respect to genuinely progressive policy reforms being adopted on the basis of relevant evidence, it is fair to suggest that the experiment strengthened and improved the relationships that are necessary to this process, without directly having determined the outcomes. The complex and highly politicized nature of policy determination means that enhanced policy dialogue does not always lead to policy change (Griffiths 2013).

Ultimately the assessment of this experiment will be provided by each of the three groups involved—academics, government and NGOs. Each will weigh the benefits against the significant resource commitments that were made. The CERIS Community Research Symposia provide a concrete example of discernible community benefit for NGO investment in CERIS research partnership. Evidence of this is found in the strong endorsement by the founding community partners and other settlement NGOs, as expressed at the 2013 event “Celebrating 17 Years of Immigration and Settlement Research: Achievements and New Directions”, in seeing CERIS continue beyond the life of the larger Metropolis Project itself and their continued willingness to devote precious resources to make this happen.

The reality is that each set of actors in a research partnership brings to the relationship different resources resulting in power imbalances, along with distinct expectations, values, cultures, capacities and needs. Successful partnerships require that these differences be acknowledged and accommodated. Trust, understanding and a sharing of resources between the parties are necessary foundations for effective knowledge exchange, translation and mobilization. In the end, KT and KM are about relationship building, not simply about doing and disseminating research. In this sense, CERIS and the Metropolis Project in general and the symposia in particular must be judged as successful as they demonstrate concretely the mutual respect that has developed between parties and the successful construction of a broader community engaged in meaningful and sustained knowledge exchange. This case study highlights the values and importance of evidence-based knowledge mobilization and transfer to policy-making in the immigration field and the key role that NGOs can play in this process. The CERIS experience offers valuable insights into successful knowledge exchange from which the immigration policy community composed of academics, NGOs and government officials in locations well beyond Ontario can learn.