1 Introduction

The manner in which we think about society, politics, and policy-making, how we try to understand and explain the causes and effects of state intervention and “societal problem processing” (Mayntz 1982), has been less a continuous process of knowledge enhancement than a discontinuous advancement that repeatedly was influenced by fashion waves—coming, going, and returning. However, this does not mean that no progress has happened. Scientific progress, at least in the social sciences, is no gradual ascending process in which the shelves in the warehouse of knowledge are increasingly replenished. New perspectives trigger shifts in the focus of analysis, improve precision and explanation in some areas, but deteriorate analysis in others. Important explanatory factors sometime step into the background until they get rediscovered after the fashion wave has faded.

The main thesis of this contribution is that the analysis of policy processes in recent decades has focused too much on governmental and conventional political actors, on the one hand, and too much on actor-centered bottom-up perspectives. As the microfoundation of social explanations has moved to the fore in the past decades, actor constellations became the core of policy explanations. City Hall policy then was treated in a similar way as national policy-making or even negotiations in global politics. This pushed contextual factors and systemic perspectives into the background.

The following proposal for a renewed perspective on public policy aims to bring social factors back into play, particularly at macrostructural level. This means not only that non-governmental and civil society organizations should be given greater consideration, but even more important are various forms of structural differentiation at the macro level of societies which should be reintegrated into policy explanations.

Such macroscopic perspectives seem to be particularly fruitful in cross-national analysis. In this respect, for example, it is important to identify not only differences in the policy subsystems, but more generally in their embeddedness in the web of political, economic, scientific, and media subsystems of society in order to understand how different societies cope with important challenges. An example of the fruitfulness of such a macroscopic perspective is climate policy, where the social origins of this creeping policy problem also suggest a societal perspective for coping with this challenge. In this policy domain, we can clearly see how varying national political systems and their interaction with science, economy, and media have differential influence on the perception and communication of this policy problem as well as the translation of this awareness into concrete policy formulation and implementation processes (Schneider et al. 2013; Satoh et al. 2018). From a more comprehensive view, this problem of adaptation in the process of social evolution is a central topic of complexity-oriented approaches (Schneider and Bauer 2007).

The plea for a macroscopic perspective in comparative policy analysis will be bolstered in this chapter by a comparison of major policy theories which show the range of how policy explanations can incorporate societal factors into their explanans. It will be shown that many theories, frameworks, and approaches shift major societal determinants into the background.

This paper proceeds in five sections. After this introduction, a conceptual part first aims to specify the variants of meaning of the central concepts—public policy and society—in this essay. The third part compares two dozen policy theories in terms of their analytical dimensions and the role that societal aspects play in description and explanation. In the fourth section, a proposal is presented on how macrostructures of “society at large,” different forms of societal differentiation, and particularly civil society and other non-state sectors should be reintegrated into public policy analysis. The paper concludes with a short summary.

2 Public Policy and Society

Before we enter into the comparison of theories, we have to clarify what we mean by the central concepts exposed in the title of this essay. These are not self-explanatory and often ambiguous. As so often in the social science, there is conceptual confusion, and this is very often due to the fact that the meaning of these concepts is based on their multiple positions in the different “theory nets” (Stegmüller 2013) in which they are embedded.

2.1 Society as Part and Whole

Recent overviews in social sciences and philosophy show that there are multiple versions of the society concept (Schwinn 2011). For instance, within Grand Theories à la Parsons and Luhmann, the term is theoretically much more presuppositional (Parsons 1966; Luhmann 1977). A sociological systems theory conceives society as a supra-individual entity with emergent characteristics. It includes specialized social spheres or “subsystems” such as economy, culture, and politics at the national and global level. The German language offers the terminus Gesamtgesellschaft (society at large) for such a comprehensive view.

Marxists use the term “social formation” for this inclusive view, which encompasses the totality of all social relations and conditions. Both the Marxist and the system perspectives are examples of a holistic view in which society “hovers” above its members.

With regard to Luhmann’s inclusive concept of society, it must be noted that it covers all subsystems of the social, from economy, law, politics, media, education, etc. and of course includes social entities such as the state although Luhmann has often stressed that the system perspective makes the concept of the state obsolete. A further specificity of his society concept is that it includes solely communication relations and subsystem-specific codes of communicative interchange and completely ignores that also other relations such as energy exchange exist in all types of societies.

In contrast to this “society at large” concept, we can distinguish society as a subsystem of this supra-system. We find this use in German “Staatswissenschaft” (science of the state) of the eighteenth and nineteenth century. Society herein was distinguished from other social spheres such as the economy, the law, and the state. The most prominent perspective of this kind of differentiation had been provided by Hegel, treating society as a specialized sphere which is separated from family on the one hand and the state on the other (Hegel 1820). In Hegel’s dialectical conception, bourgeois society was the antithesis to the family, a contradiction in the social world that was resolved by intermediate powers such corporations and the police, and finally the state as superior force of integration. However, Hegel also used the concept of the political state, which comes closer to the conventional concept of government to be distinguished from the state as such.

Hegel was influenced by Enlightenment philosophy. Influential were Montesquieu, who distinguished between “l’état politique” and “l’état civile,” and British social philosophers like Locke, Smith, Hume, and Ferguson which supported the idea of an autonomously self-regulated sphere to be separated from the state. Ferguson coined the notion of “civil society.” Cohen and Arato argue very convincingly that the conceptual separation of state and society was a result of Enlightenment philosophy in which society in a way was mobilized against the absolutist state (Cohen and Arato 1994).

For Marx, for his part influenced by Hegel and the British philosophers, the state lost the superior macro position and became a part of society, i.e., a power instrument of the rulers in a class society to defend their social supremacy. Marx used the concept of society at the highest, inclusive level, but stressed that in capitalist societies economic conditions would dominate all others relationships.

A narrower society concept is used in the sociology of Simmel and Weber. For Simmel, society is not a supra-individual but only an inter-individual entity, the sum of relationships that socialize people. Important in this perspective is his idea of intersecting social circles.

Weber is a special case. He never explicitly defined the concept of society in his main work entitled “Economy and Society” where it remains unclear whether economy is juxtaposed with society or whether economy represents a certain order of society. However, he also speaks of Vergesellschaftung (socialization), which he distinguishes from Vergemeinschaftung (communitarization) in the sense of Tönnies, who uses the narrowest concept of society.

In Tönnies’ writings, society is not just a network of relations but a specific type of relational configuration that is based on contractual agreements. For social relations such as kinship, friendship, and neighborhood that create social proximity, he reserved the concept of community. This connotation of society thus is close to the original meaning of the Latin word societas, which designates contractual agreements for the joint pursuit of common interests.

On the basis of this conceptual analysis, at least six versions of this term can be distinguished. Society can be:

  • A super system, including everything social (people and their relations) in a country or the whole planet (world society) (S1)

  • A system with emergent, supra-individual properties that encounters individuals as superior power (S2)

  • A non-governmental subsystem that is separated from the state (S3)

  • A subsystem that is non-governmental but also separated from economy or business (S4)

  • An inter-individual entity, i.e., a network of social relations (S5)

  • A network of contractual relations (S6)

The main differences are thus that S1 and S2 conceive society as the all-inclusive macro level, from whose perspective state and politics represent only subsystems. S3 and S4 treat society as a subsystem of “society at large,” and S5 and S6 understand society only as network of relations. More recent approaches to social theory, which speak of “sectors close to the state” or “societal capacity to act” (Mayntz and Scharpf 1995), for example, or which conceptualize the “penetration of society by state power” as the infrastructural power of the state (Mann 1984), tend to use the terms S3 and S4. There are few philosophers and theoreticians who combine S2 with the concept of the state (Etzioni 1968; Willke 1995; Bunge 1998).

2.2 Public Policy and Governance

The term “public policy” has also been around since centuries, but a more theory-inspired sophisticated use only emerged since the 1950s, when policy sciences and policy analysis emerged. Implicitly, this term was understood to designate exclusively government action, although the word policy is more abstract and can be applied to all principled strategies of action at all social levels (individual, organization, state, society). Lasswell, a central initiator of policy analysis, used the term “policy” in this general way (Lasswell and Kaplan 1952). Only later the term was narrowed down to public policy or state policy-making. Thomas Dye defined policy analysis as “finding out, what the governments do, why they do it, and what difference it makes” (Dye 1972). This conceptual restriction to government action was clearly an effect of the Bringing-the-State-Back-In movement, in the course of which the political system was “besieged by the state” (Easton 1981) and finally defeated in such a way that even pure sociologists spoke only of “the organizational state” (Laumann and Knoke 1987). The German political scientist Manfred Schmidt translated this branch of research as “state activity research” (Schmidt 1993).

This state-centric perspective was only broken up by governance research, in which the formulation and implementation of collective decisions was conceived as the production of public and common goods, in which also a broad spectrum of private actors and non-governmental coordinating mechanisms could be involved (Mayntz 2003). Private actors in this function were referred to as “private interest governments” (Streeck and Schmitter 1985).

Particularly Ostrom demonstrated in her “Governance of Commons” that non-state actors can provide important contributions to societal problem-solving (Ostrom 1990). Governance became an extended view of policy processes in which a wide range of actors, organizational forms, and coordinative regimes were included into the analysis and reconstruction of societal problem-solving mechanisms (Mayntz 2003; Schneider 2004; Grande 2012).

But here, too, we must exercise conceptual caution since governance is a pretty ambiguous concept. A narrow version equals governance with public policy, reducing this activity domain to the state sector. A broader concept of public policy conceives state policy-making only as a subset of governance, which includes governmental and non-governmental actors, as well as hierarchies and networks as coordinating mechanism. The most extended concept of governance is provided by industrial economics and includes in addition markets as coordinative mechanisms (Williamson 1979).

The concept of society and the public policy perspective are very different centers of theoretical gravity around which different scientific discourses revolve. However, we can combine both conceptual perspectives. Society can be mapped as a landscape with different areas and levels that influence policy-making and governance processes. From this perspective, society impacts on public policy from three different directions:

  • Influences from “society at large,” i.e., the macro level of society where specific patterns of functional differentiation and the particular interconnectedness of societal subsystems are shaping policy processes

  • Influence by the relational level by which the whole spectrum of networks shows effectiveness

  • Influences from the actor level, i.e., non-governmental actors from civil society and private business participate in policy formulation and implementation

Society thus influences policy processes through various pathways. In the next part of this chapter, a meta-theoretical inquiry will examine in which policy theories such influences were taken into account.

3 Policy Theories and the Impact of Society on Policy-Making

The theories (in the broadest sense) which have been developed to explain or understand public policy-making can be thought of as a landscape that has been increasingly populated over the last 100 years. As the architectural settlement shows geographical differences, there are also different cognitive constructions in theory formation, ranging from metaphors to paradigms, approaches, and frameworks, to formal theories of policies as research objects.

In this theory landscape, a large number of villages and agglomerations emerged that makes it difficult to maintain a clear overview. The following is an attempt to map this landscape. Many theories and frameworks are listed in reviews and textbooks. Because a systematic account is not yet available, our classification presented here will include only two dozen theories. The theories to be compared are listed in Fig. 3.1, including classical social science theories of the early twentieth century up to most recent perspectives in policy analysis. They are grouped into different types of theory, ranging from grand theories, which have a general and overarching claim to explanation, to middle-range theories, which explain sub-areas of the social, to mini theories, which deal with only a few variables or aspects. Approaches and frameworks tend to represent networks of theories and methods specified to the explanandum “policy process” or “outcome.”

Fig. 3.1
A matrix of the various societal influences in the macro, meso, and micro levels on the various public policies of the state, civil societies, and private sectors. The theories listed in the figure include politics, economy, science, education, media, transactional, communicative, communitarian, associations, and firms.

Societal influences on public policy

Full size image

From a comparative perspective, we will examine these with respect to a dozen properties, such as levels of analysis, facets of reality, relationships, and societal subsystems. For each feature, we check whether it is taken into account in a particular theory.

Levels. Theories can have a purely individualistic orientation and try to trace policies back to the micro level (mostly based on methodological individualism), and most popular here is rational choice. The counterpart is holism. Holistic theories explain policies by macrostructures, and most prominent here is Marxism in which all societal spheres (politics, law, culture, etc.) are derived from the operating logic of capitalist economy.

Between micro and macro, there are further layers that may be distinguished. Looking bottom-up, the next societal layer is the level or organizations. Individuals join such collective constructions as members or employees by creating corporate actors with emergent properties (Coleman 1974; Mayntz 1986). Above this level are social groups or classes, and the highest level of a macro-societal perspective is the level of societal subsystems based on functional (politics, economy, science, etc.) or institutional (State, Civil Society, Private Sector) differentiation.

Facets. The second group of dimensions differentiates between three facets of social reality—the material, institutional, and ideational. The material facet refers to concrete, physical, biological, etc. interests, resources, and interactions. Typical material interests refer to human basic needs. The institutional facet refers to rules and norms that structure problem perception, interests, and actions. This institutional sphere is sometimes difficult to separate from the ideational. For instance, sociological institutionalism is merging both facets.

The ideational facet refers to cognitive representations of reality. It is the sphere of ideas, beliefs, and images. Recent policy theories put emphasis on this facet by analyzing discourses, frames, and narratives in the policy process.

Relations. Links or ties between policy actors are an important aspect of actor constellations in the policy process. Although the spectrum of possible relationships is huge (Borgatti et al. 2009), a rough distinction is made between three main types of relationships that are relevant for policy processes. Firstly, these are transactional relationships involving a utilitarian orientation in the exchange of resources (Williamson 1975). Secondly, relations may relate to communication and exchange of information (Leifeld and Schneider 2012). Thirdly, communitarian relationships can be involved, such as friendship or face-to-face meetings that generate social proximity and trust (Odella 2011).

Spheres. In contrast to the vertical differentiation of society into levels, a horizontal differentiation into domains with different rights of action and institutional status can be conceptualized. We differentiate in this respect between the state, civil, society and the private sector.

In the following comparison, two dozen theories, approaches, and frameworks will be examined. Each is checked if it takes into account 1 of the 12 dimensions outlined above. We sketch the theories with their major elements, their explanations of public policies, and of course we are particularly interested in whether and how social factors and components are included in a theory.

We begin with Marxist Class Struggle Theory (CST) as the oldest framework considered here, where society at large is divided into social classes based on economic structures. The basic traits of this perspective are laid down in The Communist Manifesto of Marx and Engels (Tucker 1972). Public policies here express the material interests of the ruling class or alliance of classes. In this perspective society comes in at the macro level as well as the actor level, where large societal aggregates (classes) but also individual capitalists shape politics and policy-making. Civil society came into the play only by neo-Marxists such as Gramsci.

Pluralist group theory (PGT) broadened this understanding of policy-making as power struggle to a group conflict that is not only restricted to material interest but also includes ideational groups (Bentley 1967). This pluralist perspective conceives policies as a resulting vector in a parallelogram of forces, and society enters the picture at the macro level by background factors generating these group structures.

It was only with System Theories (ST) that concepts became popular in which societies had to deal with challenges and problems of adaptation to preserve societal orders (Parsons 1966; Luhmann 1969). Policy sciences emerged at that time, when systemic thinking flourished. Its basic idea was that societal problems could be better solved if scientific expertise would be pooled across disciplines. Some of these approaches applied cybernetic models to politics and conceived governing as a process of self-regulation in which the political system detects undesirable states and initiates corrective action by the formulation of public policies (Almond 1956; Easton 1957).

Such theories have a holistic and functionalist orientation. In the 1980s, the derivation of policies as functionally necessary problem-solving was increasingly criticized (Elster 1982). The critics from the individualistic camp in contrast emphasized the micro level as the appropriate level of analysis, and collective action in policy processes was explained by the rational choice of actors in the pursuit of their material and power interests.

Individualist exchange theories first developed in sociology and subsequently diffused into political science, particularly by James Coleman’s model of political exchange (Coleman 1990). This way Exchange Theory (ET) entered the realm of policy studies, often using the sophisticated mathematical modelling of exchange and bargaining relations. In these models, social embedding fades completely into the background (Knoke et al. 1996).

The new grand theory, which largely superseded systems theory, was Rational Choice Theory (RCT) in which policy preferences and outcomes are explained by rational interaction of individuals or organizations (Shepsle and Bonchek 1997). While the analysis usually includes the micro level and only in rare instance the meso level, rational choice theory ignores aspects of social embedding.

Neo-corporatism (NC) is as a combination of group theory and exchange theory at the macro level of a political system. Policies in this perspective are bargained compromises between the government and large societal groups, particularly capital and labor (Schmitter and Lehmbruch 1979). Since the concrete structure of organized interests and also the tendency of governments to negotiate often have historical causes, societal influences of policy processes play an important role in this theoretical perspective. The distinction between corporatist and pluralist societies has therefore established itself not only in policy research but also in comparative politics (Lijphart 1999).

Some mini-theories explore only the influence of one or only a few variables on public policy. Each of these theories focuses on a singular type of actor. Power Resources Theory (PRT) explains public policies, e.g., the expansion of the welfare state, largely by the organizational power of trade unions (Korpi 1985). Governing Party Theory (PGT) explains public policy mainly by party politics and the impact of governing parties or party coalitions. It hypothesizes, for example, that it makes a difference in government spending or privatization whether left-wing or right-wing parties rule (Schmidt 1996). From a societal perspective, party government theory is state-centric and narrows the spectrum of actors to conventional politics, while power resource theory, like neo-corporatism theory, at least includes the trade unions as “societal superpowers” in their explanations.

The Networked Governance Approach (NGA) sees networks as a distinct form of governance (Powell 1987). It emphasizes on the one hand the involvement of a pluralistic spectrum of actors in the formulation and implementation of policies and combines this with theoretical concepts of exchange and negotiation in order to describe a polycentric and a network-shaped constellation of actors in the production of public policies (Kenis and Schneider 1991; Mayntz 1993). Societal aspects come into play in this perspective by the wide-ranging actor structure (e.g., also private and civil society are included) but also by the analysis of network structures.

Historical-Institutionalist (HI) approaches strongly emphasize the impact of evolutionary, historically grown formal and informal institutional arrangements on policy-making (Thelen 1999). Examples are the structuring of policy subsystems by long-term social processes. For instance, to use a recent example, the German energy system transformation cannot be explained without reference to the traditional strength of the German environmental movement (Rinscheid et al. 2019). This approach is open to the influence of societal factors on public policy, whether through actors, institutions, or ideas.

A specialized rational choice perspective close to the party government and power resources approaches is Tsebelis’ Veto Player Theory (VPT), which also relies on the institutionalist perspective (Tsebelis 2002). Policy influence here is derived from veto positions that policy actors acquire via institutional structures and party system configurations. This may be, for instance, the power of a second parliamentary chamber to veto a law or the power to stop action by each of the coalition partners in a party government. Political systems differ by their number of veto players, and mathematical analysis in rational decision-making then claims that the greater the number of veto players is, the more difficult it is to generate policy change. A serious shortcoming of this theory in our perspective is that societal factors only play a marginal role.

There are further neo-institutionalist approaches that combine actor-centered with institutional analysis and assume that actors are constrained but also enabled by institutional systems specifying sets of rules that are relevant in policy processes.

The Institutional Analysis and Development (IAD) framework of Ostrom (Ostrom 2011) emphasizes that policy-making systems not only contain rules that specify actor’s attributes and interaction outcomes, but also position in policy arenas (“access rules”) and relations structuring information exchange (“Information rules”) (Ostrom 1986).

The Actor-Centered Institutionalism (ACI) (Mayntz and Scharpf 1995; Scharpf 1997) conceptualizes policy systems by multiple actors with different interests, specific action orientations, and typical conflict constellations. Diverse institutional arrangements mediate conflicts and facilitate or hamper certain policy outcomes (e.g., structural reforms of redistribution). Since institutions facilitate coordination, various forms of coordination imply different transaction costs. Scharpf (1997) distinguishes between positive and negative coordinations. In the first mode, all actors bargain with all other actors on each policy option, while in the second, all policy options are excluded to which at least one actor objects. This reduces coordination and transaction costs. Society comes in here only by actor constellations and institutions that include societal actors.

A new conceptual development in a similar direction is the Ecology of Games (EoG) approach (Lubell 2013). Its central idea underscores nestedness and overlap of policy games. Actors are involved in multiple and cross-cutting conflictual relations, and policies are explained by decisions that maximize the different player’s combined outcomes. An instructive example of such a game network is Scharpf’s analysis of economic policy of the 1970–1980s where games between party governments and trade unions had been linked with voting games (Scharpf 1997). Another example is the concatenation of an innovation game with a regulatory game, by which coordination problems and technological frictions in technology policies can be explained (Dutton et al. 2012).

Recently, this perspective has been linked to the Collaborative Governance (CG) approach (Emerson et al. 2012). Specific arrangements and relations are emphasized which facilitate and support cooperation, information sharing, and joint problem-solving. The CG thus adds a clear relational facet to the IAD and ACI perspectives by combining the analysis of conflict constellations with institutional affiliations and collaborative contacts. Here too, it depends on how inclusive actor constellations are conceptualized in order to include societal aspects into the policy process.

A special perspective is taken by Ecological System Theories (EST) which seems to be a new Grand Theory that integrates different versions of evolutionary approaches and complexity theory. This stream of thought was prominent in multiple variants during the last decades (human ecology, population ecology, and organizational ecology). Eco-approaches apply biological concepts of the eco-sphere to social areas, and some of these concepts are also applied to policy-making. This perspective examines habitats, resources, and multiplex relations between species (symbiosis, mutualism, but also predatory relations, etc.) to determine equilibria and adaptation capabilities of these systems. Particularly in the field of innovation policy, this perspective is quite popular, and a number studies cover “innovation ecosystems” (Ferasso et al. 2018). In such systems, different “organizational species” interact in complex ways: universities, research organizations, business firms, finance capitalists, trade associations, and governmental policy-makers. This perspective is not only open to all areas of society but is also sensitive to a variety of organizational forms (“species”) and the multiplexity of relations that are involved in policy-making (Shaw and Allen 2018).

The relational dimension plays an essential role in Social Capital Theory (SCT) (Putnam 1995). Recently, this perspective has acquired the status of a new Grand Theory in the social sciences. In policy analysis too, this theory gained currency, particularly in network studies. It emphasizes interpersonal relations under the assumption that a particular class of relationships generates long-term trust and reciprocity. A strong influence on this perspective had the assertion that multiple membership relations in associations are of great importance for the emergence of social trust. This idea is close to Simmel’s idea of intersecting social circles, and it brings Tocqueville’s idea back to memory that membership in associations is an effective remedy against individualism, which ultimately would destroy social cohesion.

Another group of policy approaches highlights the role of ideas in policy processes. One of these is the Policy Paradigm Framework (PPF) operating at the macro level of politics. In applying a Kuhnian perspective to the world of policy-making, policy conflicts are largely understood as struggles among “policy paradigms,” such as between Keynesianism and Monetarism in economic policy-making (Hall 1993). Policy preferences and outputs are explained by the dominance of a given policy paradigm or by paradigm change. Society comes in at the macro level at which such world views are shaped.

An approach applying a similar principle at the level of belief systems is the Advocacy Coalition Framework (ACF) (Sabatier 1988). Its central premise is that policy actors form advocacy coalitions based on competing beliefs. Beliefs are embedded in systems with hierarchical structures. There is a deep core of basic convictions about normative and ontological aspects of the world, and at the periphery are secondary beliefs on instrumental and informational aspects that adapt to changing circumstances most easily. ACF assumes that policy brokers mediate the conflicts between coalitions and belief change occurs by means of external shocks, communication, and learning. Since communication plays an important role in maintaining coalitions, there is a range of relations to be studied in this perspective. Societal factors only play a role if actor coalitions include also societal actors and extensive networks.

The Multiple Streams Framework (MSF), too, has an ideational orientation and is mainly interested in the first phases of the policy cycle (Zahariadis 2007). It views policy-making not as a rational decision process but as a contingent confluence of three autonomous processes: (1) a process in which problems are defined; (2) a process in which ideas for problem solutions emerge and diffuse; and (3) processes at the macro level of politics (elections, legislative turnover, etc.) that support or suppress policy issues. The coupling of these streams opens policy windows in which political entrepreneurs can push their issues on the policy agenda. Societal factors flow in here via the level of macro-politics and private interest groups.

A closely related perspective is Punctuated Equilibrium Theory (PET) which combines group interaction and macro dynamics in political systems (Baumgartner and Jones 1993). Its key idea is that policies usually evolve incrementally and only occasionally depart from a steady path by big aberrations. Such policy punctuations depend on the capacities of groups for agenda setting. PET distinguishes between subsystem politics and macro-politics, and issues may shift from a rather technical matter to a macro-political issue when there is a switch in policy images. Society comes in by the same trajectories as in the MSF.

In the last few years, various currents of ideational policy research have emerged, which not only analyzes discourse formations but also emphasizes power structures. Discourse involves power in the sense that policy problems are defined in a way that certain social groups benefit by a given definition. One approach of this view is the Narrative Policy Framework (NPF) which emphasizes the role of narratives in the definition of situations (Shanahan et al. 2011). Social factors play a role here insofar as the general predominance of certain narratives in society has an influence on the perception of policy problems.

Another group of theories emphasizes the economic sphere of society at large. These are the Varieties of Capitalism (VoC) theory (Hall and Soskice 2001) and Post-Democracy Theory (PDT) (Crouch 2004). In contrast to the ideational approaches just discussed, these two perspectives focus strongly on the material sphere of politics, and both bring into play a modernized version of Marxism in which macroanalysis and economic structures are placed at the center of the analysis.

VoC has a clear macro-orientation in explaining policies but includes society only partially. Just as Marxism holistically understands current society as a social formation shaped by the capitalist economy, this approach emphasizes the dominance of economic interests and constraints in political processes. Its innovative point is that different versions of capitalisms exist. Capitalist economies (based on specific institutional settings) vary in their economic and social policy performance due to different forms of coordination (market versus state plus associations, to put it simply). In order to explain policies, this approach emphasizes the role of large companies and associational structures in the business sector.

PDT is currently quite popular and can also be applied to policy analysis (Schneider 2015). The prefix “post” pretends a trend reversal of democracy since the 1980s when globalization transformed large corporations into the main business actors and marginalized business associations and trade unions. Governments would get dependent on the expertise from large corporations, and policy formulation would move to small circles including central government and the business elite. Elections and parliamentary debates would decline to pure theatrical spectacles, boosted by media conglomerates with little effect on policy formulation. PDT is clearly macro oriented by emphasizing large material structures but also aspects of ideational domination. Society as subsystem comes in particularly through private business and institutional structures of the economic system, but also by links to science and education systems.

A comparison of the different theoretical profiles in Table 3.1 shows that, with a single exception, all the theories have different property characteristics with respect to the 12 dimensions. This becomes evident when we translate the property profiles into binary vectors to perform multidimensional scaling and cluster analysis (Chatfield and Collins 1980).

Table 3.1 Theories and approaches in public policy analysis
Full size table

The integrated results of this analysis are shown in Fig. 3.2 where the 24 theories are grouped into 4 clusters:

  • The largest cluster is located at the top of the diagram and consists of actor-centered and institutionalist theories.

  • The second largest cluster in the lower half includes three ideational theories, two relational theories, and one macroscopic theory.

  • The cluster to the left comprises includes ecological, systemic, and relational theories.

  • The cluster at the right border includes two theories that are actor-centered but also include some macroelements in the analysis.

Fig. 3.2
A cluster diagram of theories and approaches. The first cluster is for institutional theories, the second for ideational theories, relational theories, and macroscopic theory, the third for ecological, systemic, and relational theories, and the fourth is for macro elements.

Clusters of theories and approaches

Full size image

In spatial terms, the EoG approach and the class structure theory are close to the theories involving varieties of a systemic perspective.

4 Actors, Networks, and Systems in Public Policy

Conceptual analysis and theory mapping have shown that policy-making is not just a matter of state activities and politics, but also societal factors and relationships come in at several levels and venues. Firstly, from the broader spectrum of actors, in which not only governmental actors but also non-governmental actors from civil society and private business are involved in policy-making. Secondly, societal influences occur via social relationships that facilitate exchange of resources, discourses on policy topics, and social bonds that generate social capital. Thirdly, individual actors not only join in organizations and social movements, but also assume specific social positions in the increasingly complex societal division of labor. Policy actors thus have structural positions in the policy process that are dependent on patterns of functional, institutional, relational, and organizational differentiation. In order to make use and integrate these different conceptual perspectives, we draw on five different literatures: Social Network Analysis, Governance Theory, Systems Theories (of societal differentiation), Eco-System Theories, and Organizational Ecologies.

4.1 Institutional Differentiation Between Societal Sectors

Most institutionalist theories and approaches agree with the assertion that institutions are social rules that enable certain actions and restrict others. They structure action situations and stabilize mutual expectations. This leads to distinct categories of actors that are equipped with varying opportunities and constraints. We conceptualize these social processes as mechanisms of institutional differentiation that ultimately distribute action resources and control positions among the actors. In our context they regulate access to policy arenas and allocate decision-making power. Some of the theories outlined above put large emphasis on this role of institutions. In particular Ostrom has tried to systematize the various rules by distinguishing between access rules, position rules, area rules, authority rules, etc. (Ostrom 1986).

On an abstract level, each rule configuration—as governance structure—represents a complex distribution of control. In this respect, a broad spectrum of institutional mechanisms and societal status is conceivable. Firstly, between hierarchy, networks, and the atomistic market as an extreme point of decentralized control and, secondly, between the public and the private sector.

The most important dimension of differentiation is the distinction between “the private” and “the public.” Since these status positions are in most cases determined by sets of rules, an institutional status is always based on a “bundle of rights” which allows many combinations. This leads to the observation that a private or a public status is not a binary institutional state such as black and white, but there are many shades between the two poles. If one looks just at public administration, one can find a spectrum of diverse organizational forms ranging from pure state-controlled organizations via independent agencies to public corporations that are only indirectly under public control (Gill 2002).

The spectrum of differentiation between the public and the private sector organizations also includes an intermediate area, which some designate as the “third sector,” others as civil society or a non-governmental sphere. These terms are not very sharp, and sometimes they denote very different things and are even inconsistent. For instance, the term “non-governmental organizations” usually does not contain all organizations with a non-governmental status and excludes, for instance, business firms. The term “civil society” no longer refers to the entire non-governmental part of society, as British moral philosophy used it, but excludes business with its companies and associations.

The business world, too, is no longer involved in policy processes only by business associations, but there is a variety of organizational forms (Streeck et al. 2006; Grote et al. 2008). The public-private axis of institutional differentiation thus covers multiple organizational forms, a situation that has similarities with biodiversity in ecosystems. In this respect, it makes sense to make use of organizational ecology approaches to describe these differentiation processes at organizational level. This aspect will be discussed in more detail below.

4.2 Relational Differentiation in Multiplex Networks

Another venue for social influences in political processes is personnel and organizational networks. As described in the previous part, from a functional point of view, we can differentiate between three network forms:

  • Transactional networks in which the focus is on the utilitarian exchange of resources on a quid pro quo basis. These may include not only material items such as money and personnel, but also technical information and specialized expertise.

  • Communicative networks in which symbolic interaction, discourse, argumentation, and persuasion with regard to policy goals and instruments are at stake.

  • Communitarian networks in which social bonds based on friendship, neighborhood, and “intersecting social circles” become effective in the creation of mutual trust and social cohesion.

Networks based on these types of relations involve all the categories of actors in the institutionally differentiated spectrum described above. Among the theories discussed, the networked governance approach particularly emphasizes this relational level in the production of public policies. However, networks should not only be reduced to this governance perspective (Pappi 1993). For some time now, ideational and institutional frameworks such as the ACF and the HI also include communicative networks in their analysis and examine whether, for example, belief changes are induced by information exchange and collective learning (Weible 2005; Rinscheid et al. 2019).

From a macro perspective, it is interesting to see which network roles the various actors play in a total network (Ferligoj et al. 2011). The respective roles are of course dependent on the network type. For instance, in a communication network roles can be sender and receiver, leader, follower, exchanger, gatekeeper, multiplier, etc. (Friemel 2010). Roles in the resource networks can be producer, user, exchanger, distributer, etc. By social network analysis, relational structures can be examined both at the individual and group level as well as at the macro level of total networks. In this way, specific macrostructural positions can be conceived as roles in a total network.

4.3 Functional Differentiation and Societal Subsystems

A further path of societal influence in public policy-making is functional differentiation at the macro level of societies and the effects of these structures on policy actor constellations. Functional differentiation impacts on the partition of actor constellations into various subsystems. Laumann and Pappi applied such concepts in the late 1970s and therefore represent a rare combination of systemic and relational analysis (Laumann and Pappi 1976).

The affiliation of actors to societal subsystems implies specific action orientations. Each subsystem is focused on specific criteria of relevance that are important within specialized social spheres. In politics, the name of the game is “how to gain and retain power,” in business “how to make money,” in science “how to find truth,” in the media sector all is about “attention,” and so on. Luhmann’s merit was to systematize this basic idea of communicative differentiation. However, the baseline of this logic of differentiation we can find in many theories. In some respects, we can trace this idea back to the ancient Greeks: In Politeia and in Politikos, Plato has emphasized the different action orientations of statesmen, helmsmen, doctors, and shepherds.

The affiliation of policy actors to specific societal subsystems thus implies that macrostructures influence their basic orientations, e.g., scientists have different objectives than politicians or businessmen. Systems theories à la Parsons and Luhmann but also actor-centered perspectives to functional differentiation share this idea of subsystemic specific modes of orientation (Luhmann 1977; Mayntz et al. 1988; Schimank 2015).

In the light of diversity and complexity of modern societies, Luhmann’s macro perspective appears more attractive than that of Parsons because it expands the rather parsimonious fourfold AGIL scheme to an almost unlimited array of societal subsystems. Not only politics, economy, and culture differentiate into partially autonomous spheres, but also science, education, the media sector, and many more societal domains emerge in the great division of societal labor. A serious shortcoming of Luhmann’s perspective, however, is its mono-relational focus. Society is exclusively based on communication and domain-specific codes. Communication may be an important facet of society, but there are additional levels and building parts of the complex social fabric that makes up our modern world. In addition to communication, there are multiple relations involved which connect individuals and organizations on many levels. Of particular importance is the exchange of resources, a kind of counterpart to energy exchange in ecosystems.

More recent system-theoretical approaches try to overcome the holistic and mono-relational deficits of the old theories à la Parsons and Luhmann. They address both the multi-layeredness and the multiplexity of social structures in their analysis. For some time now, these have been discussed as the third wave of systems theory (Schneider and Bauer 2007; Schneider 2012; Waldherr 2017).

The reduction of societal relations to communication leads to a kind of “pigeon hole thinking” with respect to systemic differentiation, in which each subsystem is completely sealed off and self-referential like Leibniz’s windowless monads. However, inter-systemic links and overlaps between social subsystems, for example, between science and politics, or between the science and education are so obvious and omnipresent that it is surprising that Luhmann did not capture these outstanding features. Societal subsystems are intertwined, nested, and embedded in complex ways.

The key features of modern societies are critical infrastructures and “large technical systems” like energy systems, transport and telecommunication systems (Mayntz and Hughes 1988). These systems are transversal to functional subsystems and often interpenetrate these subsystems. A major feature of modern societies is precisely that different societal components are functionally so densely coupled that a failure in one part has fatal effects on many other parts. The idea of self-referential closure may be plausible for symbolic systems, but for society at large it is unrealistic. More convincing is Bunge’s systemic perspective of societal subsystems that are interconnected and embedded in other systems via “exo-structures” (Bunge 1996).

4.4 Organizational Differentiation and Policy Ecologies

A more refined form of institutional differentiation is treated here as organizational variation. This refers to specific organizational forms and makes use of ecological approaches to understand how “organizational species” adapt to specific habitats. Among the theories discussed above, it is the EOG and the Ecological Systems Theories in particular that are closest to such a perspective.

The higher granularity of this differentiation can be demonstrated using the example of the economic and the scientific subsystems. The organizational ecology of the business world related to politics has become much more complex, and the spectrum of organizational forms is nowadays much more diversified. It ranges from direct lobbying by individual companies to manager round tables, think tanks, forums, and even business movements.

A number of these organizations have the so-called boundary spanning functions by working at the intersections of systems, e.g., by mediating between politics and business or science and politics. Such cross-system organizations include various forms of organized business interests that mediate between business and politics. There are also specialized organizations at the interface between business, science, and politics, such as think tanks financed by private industry who also perform important lobbying functions. Even political parties can be seen as particular organizational species that perform complex mediation functions between civil society and the state (Mair 1994).

At the systems level, institutional and ecological perspectives should be integrated in a fruitful way. In this perspective, not only specialization patterns between social systems are important for the analysis, but also the differentiation into specialized organizational species. If one tries to model adjacent subsystems in a geometric way, then one quickly reaches an upper limit with more than three subsystems. Figure 3.3 therefore attempts to represent organizational forms associated with multiple subsystems by means of affiliation relations. The organizational forms indicate their institutional status by different scales of gray.

Fig. 3.3
An illustration explains the various subsystems and organizational forms of society. It also includes various institutional statuses like education, politics, media, economy, and science.

Societal subsystems and organizational forms

Full size image

Organizations that belong to two or more subsystems have boundary spanning functions and must mediate the different systemic orientations with each other. A public think tank in the field of civil society, for example, has to mediate between the different logics of politics, science, and not least the media system, which is becoming increasingly important also in policy processes (Waldherr 2017).

Macro perspectives stressing structural differentiation between the economy, politics, and science should be combined with the aforementioned ecological approaches. In each subsystem, specific organizations develop that are particularly adapted for specific tasks. Within the economic system, the corporate organization of large firms, the specific role of business associations, and the variety of finance institutions are important aspects of the social organization of this subsystem. In this respect, the institutional differentiation perspective within the VoC approach can be combined with an ecosystem perspective. Different organizational fields occupy different habitats with specific resources, conditions of survival, and interest positions. For instance, in the current research on climate policy, it is important to distinguish between the traditional and the green economy. Both have different material interests and not simply different preferences due to different belief systems.

Another instructive example is the science system, which differs strongly among countries: by the variety of specialized organizations and by the networks in which these organizations are embedded. If we are not only interested in beliefs or narratives that are dominating given organizational contexts, but also assume that good and evidence-based knowledge can significantly improve collective problem-solving, it is of great importance how the production process of policy-relevant knowledge is working in a country, and how this knowledge is distributed and is funneled into relevant policy venues. For instance, if we take independent public organizations in the German science system such as Max Planck and Fraunhofer Institutes or public universities, we assume a more reliable production of facts-based knowledge on climate change than knowledge produced by a think tank that is financed by the oil industry.

In this respect, the analyses made at the Max Planck Institute for the Study of Societies in the 1980s and 1990s of the German science system are very instructive (Hohn and Schimank 1990). In Germany, besides research at almost exclusively public universities, there is a very important field of non-university public research, which is carried out by Max Planck and Fraunhofer Institutes. In addition, as in other countries, much of the research takes place in the private sector (Grande and Häusler 1994). A further special organizational form in the science system is departmental research (Barlösius 2010). Overall, the organizations of the German science system form a specific organizational ecology that has specific effects in various science-related policy fields.

From a systemic and complexity-theoretical perspective, it is important to identify both the internal structural differentiation (endo-structures) of a system and its interconnectedness with the other subsystems such as politics and the economy (exo-structures). Especially in international comparative policy research, such structural comparisons promise to be particularly fruitful. As part of a larger international research network (Broadbent 2016), we are currently investigating how political, economic and scientific subsystems interact in Germany and Japan (Satoh et al. 2018). Our analysis shows that climate policy-making is embedded quite differently into the national systems of policy knowledge production and distribution. The reliance on evidence-based knowledge in German policy-making seems to be more intense and also more pluralistic than in Japan. Germany differs from Japan particularly by the fact that its organizational ecology is more diversified and civil society organizations are more strongly involved in the production and exchange of policy-relevant knowledge. We assume that this difference explains to a large part the differential performance of the two countries in this policy domain.

5 Conclusion

This chapter began with the observation that in recent decades the analysis of public policies has focused too much on the micro and meso levels and too narrow on governmental and conventional actor constellations. This narrowing of the analytical perspective gets particularly clear, when we compare the current situation with that of 50 years ago. At that time, a systemic view based on an inclusive concept of society included not only non-governmental actors, but also other societal subsystems were also regarded as important context factors for the processing of societal by means of public policy.

Since the 1980s, policy analysis has become increasingly state-centered in its main orientation, especially in light of the “bringing-the-state-back-in” movement. Only governance broadened the analytical perspective again by including non-state actors and non-hierarchical coordination mechanisms in the analysis of policy processes.

However, this enhancement is not enough. This chapter argues for a systematic and comprehensive inclusion of societal factors in the analysis of public policies whereby particularly the influences from the macro level play an important role. From a conceptual point of view, therefore, it was first clarified which ideas about society and governance can be combined in a public policy perspective. In a comparison of two dozen theories, an immense opportunity space of theories and approaches was outlined, as to how societal aspects can influence policy-making. Based on the conceptual and metatheoretical analysis, a new perspective was presented to combine several approaches. In this respect, it is particularly important to embed the analysis of actor constellations into structures of societal differentiation on the macro level. In addition, also actor positions with regard to specific network roles should be taken into account. Institutional differentiation structures between the state and the private sector should be linked on the one hand with structures of functional differentiation and on the other hand with structures of relational differentiation in order to be able to analytically grasp the complex social embedding and entanglement of social subsystems in modern policy processes. Ultimately, it is argued that approaches and analytical elements of complexity theory should be made fruitful for the analysis of public policies.