Keywords

1 Introduction

Governments initially used the Web 2.0 social media mainly in order to disseminate information to the public about their activities, services, policies and plans, influenced strongly by the one-way communication patterns of the Web 1.0 era. Then they started exploiting the extensive two-ways communication capabilities provided by the social media, in order to collect from citizens useful information, knowledge, opinions and ideas concerning the problems and needs of modern societies, and the public policies they were designing and implementing for addressing them, and more recently in order to apply crowdsourcing ideas in the public sector context and promote ‘citizensourcing’ [12–14, 23, 33]. In this direction governments first used their own accounts in various social media, in which they provide information about specific social problems and public policies, and solicit citizens’ feedback on them, aiming to conduct ‘active’ forms of citizensourcing [10, 17, 32, 33]. Recently, they attempt to extend these practices beyond their own social media accounts, in order to exploit the extensive public policy related content developed by the citizens in various political forums, blogs, news websites, and also in various Twitter, Facebook, etc. accounts, without any stimulation from government, aiming to combine the above active forms of citizensourcing with more ‘passive’ ones [3, 11, 25]. This has been driven by the social media monitoring (SMM) practices developed in the private sector, in order to collect opinions, complaints and questions that have been posted in various social media (e.g. forums, blogs, Twitter, Facebook, news feeds, etc.) about their products and services, which are processed and used for improving products and services, and also for designing communication strategies [15, 22, 37]. Government agencies, which have been traditionally monitoring citizens’ opinions and attitudes towards their policies and plans (e.g. through surveys based on representative citizens’ samples), and also relevant articles in the ‘traditional’ media (e.g. newspapers), show an increasing interest in SMM for collecting useful information, knowledge, opinions and ideas from the citizens concerning their problems and needs, and also existing and planned public policies.

However, the above constitute big innovations in the government – citizens communication and the policy formulation processes and practices of government. Therefore it is important to evaluate them from various perspectives, in order to learn from them as much as possible, identify their strengths and weaknesses, and also address the latter and strengthen the former, and finally achieve higher levels of effectiveness and maturity of these highly innovative practices. This paper makes a two-fold contribution in this direction:

  1. (i)

    It develops a framework for evaluating such passive social media citizensourcing innovations, which is based on theoretical foundations drawn from previous research on crowdsourcing (see Sect. 2.3) and innovation diffusion (see Sect. 2.4).

  2. (ii)

    It uses this framework for the evaluation of three pilot applications of a novel method of government passive citizensourcing based on SMM (described in more detail in [11, 25], which has been developed as part of the European research project NOMAD (“Policy Formulation and Validation through Non-moderated Crowdsourcing” – for more details see www.nomad-project.eu/), partially funded by the “ICT for governance and policy modeling” research initiative of the European Commission.

The paper is structured in seven sections. In the following Sect. 2, the background of our research is presented. The abovementioned novel method of government passive citizensourcing is outlined in Sect. 3. Then in Sect. 4 the proposed evaluation framework is presented. Our research method is described in the subsequent Sect. 5, followed by the evaluation results, which can be found in Sect. 6. Finally, the conclusions are summarized in Sect. 7.

2 Background

2.1 Social Media in Government

Social media have been initially used by private sector firms, mainly in their marketing and customer service activities, and then adopted and utilised by government agencies as well, in order to take advantage of the large numbers of users that social media attract, and the unprecedented capabilities they provide to simple non-professional users for developing, distributing, accessing and rating/commenting various types of digital content, and also for the creation of on-line communities [4, 5, 11, 13, 14, 30].

There has been considerable research analysing the potential of social media for supporting, enhancing and transforming critical government functions, which has identified significant opportunities they provide to government agencies: (i) to increase citizens’ participation and engagement, providing to more groups of modern societies a voice in debates on public policies development and implementation; (ii) to promote transparency and accountability, and reduce corruption, by enabling governments to open up large quantities of data concerning their activity and spending; (iii) to drive important innovations in both the internal operations of government agencies and the ways they communicate and interact with the public outside their boundaries; (iv) to collect useful information and knowledge from the citizens’ concerning the complex problems and needs of modern societies; (v) to exploit citizens’ creativity in order to develop innovative solutions to the serious and complex problems that modern societies face, and in general to apply crowd-sourcing ideas in the public sector (citizensourcing); (vi) to proceed to public services co-production with citizens, enabling government agencies and the public to design jointly government services [4, 5, 10, 17, 27, 30, 32, 33].

The first generation of social media exploitation by government agencies included the creation and operation of their own accounts in several social media, the publication of policy-related content to them (concerning specific social problems and public policies, in order to stimulate relevant discussions with citizens), and the analysis of citizens’ interactions with this content, such as views, likes, retransmissions, textual comments, etc. (active citizensourcing); this is usually performed manually, however there is a trend towards higher levels of automation, based on applications accessing various social media platforms through their application programming interfaces (APIs) [10, 17, 32, 33]. As mentioned in the Introduction, recently a second generation of more passive social media exploitation by government agencies has emerged, which is based on adaptations of private sector SMM practices. In particular, it includes collection and processing of policy-related content created by citizens beyond government social media accounts, in various political forums, blogs, news websites, and also in various Twitter, Facebook, etc. accounts, and sophisticated analysis of them, through a central system (passive citizensourcing) [3, 11, 25].

2.2 Social Media Monitoring

SMM, defined as ‘the continuous systematic observation and analysis of social media networks and social communities’ [16], has been initially developed for and used by private sector firms, in order to address their fundamental need for listening to their existing and potential customers, in a better and more efficient way than the traditional methods used for this purpose, such as surveys, by exploiting the wealth of user-generated content available online [15, 16, 22, 37, 39]. SMM is based on ICT platforms, which enable listening to the social media users, accessing real customers’ opinions, complaints and questions, at real time in a highly scalable way, analysing and measuring their activities concerning a specific brand, or an enterprise, or specific products and services, and processing of this information; this leads to valuable insights for firms, regarding how customers view them, their services and solutions, and also their competitors.

However, there is a lack of frameworks for the evaluation of SMM platforms, practices and approaches in general, which is quite important for achieving higher levels of effectiveness, maturity and diffusion. There is only a framework for evaluating SMM tools proposed by Stavrakantonakis [39], which is oriented towards their private sector use. It comprises a set of evaluation criteria that can be used to analyze and assess the functionality of SMM tools from three perspectives: the concepts they implement (data capture and analysis, workflow, engagement – reaction to posts, and identification of influencers), the technologies used (listening grid adjustment, near real-time processing, integration with third party applications, sentiment analysis, historical data) and the user interface (dashboard, results’ export) they provide.

Quite limited is the previous literature concerning the use of SMM by government agencies. Bekkers et al. [3] investigate SMM practices of four Dutch public organizations, examining the goals of SMM, its operation and effects. They discriminate between four types of monitored citizens’ electronic discussion media based on two criteria: the level of perceived privacy (low or high), and the type of issues discussed (personal or societal); they recommend that more ethical questions arise, so government agencies should be more careful and also transparent, if the citizens’ electronic discussion media monitored are characterised by higher perceived privacy and host discussions on more personal issues. There is a lack of frameworks for evaluating the use of SMM by government agencies from various perspectives, which would be quite important for the development of knowledge and effectiveness in this area. Such an evaluation framework should include government SMM assessment from various perspectives. Since this SMM aims to support a citizensourcing innovation, it is necessary to assess it both from the crowdsourcing perspective (see following Sect. 2.3), and also from the innovation diffusion perspective (see Sect. 2.4).

2.3 Crowdsourcing

The great potential of the ‘collective intelligence’, defined as a ‘form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills’ [24], to contribute to difficult problem solving and design activities has lead to the emergence of crowdsourcing and its adoption, initially in the private sector, and subsequently (still experimentally) in the public sector as well. Crowdsourcing is defined as ‘the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call’ [19] or as ‘a new web-based business model that harnesses the creative solutions of a distributed network of individuals’, in order to exploit ‘collective wisdom’ and mine fresh ideas from large numbers of individuals [9]. While the use of the collective intelligence of a large group of people as a help for solving difficult problems is an approach that has been used for long time [8, 9, 19, 20, 40], it is only recently that crowdsourcing started being widely adopted by firms as a means of obtaining external expertise, accessing the collective wisdom and creativity resident in the virtual crowd.

Crowdsourcing started being applied initially in the creative and design industries, and then it expanded into other private sector industries, for solving both mundane and highly complex tasks. Recently it has started being applied by government agencies as well, usually exploiting the capabilities offered by social media for this purpose, as they enable the wide and low cost application of the ‘crowdsourcing’ ideas by government agencies, termed as ‘citizensourcing’ [7, 18, 27, 28, 33, 34], which can be highly useful for public policy making. Social media platforms enable government agencies to mine useful fresh insights into social needs and problems, and ideas concerning possible solutions to them, new public services or improvements of existing ones, or other types of innovations, from large numbers of citizens. Initially it had the form of ‘active crowdsourcing’, in which government agencies have active role, posing particular social problems, questions or public policy directions, usually in their own social media accounts, and soliciting relevant information, knowledge, opinions and ideas from citizens. Recently a second form of ‘passive crowdsourcing’ is emerging, in which government has more passive role, collecting through SMM content on a specific topic or public policy, which has been freely generated by citizens in various sources, and performing sophisticated processing of it [11, 25].

However, previous literature notes that the outcomes of crowdsourcing are uncertain, and identifies some important critical success factors, such as the existence of sufficient active crowd, and their quality from a human capital viewpoint, and also some inherent risks, such as digital divide related problems and participation inequalities (i.e. under-representation of some stakeholder groups, and over-representation of some others), and possible manipulation of the crowd [1, 6, 21, 38]. Therefore it is important to evaluate the passive citizensourcing through SMM from a crowdsourcing perspective, assessing the existence of the above critical success factors and risks.

2.4 Diffusion of Innovation

As mentioned above the passive citizensourcing through SMM constitutes a big innovation in the government – citizens communication and in the policy formulation processes and practices of government agencies, so it is important to examine it from this perspective as well, and assess to what extent it has the fundamental preconditions for a wide diffusion. Extensive research has been conducted concerning diffusion of innovations, in order to understand it better and identify factors that favour it [29]. One of the most widely recognised and used theories of innovation diffusion is the one of Rogers [36], which has been extensively employed for analyzing ICT-related innovations in both the public and the private sector [2, 26, 35]. According to this theory, there are five critical characteristics of an innovation that determine the degree of its adoption:

  1. (i)

    Relative Advantage (=the degree to which an innovation is perceived as better than the idea, work practice or object it supersedes);

  2. (ii)

    Compatibility (=the degree to which an innovation is perceived as being consistent with the existing values, past experiences, and needs of potential adopters);

  3. (iii)

    Complexity (=the degree to which an innovation is perceived as difficult to understand, implement and use);

  4. (iv)

    Trialability (=the degree to which an innovation may be experimented with on a limited scale basis);

  5. (v)

    Observability (=the degree to which the results of an innovation are visible by the external environment).

Therefore it is important to assess to what extent the passive citizensourcing through SMM, viewed as an innovation, has the above characteristics that determine the degree of its adoption and diffusion.

3 A Method of Government Passive Citizensourcing Using SMM

A method of government passive citizensourcing based on SMM has been developed as part of the abovementioned European research project NOMAD [11, 25], aiming to support the formulation of public policies taking into account relevant citizens’ knowledge, opinions and ideas. A brief description of it is provided in this section. It consists of four steps:

  1. (i)

    The first step is to build the ‘domain model’, which is an ontology-based representation of the objects of the “world” (domain) we intend to intervene in through a policy (e.g. energy domain, education domain, health domain). The main entities-terms of this are inserted, as well as relations among them, in a tree structure, using a graphical modelling tool.

  2. (ii)

    Then the second step is to build the ‘policy model’, which is a representation of the public policy we want to collect relevant content about from the social media; it consists of a number of ‘policy statements’ associated with one or more nodes of the policy model, and for each of them some positive or negative ‘arguments’. A policy model is inserted on a policy model (used as a basis for it) using the above graphical modelling tool.

  3. (iii)

    Upon the completion of the models, the user provides a list of social media sources (e.g. blogs, news websites, and also Twitter, Facebook, etc. accounts) which are going to be crawled, in order to find relevant content about the topic or public policy of interest (=places on the web that according to our previous knowledge might contain relevant user-generated content, i.e. where citizens are likely to have expressed relevant opinions and suggestions).

  4. (iv)

    The defined sources (in step iii) are searched against the above domain and policy models (defined in steps i and ii respectively), and the collected content undergoes sophisticated processing using opinion mining techniques: initially opinions and arguments are extracted, and then sentiment analysis of them is performed (the processing is described in more detail in [11]. The results are presented to the user in visualised form; a typical results’ visualisation screen (see Fig. 1) includes:

    Fig. 1.
    figure 1

    A typical results’ visualisation screen

    Full size image

    • In the upper left part of the screen is shown an estimation of the volume of discussion and the cumulative sentiment for all the elements of the domain or policy model (according to the selections made just above it), the former being visualised through the height of the corresponding rectangle, and the latter through its colour (with the green colour denoting positive sentiment, and the orange denoting negative sentiment).

    • For the above model, or for a selected element of it, below (in the lower left part of the screen) is shown the distribution of the volume of discussion over time and also across age groups,

    • while in the upper right part is shown a word cloud depicting the most frequent terms-topics discussed online (coloured according to the corresponding sentiment),

    • and in the lower left part we can see a list of text excerpts from the sources with relevant content (concerning the selected model or element of it).

Also an ‘audience comparative view’ can be provided, which shows differences among selected different age, gender or education groups, or differences over time, in the discussed topics (concerning volumes of discussion and sentiment, terms-topics frequencies).

4 An Evaluation Framework

Having as theoretical foundation the background presented in the Sect. 2, a framework has been developed for evaluating passive citizensourcing through SMM, which is shown in Table 1 (in the first column). As this constitutes a big citizensourcing oriented innovation in the government – citizens communication and the policy formulation processes and practices of government agencies, it is necessary to assess it from both the crowdsourcing perspective and the innovation perspective.

Table 1. Average ratings for all questions - evaluation metrics
Full size table

Therefore the first evaluation perspective is the crowdsourcing one, which examines to what extent it is useful/beneficial for assessing the feelings and attitudes of citizens concerning a prospective or existing policy, and for identifying new relevant issues/topics emerging in the society. Furthermore, it examines to what exent exist the main critical success factors and risks of crowdsourcing which have been identified by previous relevant research [1, 6, 21, 38]. In particular, it examines to what extent the active crowd who created the content collected from the monitored social media is quantitatively sufficient, and also to what extent the views, opinions and suggestions it expresses online are representative of trends and opinions prevailing in the society as a whole, are non-biased and non-manipulated, and also of good quality; furthermore, to what extent the results produced are useful for the formulation of public policy for the specific domain, and for other domains.

The second evaluation perspective is the innovation one, examining to what extent passive citizencourcing through SMM has the characteristics that according to Rogers diffusion of innovation theory [36], lead to high levels of adoption and diffusion (see Sect. 2.4). In particular, it examines to what extent it offers a relative advantage over the existing alternatives for the same purpose, it is easy to use, it is compliant with the policy formulation processes as they are applied in Europe, and also with the needs, the mentalities and the values of the people designing and applying public policies, and it has trialability (i.e. can be tried and experimented on a limited scale basis). We have not included the observability, as such SMM methods due to their nature are not meant to be visible by the external environment.

5 Research Method

Three pilot applications of the above method of passive citizensourcing through SMM have been conducted as part of the NOMAD project, and evaluated using the evaluation framework presented in the previous section. Since this method is intended to be used not only by government agencies in their policy formulation processes, but also by other public policy stakeholders (who want to know citizens’ opinions, sentiments/attitudes and suggestions concerning various policy relates topics before submitting relevant policy proposals to government) as well, two of these pilots were carried out by government organizations: the Greek and the Austrian Parliament; while the third one was carried out by an important non-government policy stakeholder in the health domain: the European Academy of Allergy and Clinical Immunology. A detailed scenario has been designed for each pilot, which describes how this method can be used for supporting the specific policy formulation objectives. The topics of these pilot applications were selected so that on one hand they reflect current debates and interests of their owners, and on the other hand cover quite different and diverse domains.

The first pilot application was conducted by the Greek Parliament, and concerns the legal framework for energy management, i.e. the “Greek strategy for energy planning”. The objective of the pilot application was to assess public opinion and attitude/sentiment against this prospective legislation, and based on the collected information to propose amendments to the existing legislation. The second pilot application was conducted by the Austrian Parliament, and aimed to monitor the on-going public debate on a ‘freedom of information act’, i.e. a coherent legal basis for “open government information in Austria” and the open government data policies at large. The third pilot application was oriented towards a more scientific policy topic: it was conducted in collaboration with the European Academy of Allergy and Clinical Immunology (EAACI) in order to assist it in discovering the public stance against “allergy diseases and immunotherapy”, and based on this information to design policies for raising awareness in this area, and also to formulate relevant policy proposals to be submitted to the government.

In particular, for each pilot the following process was followed:

  • Initially the detailed SMM use scenarios in the selected thematic domain have been defined in cooperation with the ‘owners’ of the pilot, and then the domain and policy models required for data crawling were created by them, and finally a list of targeted social media sources (which, according to previous knowledge of the pilot owners might contain relevant user-generated content) has been specified.

  • After the above preparation, the owners initiated the crawling the specified sources against the corresponding domain and policy models, and the processing of the collected content.

  • Then the personnel of the owner organization who participated in this pilot inspected the results, understood them in detail, assisted by members of our research team, and used them in order to draw conclusions about citizens’ opinions, sentiments/attitudes and suggestions concerning the topic of each pilot.

  • Finally, for each pilot an evaluation focus group discussion was organised, which attended by personnel of the owner organization who were involved in this pilot, and also other invited persons who had relevant knowledge and experience (e.g. for the pilots of the Greek and Austrian Parliament were invited advisors and assistants of Members of the Parliament and journalists specialised in the corresponding domain; for the EACCI pilot were invited doctors, experts and journalists specialised in allergy and clinical immunology). During this focus group discussion the proposed method was introduced to the audience, and particular applications of it with the corresponding results were showcased. Then the attendees had the opportunity to interact with the ICT application, performing some predefined tasks, under the observation of organizers’ staff, who supported them in completing these tasks, and recorded any comments or difficulties.

In order to collect evaluation data from the attendees of these focus group discussions we used a combination of both qualitative and quantitative techniques. According to research methodology literature (e.g. [31, 34, 42], the qualitative techniques allow a more in-depth examination of a phenomenon of interest, and therefore the generation of a deep knowledge about it, which is not limited to a predefined number of variables, enabling a better and richer understanding of ‘why’ and ‘how’ things happened; on the contrary, the quantitative techniques focus on a predefined number of variables and enable condensing/summarizing a large quantity of evidence in a few numbers, which makes it easier to draw conclusions. For these reasons in each of these focus groups we conducted initially qualitative discussions focused mainly on the questions of the two perspectives of our evaluation framework shown in Table 1, in order to identify positive and negative elements along each of them, and gain a deeper and richer understanding of why the attendees perceive a low or high level of it. Then we asked the attendees to fill an evaluation questionnaire, which included the questions of the two perspectives of our evaluation framework: these questions were converted to positive statements, and the respondents were asked to provide the degree of their agreement/disagreement with each of them in a five-levels scale (1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree), which condenses/summarizes all positives and negatives identified with respect to this dimension. The above qualitative discussions were recorded with the consent of the participants, and then transcribed and coded manually using an open coding approach [31]; the data collected through the questionnaire were processed using Excel.

6 Results

In Table 1 (in the second column) are shown the results from the processing of the quantitative data collected through the questionnaire (average ratings for all questions - evaluation metrics).

With respect to the crowdsouring perspective from Table 1 we can see that the respondents perceive as high to very high (closer to the former) the usefulness of this method of passive citizensourcing through SMM for evaluating citizens’ feelings and attitudes against a prospective or existing policy (average rating 4.17), and also moderate to high (closer to the latter) its usefulness for identifying emerging new relevant issues/topics in the society (3.74). In the focus group discussions there was an overall agreement that this method provided a time and cost efficient channel to assess citizens’ attitudes and feelings on a policy related topic of interest, both from a quantity (e.g. the volume of discussion about it) and a quality (e.g. the sentiments, the most popular topics within relevant discussions) viewpoint, which is better and less expensive than the traditional citizens’ surveys conducted by government agencies. However, they mentioned the risk of misusing such SMM results for promoting individual interests, by focusing selectively on some of the results that support their own positions, and hiding some others. Furthermore, they also mentioned the risk of monitoring citizens’ postings that are perceived by them as private, which would seem to them as an intrusion into their private sphere. In general, it was concluded that the benefits for society from the use of any web monitoring tool by government depend critically on how this technology is utilised and how its results are exploited. Furthermore, the participants in the focus group discussions mentioned that this method enables to some extent the identification of emerging new relevant issues/topics concerning a particular domain or public policy of interest, however not to the extent they would expect. The word cloud does not seem appropriate for the early identification of new issues, topics or tendencies, as it is dominated by the well established topics-terms (shown with big character sizes, as they are more frequently mentioned by citizens), while the new ones are hardly visible (only some of them are shown with much smaller character sizes, as they are much less frequently mentioned by citizens); new issues, topics or tendencies can be identified mainly by reading the list of text excerpts from the sources with relevant content (lower left part of the typical results’ visualisation screen – see Fig. 1). In order to have improvement in this direction two suggestions were made: (a) to add the functionality of temporarily removal out of the word cloud the most frequent topics-terms it includes (shown with big size), so that other less frequently mentioned topics-terms become more visible; (b) to process further the above text excerpts using various opinion mining techniques, in order to automatically identify new topics-terms.

Also, the respondents find the quantity of the active crowd who create the content collected from the monitored social media as moderately sufficient (3.26). Also, they believe that the views, opinions and suggestions it expresses online are to a moderate to large extent representative of the trends and opinions prevailing in the society as a whole (3.40); however, they perceive their quality as low to moderate (2.89), and also that only to a small to moderate extent (but for both closer to the latter) free of bias and manipulation (2.63). Despite these drawbacks, the respondents perceive as moderate to high (being closer to the latter) the usefulness of the results provided by this SMM method for the formulation of public policy for the specific domain, or other domains (3.86 and 3.69 respectively). In the focus group discussions there was skepticism about the representativeness of the citizens’ groups who produce the content collected from the monitored social media, and also its reliability (i.e. whether it is non-biased, non-manipulated and of good quality). There was wide agreement that the selection of the social media sources to be monitored is of critical importance in this respect; it was emphasized that it is necessary to select a representative set of high reliability and quality social medial sources to be monitored. Furthermore, a suggestion that emerged was to provide the capability to isolate the results from specific sources (e.g. from sources of a specific political orientation, or corresponding to a particular professional group), or even access the individual source from which a term-topic from the word cloud or a relevant text excerpt originates. It was stressed that it is of particular importance in order to be able understand an opinion, argument or suggestion, or to assess a sentiment, to know the context in which it has been expressed.

Finally, with respect to the innovation perspective from Table 1 we can see that the respondents perceive that this method offers high relative advantage over the existing alternatives for the same purpose (3.94), and has high degree of trialability in a small scale (3.89); also, it has moderate to high compatibility with the policy formulation processes as applied in Europe, and with the needs, the mentalities and the values of the people designing and applying public policies (3.54 and 3,57 respectively). It has been confirmed in the focus group discussions that the potential impact from the integration of the proposed methods and tools in the policy formulation process is positively perceived, as it offers significant relative advantages over the citizens’ surveys, which is the main alternative for the same purpose currently in use by governments. It has been mentioned that surveys can neither capture public sentiment nor provide detailed information (e.g. like the frequently mentioned terms-topics, relevant text excerpts provided by this method) about a domain or public policy of interest. However, the ease of use of the whole method is perceived as moderate (3.02). In the focus group discussions it was mentioned that the application of this method does not seem to be easy. The main reason for this is the need to build complex models of the specific domain and also the particular policy we are interested in, which requires much time and effort. As a possible solution for this was suggested the highest possible re-use of existing domain ontologies or vocabularies as a basis for this (and probably add or subtract entities-terms), so the functionality of the ICT application should be enriched in order to provide such import capabilities.

7 Conclusions

Government agencies initially adopted simpler forms of social media exploitation, however latter they started experimenting with more sophisticated ones. Furthermore, while initially they used social media mainly in order to disseminate information to the public about their activities, services, policies and plans, then they started using these highly popular platforms in order to collect from citizens useful information, knowledge, opinions and ideas concerning the problems and needs of modern societies, and the public policies they were designing and implementing for addressing them. Recently they attempt to use the social media in order to apply crowdsourcing ideas in the public sector context, and make ‘citizensourcing’ oriented innovations in their government – citizens communication and the policy formulation processes and practices. It is therefore important to conduct systematic research in order to create effective methods of using the social media in various ‘active’ or ‘passive’ manners for conducting citizensourcing, and then to evaluate them from multiple perspectives based on ‘real-life’ pilot applications, in order to identify their strengths and weaknesses, design improvements of them, and finally achieve higher levels of effectiveness and maturity of them.

This paper makes a two-fold contribution in this direction. Initially it develops a framework for evaluating passive citizensourcing innovations through SMM, which is based on theoretical foundations drawn from previous research on crowdsourcing and innovation diffusion. Then this framework is used for the evaluation of three pilot applications of a novel method of government passive citizensourcing based on SMM, which has been developed as part of the European research project NOMAD.

These evaluations have provided some first evidence that this method of passive citizensourcing using SMM can provide considerable support for public policy making, enabling the low cost and fast assessment of citizens’ feelings/attitudes concerning a prospective or existing policy, and also the identification of emerging new relevant issues/topics in the society, contributing to the improvement of the ‘dynamic capabilities’ [41] of government agencies (with respect to their ‘sensing’ component). However, this method poses some risks, associated with the misuse of it for promoting individual interests (by reporting selectively only some of its results which are in desired directions, and hiding some others), and also with the possible intrusion into citizens’ private sphere (so in the SMM it is necessary to avoid sources in which contributors perceive their postings and discussions as private). It has been concluded that the quantity and the representativeness of the ‘crowd’ who created the collected content, on which the results have been based, was satisfactory, but there was some scepticism about the quality and reliability of this content (e.g. due to possible bias of its creators and manipulation). Therefore critical success factor of this method is the selection of an extensive and representative set of high reliability and quality social medial sources to be monitored. However, despite the above drawbacks the overall assessment seems to be positive: this method can provide considerable support for the design of public policies.

Furthermore, this first evaluation provides evidence that this method of passive citizensourcing through SMM viewed as an innovation has most of the preconditions proposed by the theory of innovation diffusion of Rogers [36] for a wide adoption and diffusion. In particular, it has been concluded that it offers strong relative advantage over the existing alternatives for the same purpose, and has high levels of trialability (i.e. can be tried and experimented on a limited scale basis); also, it has a good level of compatibility with the policy formulation processes, and with the needs, the mentalities and the values of the people designing and applying public policies. However, this method does not seem to be easy to use, as it requires building complex models of the specific domain and also the particular policy we are interested in.

The research presented in this paper has interesting implications for research and practice. With respect to research it has developed an evaluation framework that can be used as a basis for the analysis of the emerging new methods and practices of ‘citizensourcing’ (both active and passive ones) based on social media (possibly enriched with new perspectives, and adapted to the specificities of each particular method and practice), and the identification of their strengths and weaknesses. Furthermore, it has revealed specific weaknesses of the passive citizensourcing through SMM, so it provides guidance on future directions of research for addressing them. With respect to practice, our results indicate that government agencies should not limit themselves to simpler forms of social media use, but also make use of more sophisticated ones as well, and exploiting for citizensourcing purposes not only their own social media accounts, but also the extensive public policy related content developed by the citizens in various Web 2.0 sites, without any stimulation from government. Further research is required for evaluating from different perspectives the emerging methods of passive and active citizensourcing by government agencies in various contexts (e.g. various types of government agencies and other public policy stakeholders, and various types of policies), in order to understand better the benefits they can provide, and also the risks and challenges they pose and their limitations; also research needs to be conducted concerning their combination, and also their integration in the government – citizens communication and the public policy formulation processes, and their impacts.