1 Introduction

Studying visitor’s prospective is a useful tool to improve the communication of scientific researches and technological innovations. INGV’s staff utilized several methodological approaches for both permanent museums (Loomis 1987; Bitgood 1988; Hooper-Greenhill 1994; Aguiari-Amici 1995; Bailey 1995; Kotler and Kotler 1998; Avorio 1999; Bollo 2004, 2008, 2009; Zuliani 2012 and references within; Korsakien et al. 2019; Dinçer et al 2020) and temporary events and exhibitions (Getz D. 1991; Crompton and McKay 1997; D'Addezio et al. 2014; Lanza et al. 2014; Ivkov et al. 2015; Musacchio et al. 2015a; Musacchio et al. 2015b; Morgan 2015 and references within). These studies have shown that, to evaluate visitor perception effectively, it is important to know the reasons behind their visit, their expectations and their satisfaction during and after the experience.

The Istituto Nazionale di Geofisica e Vulcanologia (INGV) is one of Italy’s major public research institutes and the largest European organization dedicated to geophysics and volcanology. Its mission is to study and monitor geophysical phenomena, mainly earthquakes and volcanic events, as well as spreading scientific culture and developing a risk-prevention behavior. INGV also takes care of organizing educational programs meant for schools and the general public.

INGV experience about visitor’s feedback, performed over the years, have included permanent exhibitions, such as the Historical building of Osservatorio Vesuviano located on Vesuvius volcano (Avvisati et al 2015; Avvisati and Uzzo 2000), and temporary exhibitions, itineraries and projects (D'Addezio et al. 2014; Lanza et al. 2014; Rubbia et al. 2014; Musacchio et al. 2015a, b; Amici et al., 2018; D’Addezio 2019). These studies focused on scientist’s engagement while sharing knowledge about Earth Sciences, on the participants’ response and the relationship between the scientific message and the learning approach. In order to contribute to this approach, here we present an analysis of points of view on the “INGV Open Day” held on January 20th 2019 in the INGV Rome headquarter. We designed three different questionnaires: one for the general public, one for young visitors and a combined one for geologists, teachers and other stakeholders. The work first illustrates the framework and the contents of the Open Day, then the methodology used for the investigation as questionnaires design, data analysis and questionnaires results. Finally, we present feedback from the appreciation surveys.

The results are precious information, useful to improve the effectiveness of science communication during INGV outreach activities. Furthermore, these studies contribute to the planning of a systematic approach for any similar context.

1.1 The INGV open day held on January 20th

On January 20th 2019, the INGV organized an “Open Day” on “earthquakes between memory and prevention” to respond to the needs and the requests of the community for more information on issues regarding our planet and to engage society in a correct, straightforward and efficient way to communicate about scientific research and technological innovations. The event was designed to increase the awareness about Earth sciences and research activities and to promote public safety, as well as generate appeal to the scientific culture. The event was part of the "Regional Seismic Literacy Day" (Italian L.R. Lazio18 dicembre 2018, n. 12), established to inform on earthquake protection. During the Open day, INGV researchers and technicians guided almost 900 visitors to introduce them to useful tools applied to reduce the losses caused by natural hazards and especially earthquakes. About 300 children and young people were involved in educational workshops on: earthquakes, volcanoes and the environment. The day started with the discussion panel "Earthquakes, the role of prevention and seismic literacy” where scientific experts and authorities addressed a specialized public and stakeholders. The panel was followed by more divulgative public seminaries on earthquake. Guided tours were also offered to:

  • The seismic monitoring and tsunami alert room (Tour 1);

  • Scientific laboratories on experimental geophysics and volcanology, High Temperature High Pressure, Fluid Geochemistry, and Paleomagnetism (Tour 2);

  • INGV research and monitoring activities (Tour 3).

Specific attention was given to initiatives dedicated to children including educational games and hands-on laboratories, on earthquakes and volcanoes, such as Escape Volcano and, specifically for children under 5 years of age, Volcano and origami and Tales of the Planet.

2 Methodology

The study applied both quantitative and qualitative methods of the analysis. Quantitative approach consisted of three questionnaires for, respectively, children, adult and invited stakeholders, geologists in particular. We elaborated closed-ended questions, with multiple-choice, “yes” or “not” choice and with rating scales. Qualitative evaluation were collected adding to the questionnaires open-ended questions, in which visitor provided, in their own words, personal suggestions and spontaneous opinions on the event (Ritchie et al. 2013).

2.1 Questionnaires design

During the Open day we have organized activities planned for different audience targets. To better understand the participants’ experiences, we designed three questionnaires for: adult visitors, young visitors and stakeholder visitors (basically geologists and teachers). The questionnaires were handed out to the participants at the end of the activities. Both closed-ended and open-ended questions were included in the questionnaires, in order to assess the level of satisfaction, interest, comprehension and organizational aspects. Other questions regarded general data such as audience’s age, gender and education; all these informations are useful to describe the demographic and cultural features of the visitors. The questionnaires for general public and young visitors were developed with a similar method and are composed by 7 and 6 questions, respectively. The questions investigated the level of interest and familiarity in attending similar events, and the general satisfaction for specific activities (e.g. a visit to scientific laboratories, meeting with researchers or hands-on experiments). These forms also contained a self-assessment section, in which each participant had the opportunity to test the knowledge they acquired during the event. The questionnaire for the stakeholders contained 8 questions, focused on quality of content, level of understanding, level of completeness, level of clearness, coherence to the general theme and quality of the oral presentations. Stakeholders were also asked to indicate activities strengths and weaknesses, their will to recommend similar events to others visitors, and to give a rate to the experience on a scale from 1 to 10. A final, open-ended question, offered the opportunity to convey additional suggestions, comments, or criticisms.

2.2 Analysis of data

We welcomed about 900 participants to the Open Day, one third of whom were children. At the end we collected a total of 133 questionnaires, 52 from adults, 50 from children and 31 from stakeholders. Accordingly, 15% of the visitors completed the questionnaire, 17% of which were children. Data processing was carried out with the IBM SPSS statistics software package (v. 23 for MS Windows). Multiple Correspondence Analysis (MCA) was performed with XLSTAT Version 2016.02.27444, in order to identify groups of individuals with similar profiles within their answers to the questions and the associations between observed variables.

A multiple linear regression analysis was carried out to identify relations between explanatory or independent variables and response or dependent variables. The overall evaluation from stakeholders was selected as dependent variable. This was compared with different independent variables such as the ease of comprehension, coherence of topics, clarity and completeness of contents, effectiveness of the used support, satisfied expectations, appropriate explanations, all of which have been fully included in the analysis. Finally, a multiple linear regression examined how relations between the independent and dependent variables can be used to measure how a variation of the overall evaluation can be justified by a shift of the independent variables.

3 Questionnaire results

3.1 Adult visitor

The age of respondents ranged from 24 to 73 years, with an average of 45. Male visitors were slightly more numerous, 58%; 16% declared education to post graduate level, 59% to degree level, 23% to diploma level and 2% to middle-school diploma level.

To the question "have you ever visited a scientific exhibit or participated in educational itineraries like today?” 67% of the respondents confirmed that they had previously attended similar events. About 27% had visited events in the past year, 23% in the past 2 years, and 17% more than 2 years before. The remaining 33% had never attended similar scientific exhibits or educational tours.

All visitors answered positively to the question "Have you enjoyed today's visit?”. The comments can be further classified into: "very much" (86.3%) and "somewhat” (13.7%). No negative feedback, such as “little" or "not at all", were recorded.

Meeting researchers was mostly appreciated. 91% of respondents considered the seminars to be very interesting and useful. They were regarded as easily understood, although the organization received mixed reviews considering it to have been well or fairly well organized, respectively with 52% and 32%. The information they received were felt completely and reasonably understandable for 80% and 17%, respectively.

The feedback on guided tours was generally very positive.

In relation to Tour 1 at the seismic monitoring and tsunami alert room, almost all the answers were positive, indicating "very much" for all aspects (interesting, comprehensible, useful, well organized). Between 5 and 20% replied with a "somewhat" positive feedback, and only 6% evaluated the tour to be "little" well organized. Nobody replied with a "not" well organized at all.

Tour 2 on scientific laboratories received a very good feedback. The 83% of respondents have considered it very interesting and very comprehensible, 78% thought it was very useful, and 50% very well organized; 4% considered it poorly understandable and 9% poorly organized. Nobody replied "not at all" for either aspect.

Similar results have been obtained from Tour 3 on exhibition spaces. Most of the answers agreed "very much" across all aspects, including degree of interest, ease of comprehension, usefulness, and good organization. Between 9 and 15% of answers feel into the "somewhat" category and, just for the organizational aspects, 21% returned "little" as response.

The highest number of unanswered questions were on Tour 2 followed by Tour 3, then Tour 1 and the meeting with researchers.

From the question that concerned about the awareness of the topics addressed, resulted that 86% were already aware of the topics; "television" (40%), “specific studies” (36%) and “reading educational material” (32%), were the most common sources of previous information. Just over a third, 36%, of respondents felt they had a “great” experience, 54% had learned “somewhat” and only 10% had learned “little”; 90% stated that they had learned what they expected.

To the question "Was the amount of information too much?" the answers highlighted that 61% even wished to learn more and 39% found themselves to be "somewhat" enthusiastic.

3.2 Young visitor

Fifty young visitors completed the questionnaire. Their ages ranged from 8 to 13 years, with on average of about 10 years. Males were slightly outnumbered (53%), females (47%). Most of surveyed children attended elementary school, 62%, and 38% middle school.

The results shown that less than half of the children (48%), had already visited similar exhibits or tours in the past; the remaining 52% had never attended similar scientific exhibits or educational tours. 64% of the children enjoyed the visit "very much" and 22% "somewhat".

To the question "What do you think about the meeting with the researchers?" the answer has been generally positive. The results indicated that all aspects of the meeting with the researchers, from the interest in the discussed topics, to organization, usefulness and comprehensibility, have received positive feedback. Only 10% found the meeting of "little" interest and 5% "not at all".

Tour 1 was considered to have been very useful, 79%, very interesting, 74%, and very well organized, 69%. The information was not uniformly understood and, although the 43% considered it to be “enough”, the 33% answered they understood "somewhat" enough, 10% a “little” and 13% “not at all”.

Tour 2 received a very positive feedback too; the "very much" replies have obtained between 70 and 80% for all four aspects: interest in topics discussed, organization, usefulness and comprehensibility, while for organization, usefulness and interest and ease of comprehension, we obtained 10–20% "somewhat", 4–9% “little”, and 4% "not at all", respectively.

Tour 3 received percentages similar to Tour 2: 70% have appreciated "very much" the organizational aspect, usefulness and interest and 60% for the level of comprehension. The remaining answers are 30% for "somewhat" and 10% for "not at all".

As in the case of adults, several questions were not answered by children. Almost 60% give blank answers to questions on Tour 3, a little over 50% on Tour 2 and 40% on Tour 1.

The 66% of the interviewed children were aware of the topics shown. Their knowledge came from: "specific studies (34%), "television" (26%), "Internet" (20%) and "reading educational material" (16%). 41% of children visitors agreed they learned "very much", 27% “somewhat”, 27% “little” and 5% "not at all”. Their expectations were met "very much" for 32%, "somewhat" for 46%, “little" for 17% and "not at all" for 5%.

To the question "Was the amount of information too much?" the answers were evenly divided between affirmative (56% "little" and "not at all") and negative views (44% "very much" and "somewhat").

The “Open Day” was deemed "very much" stimulating, and for 35% of children it generated an incentive to deepen their knowledge, compared with the answers “somewhat” (32%), “little” (20%) and “not at all” (13%).

3.3 Multiple correspondence analysis

Multiple Correspondence Analysis was only applied to the questions included in both adults’ and children’s questionnaires. We have used visitors’ evaluations on specific aspects as active variables; questions on gender and awareness were added as supplementary qualitative variables. The purpose of the analysis was to identify the associations between variables. These associations were represented as maps, where it was possible to observe the distances between categories of variables and visitors to ease the interpretation of structures in the data.

Figure 1 shows a zoom on the distribution of variables and observations along the first two axis. Since the active variables show a great dispersion along the horizontal and vertical axis, Fig. 1 focuses only on the variables most correlated with axis 1. The variables most positively correlated with Dimension 1 are: 1Int-Little (Tour 1 little interesting) variable, 1Use-Little (Tour 1little useful) variable, 1Org-NotAtAll (Tour 1 not at all organized) variable, 2Com-NotAtAll (Tour 2 not at all comprehensible) variable, 2Org-NotAtAll (Tour 2 not at all organized) variable and 2Use-Little (Tour 2 little useful) variable. The remaining active variables, illustrative variables and observations gravitate around the origin so we can’t identify a group of visitors with similar profile in their answers.

Fig. 1
figure 1

MCA analysis. Biplot for variables (active in red and supplementary in grey) and observations (in blue) on axis F1 and F2: a zoom on the most correlated with F1

Full size image

Figure 2 shows a zoom on the distribution of variables and observations along the third and the fourth axis. There are many negative judgements, on the fourth axis (labelled with Not At All).

Fig. 2
figure 2

MCA analysis. Biplot for variables (active in red and supplementary in grey) and observations (in blue) on axis F3 and F4: a zoom on the most correlated with F4

Full size image

3.4 Stakeholder visitors

The 30% of the 100 stakeholders who attended the Open Day, essentially geologists and teachers, completed the questionnaire. The responses were scaled on three separate scales, two of these were scored on a 4-point Likert scale and one on a 10-point Likert scale. The majority were geologists, and so understanding their perceptions of the event, could be by multiple linear regression analysis. The purpose of the analysis is to predict the linear relationship between the dependent variable and independent variables. The description of the relation between the variables can be done by using a multifactor model to explain the variation of the overall evaluation (y) from stakeholders based on the simultaneous influence of 9 indicators of their perceptions of the event (x1, x2, …, x9):

$$y \, = \, f \, \left( {x_{1} , \, x_{2} , \, \ldots , \, x_{9} } \right) \, + \varepsilon$$

The model can be transcribed as follows:

$$y \, = \, b_{0} + \, b_{1} .x_{1} + \, b_{2} .x_{2} + \, \cdots + \, b_{9} .x_{9} + \varepsilon$$

The variables included in the model, their function and their summary statistics were reported in Table 1.

Table 1 Variables of the multiple linear regression analysis. Function and summary statistics
Full size table

The relationship between variables may be given in the form of equation, including all variables, as follows:

$$Overall\,Valuation = 3.933 + 0.172*Ease\,of\,Comprehension - 0.182*Topics\,cohesion - 0.003*Clarity\,of\,Contents{-}0.067*Completeness\,of\,Contents \, - \, 0.160*Effectiveness\,of\,Support - 0.134*Satisfied\,Expectations - 0.265*Appropriate\,Explanations - 0.050*More\,Information + 0.040*Desire\,to\,learn\,more$$
(1)

The regression model including all 9 variables has shown that Overall Valuation wasn’t significantly influenced by any independent variables. Working at 10% level of relevance, as the probability attached to the t-Statistical test is higher for all independent variables, the coefficients are not considered statistically significant (Table 2).

Table 2 Model parameters: standard error, t test, p-value
Full size table

After the introducing a selection criterion of variables (Best Model: Adjusted R2), the new regression equation was the following:

$$Overall\;Valuation \, = \, 4.039 - 0.300*Topics\;cohesion - 0.184*Satisfied\;Expectations -\, 0.292*Appropriate\;Explanations$$
(2)

The coefficient parameters were reported in Table 3. As can be seen, Topics cohesion and Appropriate Explanations were significant influence factors in the Overall Valuation of the stakeholders. The Satisfied Expectations variable is not significant because his p-value was higher than the threshold of 10%.

Table 3 Model parameters: standard error, t test, p-value
Full size table

The correlation matrix (Table 4) shows correlations between the independent variables. The most relevant are in bold type, such as the correlation between Clarity of Contents and Completeness of Contents (r = 0.627, p-value = 0.000) and between More information and Desire to learn more (r = 0.658, p-value < 0.0001).

Table 4 Correlation matrix: Pearson correlation coefficient (panel a) and p-value (panel b)
Full size table

The goodness of fit is 0.394 (R2) and 0.327 (R2 adjusted). This implies that 33% of the variability in the Overall Valuation variable is predictable from the model with three independent variables, the Topics cohesion variable, the Satisfied Expectations variable, the Appropriate Explanations variable.

In accordance with the other results of the multiple regression (ANOVA and F-test, graphic analysis of residuals, multicollinearity statistics), put in the Appendix 1, we concluded that this hypothesized model with three independent variables is correct and the regression analysis is successful in explaining the part of the variation of the Overall Valuation variable. The F-test (Appendix 1) returns a low p-value because some of the regression parameters are nonzero and the regression equation has validity in fitting the data. Random behavior of residuals and the devoid of multicollinearity (Appendix 1) confirm that this hypothesized model is correct.

Despite a significant amount of information is not explained by the model we have used due to some effects that have not been included in this analysis, this study can be deepened by means of the research of other explanatory variables able to improve the quality of the fit. The results and the level of precision can improve through growing the sample size.

3.5 Qualitative results

Adult and young visitors who compiled the questionnaire have provided answers to the open-ended question, to recommend changes and improvements for the INGV event. The answers can be grouped into the following four categories for the adults (A) and three categories for the young (Y) visitors:

  • (A1) Improve event promotion and publicize the event in advance;

  • (A2) Improvement of the general organization. In particular, to reduce waiting times and to better organize visitor groups by using suitable online reservations;

  • (A3) Extend the duration of the event to more than one day;

  • (A4) Enhancement of different tools used to present topics. Suggestions range from (a) new single technical exhibits reproducing natural phenomena, to (b) distribution of printed material, (c) improved tours to the monitoring room, (d) more technical explanations of used software.

  • (Y1) More time to visit the exhibition;

  • (Y2) More information and details;

  • (Y3) More games and hands-on activities.

The most recurrent suggestions (S) from all groups are:

  • (S1) promote the event in advance;

  • (S2) shorter questionnaires (requested by children);

  • (S3) distribution of visitors in several days dedicated to the event (for example hold the event over more than one day).

  • (S4) more time to fill out the questionnaire.

4 Discussion and conclusion

The INGV Open Day held on January 20th represented an opportunity for us to expand visitors’ knowledge in the context of informal learning scientific activities. We proposed questionnaires for adult, young and stakeholder visitors. The aim is to develop suitable tools to investigate on the experience provided to the participants during the activities. Questionnaires are a widely used tool for conducting visitor surveys, allowing the collection of information, anonymously for the respondents, immediate and at low cost (Dickenson 1992). The disadvantages and limitations pertinent to the tool used, are related to the rigid structure of most of the questions, and to the possible carelessness or inaccuracy in the compilation (Dickenson 1992).

Even if the percentage of collected questionnaires represent only the 15% of the total participants, we found precious hints from the surveys.

From our results, although a small percentage of respondents critiqued the organization, the replies have underlined a high rate of approval to the event, around the 90% of respondents.

The main criticisms were related to long waiting times and exhibition being too short (C1; §3.3). Visitors suggested to manage reservations via dedicated website and to better advertise the event (A2; §3.3).

To enhance the educational content of the presented topics, some visitors proposed the idea of improving the printed material (A4 e C2; §3.3) and to develop additional equipment for experiments and hands-on activities (C3).

In combination, the results have highlighted opportunities for improving the effectiveness of science communication during the INGV outreach activity. The key changes identified are:

  • promote the event in advance exploiting different means of communication: television, radio and the press are certainly the classic advertising channels for local and national publicity. For a greater impact it would be necessary to study all possibilities that Internet offers;

  • Introduce a more efficient online reservation system: an online reservation system was even provided for the event but scarcely used by visitors;

  • Enhance the tours with appropriate technologies, such as dedicated apps: dedicated Apps would allow visitors to carry on a smartphone the most important information, interactive maps and, even, the opportunity to provide comments;

  • use a group of people to hand out questionnaires and hold direct interviews;

  • a shorter questionnaire for children with a simple and direct language following a criterion of briefness and completeness;

  • standardize as much as possible the questionnaires of all INGV events, to create a common data base for comparative analyses. These could be replicated, optimized and adjusted.

The results from these studies, indeed, produce worthwhile suggestions to adopt future actions aimed at enhancing the overall quality of offered events. For instance, in occasion of the 2020 open day, actually subject of a publication in progress.

In conclusion, our study suggests how a combined methodology, integrated with information from dedicated apps created ad hoc, could be an effective instrument to assess appreciation and effectiveness of INGV exhibitive/educational paths, while supporting the development of innovative approaches for improving scientific communication campaigns.