1 Introduction

Science education research in Brazil dates back to the 1940s, when some isolated actions promoted the creation of the first research groups. After, in the decade of 1970, the first journals were created, and the first meetings on physics education were held in Brazil (Nardi, 2005). The increase in the quality and quantity of graduate programs in Brazil in the last decades has led to a significant increase in research in science education, and in particular in physics education. However, some authors claim that Brazil still needs a better dissemination of its scientific production in this area (Hadimani et al., 2015; Abramo & D’Angelo, 2015; Kellner, 2017; Schulz, 2019; Santos et al., 2018). In this context, current Brazilian journals on science and physics education look for high quality papers as well as readers, downloads and citations.

Impact-based metrics can be used to screen journals practices and drive management decisions. Some indices are well known and used by the researchers on bibliometrics and scientometrics, like the impact factor (citations per paper, traditionally in a 2-year period), Hirsch’s h-index (Hirsch, 2005) and its alternatives, such as the g-index (Egghe, 2006), h-index normalized by individual (hinorm, Harzing et al., 2014) and the h-index normalized by year (hiannual, Batista et al., 2006). Several other metrics can be used for the goal of screening journals, establishing profiles and estimating journal impact such as the number of papers published, views, downloads, citations and so on (Carpenter et al., 2014).

CAPES — acronym that stands in Portuguese for Coordination for the Advancement of Graduate Education — is a Brazilian government agency linked to the Ministry of Education. Since 1976, CAPES is responsible evaluating graduate programs in Brazil. CAPES’ assessment system consists of a number of tools (Marenco, 2015) where Qualis Journals is one of them. Its function is to assist the evaluation metrics in the qualification process of the bibliographic production of professors and students of graduate programs accredited by Capes (Barata, 2016).

Briefly, Qualis Journals (or just Qualis) is a categorical variable created to label journals, which would reflect the quality of the graduate programs whose professors and students publish in. During the quadrennium 2013–2016, eight strata (Table 1) formed Qualis, where A1 stands for the highest level (1st stratum) and C the lowest (8th stratum). As it was conceived, there is a Qualis value for each area of knowledge (area-dependent Qualis), what causes misunderstandings and can be deleterious for periodicals with broad scopes, since one single journal can be labelled with different Qualis categories (Kellner, 2017). For instance, the Latin-American Journal of Physics Education (http://www.lajpe.org/) is labelled A2 for Teaching, B1 for Interdisciplinary, B5 for Material and C for Astronomy/Physics.Footnote 1

Table 1 Strata for area-dependent and -independent Qualis
Full size table

Even though Barata (2016, p.5) says clearly that “Qualis Journals should not be considered as an adequate source for the quality rating of scientific journals for purposes other than the evaluation of graduate programs”, Marenco (2015) notes what is very natural and intuitive: Qualis drives researcher submission. Therefore, indirectly, Qualis says something about the quality of the journal. If so, how can a journal have several quality levels at the same time? The discussion must consider several arguments. First, Qualis is not concerned with the absolute journal quality itself. Qualis embeds the policy of the area, considering researchers that publish in journals specialized in other areas. So, the area may (dis)encourage this practice adopting a higher or lower Qualis. Furthermore, as the criteria to establish Qualis vary from area to area, it is not surprising that the same journal show different strata, because the quality requirements of one area may be harder to achieve than those adopted by another. Finally, the relativization of Qualis prevents “opportunistic papers” from being published in journals of secondary areas and counting as if they had been published in journals of the main area.

We cannot fail to mention that in 2019, CAPES has informed the academic community that Qualis Journals may soon be replaced by a set of bibliometric indexes, which will qualify the bibliographic production of graduate studies with internationally recognized metrics and more credible than just a single indicator.

El-Hani et al. (2008) led us to perceive certain weaknesses in the bibliometric indexes, including Qualis, which acts in the internal process of evaluating the bibliographic production of Brazilian graduate studies (including research in Physics education). Some arguments that influenced us:

[...] several of the best journals focused on research in science education, [which have already been classified] as International “A” in Qualis, are not indexed in bases such as ISI's Web of Science, such as Science & Education, Journal of Science Education and Technology, Physics Education, International Journal of Science and Mathematical Education or Enseñanza de las Ciências. In addition to several well-qualified national journals, in which the community of scientific education researchers must publish, due to the very nature of their work and commitment to improving science education in our country (Brazil). Despite not being indexed on such a basis, they are widely consulted by researchers in the field, with a consensus within this academic community that they are journals of the highest quality. […] This further exposes the fragility of supporting our judgments about the quality of research only in procedures dependent on indexing. It is not that they cannot be used, but that they must be used in conjunction with a greater number of elements of assessment and always bearing in mind the differences between the areas of knowledge. (El-Hani et al., 2008, p.4-5)

In view of this, what ISI director James Testa says is limited: “[…] ISI’s basic mission is to provide access to the most important and influential journals in the world” (Thomson, 2003, p.49). However, “Bradford’s Law is a bibliometric principle that states that a relatively small number of journals publish a significant volume of scientific results” (Thomson, 2003, p.5). We corroborate this principle. Not in the sense of admitting that only the journals indexed by editorial conglomerates are the journals that publish “the significant volume of scientific results”, but in the sense that the more likely it is to reach a saturation of scientific results the more specialized articles are considered. That is a condition where it is possible to detect a certain reproduction of ideas or to locate nearby ideas in works of different authors.

In addition, we need to highlight what Leydesdorff et al. (2016) claim, who carry out a “state of the art” study, seeking to expose elements that need to be observed when seeking the development of a professional and citizen bibliometry, capable of generate useful indicators for society and scientific work. These authors point out that the indicators, in general, are algorithmic artefacts without their own meaning, but they receive meaning in institutional contexts and practices. Such constructed meaning, according to the authors, mainly involve four stakeholders: (i) the producers of bibliometric data and indicators (mainly Thomson Reuters (WoS) or Elsevier (Scopus)); (ii) bibliometric specialists who develop and test indicators for the purpose of generating researcher evaluations and scientific results; (iii) decision-making bodies in research that use the indicators to decide, for example, which researches may have greater consequences and results for certain purposes; (iv) and scientists who are evaluated in their careers by means of such indexes that take such indexes as a reference to decide in which journals they will submit their research papers.

The authors continue to analyse that such stakeholders often have different positions and perspectives, sometimes conflicting about the meaning of the indicators. From then on, the authors reframe the idea of indicators as something that needs to be socially constructed, seeking to combine these different perspectives, always searching for citizen bibliometrics, in order to gain greater transparency and interpretation that is more accessible, in addition to being more professional in order to mitigate which causes divergence of positions generating serious defects such as ambivalences, difficulties to be socially accepted, distorted university rankings.

Thus, bibliometric analysis must be carefully carried out due to the diversity and multilateral nature of interests surrounding such metrics and evaluations of these metrics. In this work, we show that multivariate analysis has the necessary potential to contribute to such an analysis.

In July 18th, 2019, CAPES has announced an enhancement to its rating system, in particular Qualis Journals. The improvement need of Qualis had been previously speculated in the literature (Trzesniak, 2016). The idea is to use an area-independent criteria, based on bibliometric indicators — Scopus (CiteScore), Web of Science (JCR) and Google Scholar (h5 index) — yielding a single classification for each journal (Capes, 2019a). The day after, the same agency published that a list of journals and area-independent Qualis had been sent to programs coordinators, highlighting that such list is temporary (Capes, 2019b). Nine strata rather than eight now form the area-independent Qualis (Table 1). However, since they are different metrics, their categories are not promptly equivalent, i.e., the highest stratum of the old metric must be carefully compared with the highest stratum of the new one. It is noteworthy that the JCR was not used because most of the journals analysed did not have a record at the JCR or had only a short period.

Therefore, we emphasize that the multivariate analysis of different metrics aims to access the impact through multiple ways, since we agree that each metric has its own weaknesses when analysed separately. It is precisely in this sense that the importance of the multivariate approach resides, considering the correlations and covariance between metrics, in addition to their main values. Specifically, the multivariate approach can identify groups by similarity (cluster analysis) and then project in lower dimensions, what enables interpretation of the journals location in terms of the metrics (principal component analysis).

Several respected and valuable databases such as Scielo, Scopus and Web of Science house scientific articles and can be consulted to get the number of citations for papers of a specific journal during a period. However, as highlights Schulz (2019), a paper can affect researches, professors, lectures and students in several ways, such as influencing their monographies, dissertations, thesis, abstract for meetings and conferences and so on. That is where the Google Scholar plays an important role, accounting for citations on all kinds of scientific documents.

In this context, our main goal is to monitor Brazilian local scientific production on Science and Physics education from 2013 to 2019 using scientometric tools and give that scenario international visibility.

2 Material and Methods

It is a retrospective longitudinal quantitative and observational study on Brazilian journals on Science education, with particular focus on Physics education. We aim to access the profile by their main features and publications impact through several metrics.

The object of the study are 3557 papers from thirteen Brazilian electronic journals focused on Science education and/or Physics education, with editorial board, International Standard Serial Number (ISSN) and own website. Full names in Portuguese, acronyms and websites are given in Table 2.

Table 2 Journals, ISSN, acronyms and respective websites
Full size table

Data were collected from December 1st to 25th of 2019. We visit the websites for acquiring some information but obtained most data through the software Publish or Perish (PoP) version 7.15.2643.7260 (Harzing, 2007). PoP searched in Google Scholar database for finding the articles, citations and computing impact metrics. We used the ISSN and the full name of the journal for searching for its articles year by year from 2013 to 2019.

The publishing period of the articles of this study is from 2013 to 2019. Such period binds the quadrennium (2013–2016) for evaluating the publications of Brazilian graduate programs with the period up to the present day (2017–2019), when circulates the preliminary area-independent Qualis.

We have observed fourteen variables. Four variables referring to journal features such as number of papers publish per year, journal’s lifetime, number of authors per paper (app) and the percentage of papers that mention the word “Physics” and/or “Física” (%phy), in English and Portuguese, respectively. Six variables are rather impact metrics such as h-index, g-index, normalized and annual h-indices, cites per paper (cpp), cites per paper per year (cpppy); and four variations of Qualis index, namely Teaching, Education, Astronomy/Physics Qualis (teaQ, eduQ, aphQ) and area-independent Qualis (uniQ).

We collected Qualis data directly from the CAPES website,Footnote 2 on the websites of scientific journals (number of papers publish per year, journal’s lifetime, ISSN) and using the software Publish or Perish (for app, %phy, cpp, cpppy, h-index, g-index, normalized and annual h-indices). When accounting for citations, PoP considers the Google Scholar database, so it accounts for citations made in monographies, dissertations, thesis, annals, proceedings and, obviously, papers and articles.

According to Harzing (2007), the Publish or Perish software computes the cpp index as the sum of the citation counts across all papers, divided by the total number of papers. Since cpp is time-dependent, the cpppy is more accurate as long it is weighted by how many years they have been cited.

Statistical analysis was performed in the statistical software R version 3.6.3 (R Core Team, 2019) embedded in R Studio version 1.2.5033 (RStudio Team, 2019). Along all variables, descriptive statistics, such as means, proportions and correlations were computed (per journal, per year, along the whole period, etc.) in order to summarize the data.

Since there are some criticisms about the impact metrics individually used (Leydesdorff et al., 2016), we perform a multivariate approach, as it takes into account the correlation between metrics and is not so vulnerable to individual drawbacks. Principal component analysis (PCA) reduces the data dimensionality and represents two spaces: the variables spaceFootnote 3 and the sample spaceFootnote 4 (Husson et al., 2010). Furthermore, before the sample space is projected in a plane, the journals were grouped in cluster according to the multivariate similarity using hierarchical cluster analysis (HCA). Multivariate analyses were performed using the package FactoMineR (Le et al., 2008).

3 Results and Discussion

At first, we selected 13 Brazilian electronic journals based on their focus and scope. As can be seen in Fig. 1, some of them are traditional journals, with decades of expertise and others are quite recent ones. On the top of Fig. 1, we highlight two journals for missing issues. Coincidentally, both journals were removed from this study because their metadata are not available the way Google Scholar — and consequently PoP — needs. It causes empty returns in PoP for searches about these journals, making them impossible to be considered in this research.

Fig. 1
figure 1

Lifetime of Brazilian electronic journals of Science education. Source: Author

Full size image

Science education journals in Brazil are 9 to 41 years old what make them recent journals among other of the same area worldwide. Although the International Journal of Science Education (Taylor & Francis) is exactly 41, and the Journal of Science Education and Technology (Springer Verlag) is 28 years old, the Physics Teacher publisher papers about education since 1978 and the journal Physics Education since 1966.

Another feature related to the history of a journal as well as its flow is the number of papers published per year (ppy). Publishing a large number of articles a year is not a sign of quality per se. However, this may indicate a more popular and more proclaiming journal by researchers. Figure 2 brings an overview of how the number of papers published per year change from 2013 to 2019 for the Brazilian journals. Most journals use to publish up to 40 papers a year in 2 to 4 issues. They also tend to maintain a similar number of papers along the years but EEC that shows a clear increase from 2017 on.

Fig. 2
figure 2

Number of papers published per year between 2013 and 2019 in 11 Brazilian journals. Source: Author

Full size image

This significant increase in EEC publications is due to two factors that influenced such growth: (i) the association of EEC with professional master’s Programs that emphasize an applied and evaluated product in the classroom. In particular, Programs on national networks such as the Professional Master in Physics Teaching (MNPEF), a consortium between CAPES, the Brazilian Society of Physics and Polo Institutions (institutions that carry out the program and graduate graduates); (ii) a second factor that justifies the increase in publications is that from this year (2017), the first graduates of MNPEF start publishing their products and in fact perceive EEC as having a favourable editorial line for the publication of their applied products aimed at the science classroom, especially Physics.

It is important to mention that the Brazilian experience with professional master’s degrees in networks like the MBPEF model is still under evaluation, but it has been showing significant growth in the availability (in the format of a Creative Commons license) of materials guided by different pedagogical perspectives for Physics Teaching and to overcome problems such as the absence of playfulness in activities involving teaching Physics, difficulties in accessibility, foundations of potentially significant didactic sequences for school learning in Physics and greater exchange between classroom experiences between teachers.

Catches the eye the journal RBEF that publish around 120 papers a year in the last 7 years and is the more traditional journal on Physics education in Brazil. In this case, many papers and a long life reflect a respected journal. Schulz (2019) states that the documents published by RBEF impact beyond citations. The author had analysed data from three databases (Scopus, Web of Science and Google citations), and, regardless of the database, the impact of RBEF is increasing, in both citations and access, nationally and internationally.

The RBEF case, in particular, also includes two points of analysis: RBEF traditionally captures general physics articles, which its editorial line considers to be focused on the teaching of physics. However, it is not necessarily research in teaching physics. Thus, RBEF has a wide editorial window because it captures articles that are mostly applications and particular cases of studies of phenomena of interest in physics. Due to an approach that the editorial line considers accessible or conveniently suited to higher education in the area of physics and related areas, there are a high number of publications and citations in this journal. This editorial line gained strength with the implementation of the MNPEF. The second reason is that RBEF has maintained its editorial line practically unchanged since its creation (1979), being one of the first journals of its kind to be launched in Brazil, thus ensuring a stable audience that is interested in and nurtured such an editorial line.

As the focus of this work is on physics education and some of the investigated journals are interested specifically in physics, the percentage of papers mentioning the word Physics was computed (Fig. 3). We used the descriptors “Physics” and “Física” (Portuguese) was computed (Fig. 3). All articles examined have an abstract (in Portuguese in addition to the keywords) and/or an abstract and keywords (in English), so the search was bilingual. The aim was to check the coherence between the scope of the journal and this metric. Such word was searched in title, abstract, keywords and in the text body.

Fig. 3
figure 3

Percentage of papers that mention the word “Physics” or “Física” (Portuguese). Source: Author

Full size image

Indeed, the only two journals that are specifically interested in physics education (CBEF and RBEF) presented the word ‘physics’ in all papers published from 2013 and 2019, and there was no journal with less than 50% of the papers containing that word. Even the journal devoted to experiences in science education — that has no word physics in its objectives — presented more than 53%. Such behaviour is in line with expectations since science is hardly detached from Physics. These numbers indicate, as we will discuss, a high interest in research in physics education (perhaps disproportionate in relation to other areas of science education).

In order to evaluate the impact of publications on science and physics education — and consequently journal impact — several metrics were considered. The first one computed was the overall number of citations over the overall number of published papers per journal, over whole period (2013–2019). That metric can be understood as a kind of impact factor. The usual impact factor over a database — just like the proprietary JCR® — is computed over a 2-year period (Garfield, 1999). In this study, we computed a similar metric, in a 7-year period. However, we consider the total number of citations that papers had since publishing.

Figure 4 shows the ordered citations per paper (cpp) and reveals that journal CE holds the highest score. CE journal is indexed in Scielo and Google Scholar, presenting h5 index equal to 17 and h5 median to 23. Also catches the eye the journal ECTR with less than one cite per paper far from the second smaller, ARECM. The ECTR journal is the youngest Brazilian journal on science education (9 years old) and presents the smallest number of papers per year (Fig. 2) although a light increase in the last couple of years.

Fig. 4
figure 4

Citations per paper (cpp) published during the period 2013–2014 according to Google Scholar, for eleven Brazilian journals. Source: Author

Full size image

It is interesting to note that the two journals (CBEF and RBEF) reach 100% mentions in response to the search for “Physics” or “Física”. This answer was already expected, but the reason for presenting this fact is the interest in highlighting the contrast with journals that are dedicated to teaching science, but have high rates of responses to the search for “Physics”. For example, IEC, with 86.9%, or even EEC, with 53.1% — minimum observed value. In the case of EEC, which has a strong emphasis on Physics due to the publications coming from the MNPEF, it is interesting to note that only 53.1% of the articles respond to this descriptor. In other words, except for RBEF and CBEF, the other journals open to science education (Science in Elementary Education, Chemical Biology and Physics and so on). They mainly publish articles that respond to the search for the descriptor “Physics” or “Física”, as a fact that reflects the higher qualification of research production in Physics Education, or the Brazilian production focused on Physics Education is simply greater than in other areas of science education. On these hypotheses, the community of researchers in science education and research can be considered as developments, after all, why would we have a quantitative “de-calibration” of publications of articles originating from works in physics education to the detriment of the rest of the field of science education?

It is important to discuss that the idea of research in physics education is not consensual in the in Brazilian research community. Villani (1981) already indicated that the euphoria and growth of research in physics teaching in the community was cooled by criticism from their counterparts in the areas of physics (in general). Then, a decisive discussion for scientific education emerges, in particular initiated in the context of physics teaching research, which, according to Villani, did not go unnoticed and, by cooling the euphoria of the young research area in physics teaching, forced it to adopt the most appropriate scientific rigor criteria. Let us see what Villani (1981) argues:

[…] if a physicist was asked to list the activities that define research in physics, the answer would not be simple, as many different activities, from solving equations to building a device, can be encompassed […]. (Villani, 1981, p. 72)

However, all these activities would be considered legitimately linked to research in physics, according to Villani because of its tradition. On the other hand, research in physics education would suffer from greater resistance.

On the other hand, defining what is research in physics education is much more controversial, as the genesis of this activity has not yet been sufficiently analysed and discussed, nor its meaning fully legitimized, so that its development proceeds according to its roots. The community of researchers in the area has not yet appropriated the meaning of its nature uniquely, which is why the initiatives can vary and the community does not have the authority to recognize or ignore it […]. (Villani, 1981, p. 72)

Based on the conception of research at the “Lato Sensu” graduate level, Villani (1981) concludes that:

Any type of activity done for the sole purpose of improving classroom practice or increasing students’ motivation, without any type of systematic recording or at least reflection that clarifies, in some way, for the scientific research community, the contribution offered, is not fundamental research in Physics Teaching (but it may well be great Physics Teaching). Any type of written production (textbook, handout, problems, and exercises) without any analysis of the theoretical bases or of the purposes or conditions of applicability is not fundamental research in Physics Teaching (but it can be an excellent subsidy for the teaching activity). (Villani, 1981, p. 73) (Author’s emphasis)

Delizoicov (2004) challenges us to outline, in all the articles that we produce in the area, a section on what are the implications of such research for the Science classroom, since the fundamental research in Science Teaching must affect in the context of the science classroom for ethical imposition.

Considering what Delizoicov (2004) and Villani (1981) present us in these excerpts, and throughout the original works that we analysed in this context, we argue that such a discussion is current and needs to be permanently brought to the research community, so that the editorial lines are always be guided by the scientific rigor of their publications, without losing the ideal of having the science classroom as a primary target.

These considerations lead us to considered that, in the RBEF, a certain amount of these works may have missed the centrality and the induction of research in Physics Education (“Physics Education Research”) in favour of other topics of the its editorial line, such as “Articles” (where most of its published texts are concentrated); “Physics Education Research”; “Didactic Resources” (editorial area on the rise in the journal, since MNPEF started); “History of Physics and Related Sciences” and “Book Reviews”.

Such editorial lines for a relatively new area may represent an element of disturbance in the research community in Physics Education and thus absorb many types of research under the roof of a Physics Education journal (RBEF). This would justify its great repercussion and diffusion, therefore, its impact in the context of physics education.

In order to evaluate the longitudinal evolution of the impact factor we computed the citations per paper in each year from 2013 to 2019, weighted by how many years they have been cited. Figure 5 brings that information highlighting the cote of one citation per paper per years (cpppy) as a reference.

Fig. 5
figure 5

Citations per paper weighted by how many years they have been cited (cpppy). Source: Author

Full size image

It seems that papers published in 2019 did not have had time to be cited yet, since for all journals, the citation accounts are very low. Apart from that, 2016 seems to be a year of well-cited publications for all journals but ECTR. Specifically, catches the eye the evolution of ICE. It was in the last positions in 2013–2015 but increased to the first positions in 2016–2018. ECTR and ARECM keep low impact factors along the entire period as well as CE, CBEF, RBPEC, EPEC, RBEF and A shuffle in the higher positions.

As discussed in the first section, the Qualis is an internal index, important to qualify the bibliographic production of the Brazilian graduate courses. We cannot fail to repeat that recently, between 2019 and 2021, CAPES has informed the academic community that Qualis Journals may soon be replaced by a set of bibliometric indexes that will qualify the bibliographic production of graduate Programs with internationally recognized parameters, more credible than just a single metric.

Originally, the Qualis was not conceived to be a metric. However, it drives author’s choice when associates indirectly a label of quality to each journal. Figure 6 brings the Qualis Journals for each journal evaluated in this work, where colour intensities refer to different strata. First three columns refer to the area-dependent Qualis for Teaching, Education and Astronomy/Physics areas. Fourth column stands for the recent preliminary area-independent Qualis.

Fig. 6
figure 6

Area-dependent Qualis in Teaching (teaQ), Education (eduQ) and Astronomy/Physics (aphQ), and area-independent Qualis (uniQ) for the analysed journals. Source: Author

Full size image

In spite of been correlated areas, the eleven journals on science and physics education are poorly classified in Astronomy/Physics, since there are specific areas and committees to manage them. We can also note that most of the area-independent Qualis match the highest level among the three areas, for a journal. It has to be clear that the preliminary area-independent Qualis follows a specific classification algorithm that inputs impact metrics like CiteScore (Scopus), JCR (Web of Science) and h5 index (Google Scholar) and what we noted is nothing but coincidence. By the way, a couple of journals present a lower area-independent Qualis than area-dependent Qualis, namely RBECT and EEC.

Finally, we can note that journals RBEF, CBEF, CE, EPEC and RBPEC — that present the highest area-independent Qualis — in general, also present the longest lifetimes, mentions of the word ‘physics’, cpp and cpppy (Figs. 1, 2, 3, 4, 5). Since Qualis Journals uses impact metrics in some extent to classify journals (Barata, 2016; Capes, 2019a), it is not a surprise suggesting a positive association between them (Marenco, 2015; Santos et al., 2018).

Checking for correlation, Spearman correlation coefficient was computed and tested for all pair of variables. In Fig. 7, the colour scale indicates the correlation strength — as shows the lateral scale. The ellipses is the overview of a bivariate normal distribution section. The tighter the ellipses, the greater the correlation. Crossed ellipses represent non-significant correlations, at 5% of significance.

Fig. 7
figure 7

Spearman correlation coefficient between impact metrics, authors per paper and lifetime for eleven Brazilian journals. Source: Author

Full size image

Figure 7 shows that area-dependent Qualis (aphQ, teaQ and eduQ) are positively correlated within — as suggested in Fig. 6 — but not with impact metrics. Surprisingly, the main areas of the journals analysed here, teaching and education (represented by teaQ and eduQ) do not show association with impact metrics (p value > 0.05). On the other hand, the secondary area, Astronomy and Physics, represented by aphQ, do correlates significantly (p value < 0.05). Furthermore, the correlation between Qualis in Astronomy/Physics and the impact metrics is negative. The more impact, the lower the aphQ? To construct an explanation for it, we must remember that the journals analysed here are not devoted to astronomy or physics but teaching and education research. That is probably the reason why Qualis present such anomalous behaviour here. Even though, null and negative correlations represent an evidence that area-dependent Qualis is probably not scientifically built, in the sense discussed by Leydesdorff et al. (2016).

As expected, we found that the impact factor (cpp) is strongly correlated with h, g and hinorm indices besides the number of citations and citations per year in the journal, but surprisingly it is not correlated with hiannual. Harzing et al. (2014) proposed the individual, average annual increase of the h-index (hiannual). The hiannual is useful for the following reasons: (i) it removes to a considerable extent any discipline-specific publication and citation patterns that otherwise distort the h-index; (ii) it reduces the effect of career length; and (iii) it is meant to be an indicator of an individual’s average annual research impact. Rather than being correlated with citations per paper, in this work hiannual was found to be correlated with citations per paper per year.

Finally, the variable authors per paper (app) showed no correlation with the other metrics, what contradicts Fox et al. (2016) that found that the number of citations increase with the number of authors, manuscript length and references cited in ecological journals and Abramo and D’Angelo (2015) that found positive correlation in papers of some knowledge areas in Italy.

In a multivariate approach, we performed the principal component analysis (PCA) in average and longitudinal data, as well as hierarchical clustering analysis (HCA) for finding groups. Figure 8 brings the variable space, i.e., the first two components where the variables are represented. That plan explains 75.83% of the total variance. We can see that impact increases to the positive direction of the first component, where the impact metrics stand together. Qualis indices are also together — indicating correlation between them — while the number of authors per paper, pointing to the positive side of the second component, stand orthogonally to the impact metrics. That confirms the independence found in the correlation analysis.

Fig. 8
figure 8

Variable space representation. Source: Author

Full size image

In association with the variables space, the sample space (Fig. 9) present where journals group together. From Fig. 9, we understand which journals are similar along impact metrics and what characterize such similarity. Figure 9a shows four clusters of journals, while Fig. 9b shows the clustering tree, where we see the level of multivariate similarity. For instance, we note that clusters 1 and 2 are more similar to each other than to anyone else.

Fig. 9
figure 9

First two dimensions (A) and 3-D dendrogram (B) for clustering the eleven Brazilian journals

Full size image

The joint interpretation of Figs. 8 and 9 enables to see that low impact metrics characterizes clusters 1 and 2 (EEC, RBECT, ECTR and ARECM). Cluster 1 presents a little higher Qualis and cluster 2 more authors per paper. On the other hand, cluster 4 (RBEF and CE) is characterized by high impact metrics. Cluster 3 (IEC, EPEC, CBEF, A and RBPEC) presents average metric levels. Hadimani et al. (2015) also used impact metrics for evaluating 76 journals and profiling the publications of Indian Institute of Science Education and Research from 2008 to 2013.

Schulz (2019) analysed the impact and influence of the RBEF through citations to documents published in the journal retrieved from the Web of Science and Scopus databases and Google Citations. The author states that RBEF publish high impact papers not only due to citations but also due to the increasing number of accesses in Brazil and abroad.

In a longitudinal multivariate approach, Fig. 10 brings how each journal evolved from 2013 to 2019, along the Qualis and impact metrics. For each graph, the first two principal account for 64.33% of total variation. In this approach, in addition to the variables previously used, we consider the number of articles per article per year (cpppy).

Fig. 10
figure 10figure 10

Longitudinal Principal Component Analysis. Variables space (A), and temporal path (scores) from 2013 to 2019 for journals: RBEF (B), CE (C), CEBF (D), RBPEC (E), EPEC (F), A (G), IEC (H), EEC (I), ARECM (J), RBECT (K), ECTR (L)

Full size image

We can see in Fig. 10a that the number of authors per paper (app) is placed at the origin, what means it does not figure in the first two dimensions. However, we detected before that it seems not to be important for expressing impact in our database. The right hand side of the plots — quadrants 1 and 4 — represent the direction where the impact increases. Specifically, the positive direction of X-axis represents g, h, hinorm and hiannual indices (proper impact metrics), while quadrant 1 represents Qualis and the percentage of mentions of the word Physics in the paper. The quadrant 4 stands for the number of published papers, citations per year and citations per paper per year.

Figures 10b to 10l bring the temporal walk of the journals from 2013 to 2019. In each plot, the journal acronym is followed by 13 to 19 indicating the year. In general, the more to the right the journal/year is, the more impact it has/had. The ideal path would be always to the right; however, recent years do not provide good numbers since the papers had not time to be read and cited yet. That is why 2019 usually goes to left.

For instance, RBEF course seems to be the path more to the right, indicating the highest impact journal. In RBEF course, 2017 is the year which papers had more impact. Schulz (2019), studying impact metrics from 2015 to 2018, found that RBEF presented an increasing impact overall. An overview reveals that most journals published the more important papers in 2017 and 2018 (RBEF, CE, CBEF, RBPEC, A, IEC and EEC, ARECM, ECTR, respectively).

Regardless of the position in respect to the X-axis, it seems to be a temporal movement upwards, generally contraclockwise. It can be interpreted as an increase of Qualis over cpy and cpppy. RBECT is the only journal that moves downwards and to the left. It’s more important publication year was 2016, and its cpy and cpppy are increasing over the Qualis. Note that in Fig. 6, the area-independent Qualis attributed to RBECT is the minimum among the Qualis for Education, Teaching and Astronomy/Physics. That is the only journal whose area-independent Qualis is the minimum amongst the others, i.e., the Qualis index decreased over time.

4 Conclusions

Thirteen Brazilian journals on science and physics education were considered in this paper. Unfortunately, two of them were discarded since the metadata were not in a suitable format for Google Scholar search. From eleven journals considered, in general, journals with more impact — cites per paper, h and g indices and so on — usually present many papers that mention physics, publish many papers per year, have high Qualis Journals index and have long lifetime. The number of authors per paper was uncorrelated with every metric considered. Along time, Brazilian journals had the best (more impact) papers between 2017 and 2018 and are receiving higher Qualis index, overall. Such longitudinal multivariate analysis is able to detect patterns and be used as a management tool for driving journals to the desired position in the scientific scenario.

Our results seem to question whether research in physics education is dominant in editorials, since the descriptor “Physics” obtains a return greater than 50% even in journals with a broad editorial line for science education. We also demonstrate that there is an imbalance to be corrected. Scientific Education or science teaching journals have articles that respond to the descriptor Physics with accuracy in at least 53.1%, reaching 86.9%, which indicates a preponderance of physics audiences, despite distortions caused by the accounting of the complete editorial line of RBEF.

We can see that a radical modification of the use of Qualis is necessary — or the adoption of international metrics — that avoid distractions from losing attention to local educational problems related to physics education that are poorly addressed when new research is missing. CAPES can induce changes in this direction with new policies for evaluating graduate products and multidimensional evaluation (recently announced by the agency) that would take into account the articulation between institutional projects and regional problems. Finally, we need to emphasize that research in physics education in Brazil needs to be shared in an intentional setting in order to reach the international community and to be able to intensify the experiential exchanges of its particularities and of the particularities of other regions and similar problems worldwide. The RBEF concentrates many citations, but at the same time has an editorial policy that is less adherent to the educational issues of physics education. Therefore, it has potentially generated an excessively centralized influence on its articles in the induction of impacts in the physics classroom. It can be balanced with an editorial line more attentive to the problems related to the teaching of physics in the classroom, putting it in perspective with others science — not just physics — and not with a privileged focus on “articles” in a broad sense, that do not translate into fundamental research in teaching physics, as discussed.