Introduction

In computer science, a researcher has several ways of disseminating research results including submitting papers to workshops, conferences and journals. Accepted papers that are submitted to workshops or conferences are required to be presented either as a formal talk or as a poster and are published in the proceedings of that workshop or conference. Publications in journals do not provide the author with the ability to present the information in person. Workshop and conference venues generally have several other differences as compared to publishing in journals such as the longer time from submission to publication in journals than in conferences and workshops. Researchers who wish to disseminate their work quickly often prefer to publish in conferences and workshops proceedings.

Many computer science subareas have their own top conferences and journals. For example, in machine learning, the two conferences Intl. Conference on Machine Learning (ICML) and Neural Information Processing Systems Conference (NIPS) and the two journals Machine Learning (ML) and Neural Computation (NECO) are the leading publication venues. For software engineering, the Intl. Conference on Software Engineering (ICSE) and Computer Aided Verification (CAV) are the top conferences, while IEEE Transactions on Software Engineering and ACM Transactions on Software Engineering Methodology are the top journals in which to publish.Footnote 1 We study the computer vision subfield in this paper. We chose computer vision for several reasons. First, it is interesting to study since it is one subfield that is at the intersection of computer science, engineering and physics. Second, computer vision is a growing field with many researchers. For example, since 2006, 5878 different researchers have authored or co-authored papers published in the top-3 conferences and top-3 journals analyzed in this paper.Footnote 2 Third, we were aware that some vision researchers prefer to only publish in conferences. Finally, it is the subfield in which two of the three authors work and therefore it is the one that is most familiar.

In this paper, we study the three journals and three conference proceedings that have the highest impact factors among conferences and journals that publish computer vision (CV) research. The top-3 CV journals are IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), Springer Intl. Journal of Computer Vision (IJCV) and Elsevier Computer Vision and Image Understanding (CVIU) and the top-3 CV conferences are IEEE Intl. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Intl. Conference on Computer Vision (ICCV), and the European Conference on Computer Vision (ECCV).

The acceptance rates of the top-3 conferences areFootnote 3:

  1. 1.

    IEEE Intl. Conference on Computer Vision and Pattern Recognition, from 2003 until 2009 range from a low of 23.5% to a high of 31.9% with four of the seven years having acceptance rates of about 28%;

  2. 2.

    IEEE Intl. Conference on Computer Vision was 20.5% in 2003, 19.9% in 2005 and 23% in 2007;

  3. 3.

    European Conference on Computer Vision was 34.2% in 2004 and 21.4% in 2006.

The acceptance rates of the top-3 journals areFootnote 4:

  1. 1.

    Elsevier Computer Vision and Image Understanding is running at an overall acceptance rate of 20%;

  2. 2.

    IEEE Trans. on Pattern Analysis and Machine Intelligence receives approximately 900 submissions per year and eventually accepts approximately 25% of them;

  3. 3.

    Springer Intl. Journal of Computer Vision in general, accepts under 25% of papers.

The reason we studied the top-3 (as opposed to any more or less conferences/journals) is because the top-3 journals represent 80% of the citations of the top-10 journals in the area over the last 10 years. Similarly, the top-3 conferences represent 85% of the citations of the top-10 conferences in the area over the last 10 years. The minor increase in percentage of citations would not justify adding in any conference or journal beyond the top-3. Furthermore, the 4th and lower conferences and journals in computer vision have impact factors much lower than the top-3. This data was collected from Google ScholarFootnote 5 and Microsoft Academic Search.Footnote 6

The top-3 conferences have an impact of 14.0 relative to the year 2009, while the top-3 journals group has an impact of 29.5 using Google Scholar data.

Sometimes, workshop and conference papers are extended into longer papers and resubmitted to journals. Montesi and Owen (2008) stated in their paper on how conference papers are extended into journal articles that at the time of their publication (2008), “the topic of conference papers versus journal papers and of the process leading from one to the other still remains largely unexplored.” Kling and McKim (1999) state that “computer scientists often treat conference articles as significant forms of publication, and computer science journals are more likely to republish amplified versions of a conference article.” They also state that this is in contrast to publishing in natural sciences.

Freyne et al. (2010) did work concurrently to ours but with a different emphasis (on the status of journal and conference publications) and on computer science (CS) in general, whereas our paper focuses on the computer vision subfield of CS and has an emphasis on a top-3 CV conference publication’s relationship to a top-3 CV journal paper. Freyne et al. (2010) did highlight the fact that CS researchers place “greater emphasis on conference publications and how, as a result, CS researchers can suffer when it comes to ISI Web of Knowledge-based research assessment.” They calculated impact factor from Google Scholar citation data for evaluating paper quality and found that it does correlate well with the ISI Web of Knowledge index, but that it does not correlate well with the rejection rate of conferences, thereby validating their evaluation method and discounting the rejection rate when evaluating a conference’s quality. Using this impact factor, they found that the CS conferences they studied performed well compared to the CS journals.

Franceschet (2010) studied the different impacts of a sample of papers in computer science. The author selected papers indexed in the Digital Bibliography and Library Project (DBLP), whose titles included popular keywords such as “genetic algorithms” or “Internet.” He discovered that 78% of these papers were published in conferences, but the journal papers have on average 4.51 citations, as opposed to 0.71 citations per publication for conference papers. Thus, in general, if computer scientists seek impact, as measured by citation counts, they should concentrate their efforts on publishing in journals.

Shamir (2010) using data from CiteSeer discovered that 88 of top 100 impact factor venues in computer science were conference proceedings, while the other 12 were journals. This result is strongly biased because CiteSeer only uses freely available documents as sources, often technical reports and so this may not represent the computer science literature as a whole.

Wainer et al. (2011) analyzed all computer science papers published by ACM in 2006. They discovered that 40% of the references are to conference proceedings papers and 30% to journal papers. Also, among those papers cited at least 10 times, 41% were conference papers and 16% journal papers. This shows that conference papers are referenced frequently in computer science literature and are a majority of those that are the most highly cited references. But of course, the large number of conference papers causes the overall impact to be low. This is similar to previous results (Goodrum et al. 2001) in computer science (but using CiteSeer data). There has been other bibliometric research on conference papers in computer science as well (Martins et al. 2010; Franceschet 2011).

The works that show that journal papers have higher impact on average than conference papers inspired our work. Computer vision conferences are very competitive, with not only high rejection rates but also high self-selection (one will not even submit to one of the top-3 conferences if one does not consider the paper of high quality). Also because of time pressure and space limitation, it is likely that the submitted paper could improve, and thus increase its chance of acceptance. If conferences have lower citation rates than journals is also valid for the top-3 conferences in CV, why would researchers devote so much effort to publishing in them?

This research tries to answer whether the low citation counts for conferences is also true for the top-3 CV conferences. If that is true, are there other reasons to consider publishing in the top-3 CV conferences? For example, do they serve as a first measure of quality, that is, if a paper is accepted in a conference it has a better chance of being accepted in a journal? Is a previous conference paper a, maybe informal, requirement for a journal paper?

To try answer these questions, we did the following:

  1. (a)

    We determined how many of the 2009 papers published in the top-3 journals have a prior research paper published in one of the top-3 conferences. This measures how true the claim is that a previous conference paper is a requirement for a journal publication.

  2. (b)

    We surveyed all the authors of papers published in 2009 in the top-3 journals, regarding their opinions on the relation between conference and journal papers. This allows us to measure the different understanding regarding these issues among the community members.

  3. (c)

    We collected the citation counts for the papers published in the 2005 and the 2007 issues of the top-3 journals, the citation counts of the prior conference publications for the 2005 and 2007 papers, and also the citation counts for a sample of non-prior 2005 and 2007 conference publications. The comparison of these counts allows us to gauge if indeed conference papers have lower citation counts than journal papers, and whether the conference papers that eventually were published as journal papers already had more or less citations than the other “similar” conference papers.

  4. (d)

    We computed the time interval between submission and acceptance for all 2009 journal papers. This allows us to measure whether the acceptance process for the papers with priors was “easier” or, at least, shorter than the process for papers without priors.

In this paper, we will call “papers with priors” the 2005, 2007, and 2009 journal papers in the top-3 CV journals that were based on a previous paper published in one of the top-3 CV conferences. These conference papers are called the “priors” of the corresponding journal paper. “Papers without priors” are the other 2005, 2007, and 2009 papers published in the top-3 CV journals. Note that a paper without prior may be based on a paper published in a conference that is not one of the top-3 discussed here.

Methods

We first examined all the regularFootnote 7 papers from 2009 in TPAMI, CVIU, and IJCV to determine which papers referenced a top-3 conference paper by the same authors.Footnote 8 For all of those journal papers that reference a top-3 conference paper by the same authors we checked the content of each of the conference papers to determine whether or not they were indeed true prior versions of the journal paper. Three people (the authors of this paper), independently, compared the journal paper to the referenced conference paper by considering the titles, abstracts and the way the conference paper was referenced in the journal paper. A conference paper was deemed as a prior in any of the following situations: (a) it had the same title of the journal paper, (b) the journal paper contained a statement that it was an extension of the conference paper, or (c) the three authors independently agreed that the content of the conference paper substantially overlapped with the journal paper (e.g., with at least 30–40% overlap).

Survey

We contacted two groups of authors of 2009 journal publications, those that authored papers with priors and those whose paper had no priors. In each case, we contacted the last listed author in the papers. Of the 279 papers, there were 271 unique last authors, and 7 of those had an email address that returned some error. Thus there were 264 requests sent to answer the survey.

The questions we asked the authors of papers with prior are:

  • P1 Do you feel that having a paper accepted in one of the top-3 computer vision conferences (ICCV, ECCV, CVPR) is NECESSARY to have the paper accepted in one of the top-3 journals (TPAMI, IJCV, CVIU)?

  • P2 Do you feel that having a previous paper accepted in one of the top-3 conferences IMPROVES the chances of a paper being accepted in one of the top-3 journals?

  • P3 Do you feel that having the paper accepted and published in one of the top-3 conferences is BETTER than having the paper accepted and published in one of the journals?

  • P4 If you answered yes above, in what sense is a publication in the conferences BETTER than a publication in the journal (multiple possible answers)

    1. (a)

      The paper will receive more citations.

    2. (b)

      The paper will be more widely read.

    3. (c)

      The authors will receive more prestige.

    4. (d)

      The turnaround time is shorter for the conferences.

    5. (e)

      The other researchers in the field that I care to reach will be at the conferences and will see my work.

    6. (f)

      other (please specify).

The questions we asked the authors of papers without priors are:

  • NP1 Did you submit a previous version of your journal paper to one of the top-3 computer vision conferences (ICCV, ECCV, CVPR), that was NOT ACCEPTED?

  • NP2 If you answered yes to the previous question, by the time you submitted to one of the top-3 conferences, did you feel that having a paper accepted in one of them was NECESSARY to have the paper accepted in the top-3 computer vision journals (TPAMI, IJCV, CVIU)?

  • NP3 If you answered yes to the first question, by the time you submitted your paper to one of the conferences, did you feel that having a paper accepted in one of those conferences would IMPROVE the chances of having the paper accepted in one of the top-3 journals?

  • NP4 (same as question P3).

  • NP5 (same as question P4).

Citation counts

We performed the same “with prior” and “without prior” analysis for the papers published in the top-3 journals in 2005 and 2007 as we did for 2009. Thus, for 2005 and 2007 we have also a set of journal papers with priors, a set of journal papers without priors (from any of the top-3 conferences), and the set of priors themselves.

We also sampled a set of conference papers that were not priors of the 2005 and 2007 journal papers. These samples were obtained by determining the number of priors in each of the top-3 conferences and then randomly sampling twice that number of papers for each conference/year. Thus if there were 11 papers that were priors of the 2005 journals and that were published in ICCV 2003, then we randomly sampled 22 papers from ICCV 2003 taking care that the sample did not include any of the 11 priors. Note that some of these 22 sampled papers may have been priors of a paper published in one of the top-3 journals, in any year other than 2005, but we did not check that.

We collected the citation counts, using Google Scholar in July 2011, for all these sets of papers: journal papers with priors (2005 and 2007), journal papers without priors (2005 and 2007), the priors (2005 and 2007), and the sample of conference papers that were not priors (2005 and 2007).

There is an important bias against the journal papers when we compare their citation counts to their priors. Since conference papers that are priors are, by definition, published earlier than their corresponding journal papers, the conference papers have more time to collect citations, thereby giving them an unfair advantage when it comes to number of citations. To balance this bias, we also collected the citation counts each paper received within some equal window of time since its publication. For the 2005 journal papers and their priors we computed the number of citations received within a 5 year period after publication, and for the 2007 data, within the period of 3 years. We call it a “windowed” citation count. The standard count—all citations received since publication until July 2011—is called “non-windowed”. Unfortunately, the windowed citation counts create a small bias against the conference papers. Since the area is growing, and there are an increasing number of papers in journals and conferences, censoring the counts at some earlier time as the journal paper does not allow the priors to benefit from the natural growth in citations with time. Thus, in this paper we consider the windowed and the non-windowed counts as lower and upper bounds of what an unbiased, “correct” citation count would be.

We then performed the following comparisons:

  • the citation counts both windowed and non-windowed of journal papers with priors and the priors themselves (for 2005 and 2007);

  • the (windowed and non-windowed) citation counts of journal papers with and without priors (for 2005 and 2007);

  • the (windowed and non-windowed) citation counts of the priors themselves and the sampled conference papers that were not priors (for 2005 and 2007).

Time between submission and acceptance

We computed the number of days between the “received” date (considered the submission date) and the “accepted” date as stated in the articles themselves for each 2009 journal paper.

Statistical analysis

We performed statistical analysis when it was necessary to compare two groups of measurements, for example, to compare the citation counts or the review time of journal papers with and without priors.

The data for citation count is very skewed, and thus very non-normal, with a few papers with thousands of citation counts, while most of them have less than one hundred. Thus, to compare sets of citation counts we used non-parametric tests: Wilcoxon signed-rank test for paired data, and Wilcoxon rank sum (also known as Mann–Whitney U test) for non-paired data. These tests, in simple terms, verify if it can be said that one of the sets of data has a median that is significantly larger than the other median. Additionally, we compared sets of citation counts using the probability ranking (Franceschet and Costantini 2011; Stringer et al. 2008). The probability ranking test computes the probability that a randomly chosen article in one distribution has more citations than a randomly chosen article in some other distribution.

The probability that a random element from distribution X has more citations than a random element from distribution Y is estimated by the proportion of those elements in sample of X that indeed have higher citation counts than those in sample of Y over the total possible number of pairings. That is:

$$ pr = \frac{| \{(x_i,y_j) \; \hbox{such\; that} \; x_i > y_j,\; x_i \in X, y_j \in Y\}|}{|X| |Y|} $$

where |X| is the size of set X, and x i  > y j indicates that item x i has more citations than item y j . The time to accept data is reasonably normally distributed, so we used the standard t-test to compare the time to acceptance of the journal papers with and without priors.

All calculations were performed using the statistical software RFootnote 9 and the confidence level used is 95%.

Results

Percentage of journal papers with priors

See Table 1 for the number of papers with prior and the total number of papers considered.

Table 1 Proportion of journal papers with priors
Full size table

In 2009, IJCV published 38% of papers with priors (23 with priors out of a total of 61 papers), TPAMI published 34% of papers with priors (46 with priors, 134 total), and CVIU published 20% of papers with priors (17 with priors, 84 total.) Overall, among all top-3 vision journals in 2009, approximately 31% are papers with priors. In 2007, IJCV published 37% of papers with priors (24 with priors out of a total of 65 papers), TPAMI published 36% of papers with priors (49 with priors, 137 total), and CVIU published 16% of papers with priors (9 with priors, 58 total.) In 2005, IJCV published 50% of papers with priors (18 with priors out of a total of 36 papers), TPAMI published 27% of papers with priors (31 with priors, 114 total), and CVIU published 16% of papers with priors (10 with priors, 62 total).

On average, for the three journals and for the 3 years, 30% of the papers had priors.

Analysis of the survey

The response rate of the survey was 42% (111 responses out of 264). Table 2 displays the survey results. The results show that although computer vision researchers are not required to first publish in one of the top-3 conferences before publishing in one of the top-3 journals, they occasionally do (30% of the time) and many believe that it is required or at least improves the chances of being published in the journal. Taking into account that 30% of journal papers were submitted (and accepted) into a top-3 conference, and that 24% of authors without prior responded that they submitted (and were rejected) to one of the top-3 conferences, approximately 47.5% of top-3 journal papers are first submitted to a top-3 conference.

Table 2 Answers to the survey
Full size table

We point out some inconsistency in the answers given to question P4 and NP5. Although only 11 respondents answered “Yes” to P3 which was the prerequisite to P4, at least 15 respondents answered P4. For NP4, 5 answered “Yes” but at least 8 answered NP5. The only explanation for this is that some respondents who answered “No” to the previous question, answered P4 or NP5 and they should not have, but we report all the answers that the respondents gave.

Of the authors of top-3 journal papers with prior surveyed, 54.5% say that a previous paper accepted in one of the top-3 conferences improves the chances of a paper being accepted in one of the top-3 journals, whereas 13.6% actually say that it is necessary. In addition, 47.6% of the authors of top-3 journal papers without prior surveyed say that a previous paper accepted in one of the top-3 conferences improves the chances of a paper being accepted in one of the top-3 journals, whereas zero say that it is necessary. In other words, over half of those authors surveyed stated that a top-3 conference paper improved their chances of getting the journal paper published.

We also asked top-3 journal authors about their thoughts on whether it was better to have their paper accepted and published in one of the top-3 conferences than having it accepted and published in one of the journals. Note that we left the interpretation of the word “better” up to the survey taker, however, the next question asked for the sense(s) in which it is better, with five specific ways listed as well as an Other (please specify) option. 25% of authors of journal papers with prior say that it is better to publish in the conference proceedings, whereas 7.5% of authors of journal papers without prior say that it is better. Most of those agreed that among the reasons it is better to publish at conferences is that the turnaround time is shorter than journals (76.6% stated this) and that other researchers in the field that they care to reach will be at the conferences and will see their work (70% stated this). About half of the respondents stated that papers in the top-3 conferences receive more prestige, get more citations, and will be more widely read.

A representative sample of the other reasons the respondents stated are listed below. The first list is from authors of a journal paper without prior.

  1. 1.

    Oral presentation standards can be higher than journals; possibility of award.

  2. 2.

    Choice for journal was constrained by (a) student almost finished and applied for another job; (b) journals very important in the Netherlands to obtain funding (in competition with astronomy and mathematics).

  3. 3.

    It is always better to have a paper in TPAMI than a paper in any conference. Better still to have a paper that goes through a good conference—not only these three—and then into TPAMI.

This second list is a representative sample from authors of a journal paper with prior.

  1. 1.

    professional networking

  2. 2.

    I gave question three (better) a strictly qualified yes. The ONLY sense a conf. paper is better is that it is typically more timely.

  3. 3.

    Journal paper is a more in-depth study of the topic than that published at CVPR. It is almost impossible to replicate work based on a conference paper.

Analysis of number of citations

This first comparison is between the citation count of journal papers with priors and their corresponding priors. This comparison answers whether a paper in a prestigious journal receives more citations than the “equivalent” paper (in our case the prior) in a prestigious conference.

The median non-windowed citation count for the 2005 prior conference papers is 53 and the median for the journal papers is 64. The Wilcoxon signed-rank test resulted in a p-value of 0.4512, that is, there is no significant difference between the median of the two sets. Furthermore, the probability that a journal paper will have more citations than its prior is 0.49, that is, it is as likely that the journal paper will receive as many citations as its prior, as the other way around.

The median 5-year-windowed citation count for the 2005 prior conference papers is 40 and the median for the journal papers is 59. The Wilcoxon signed-rank test resulted in a p-value of 0.0372, that is, there is a significant difference between the median of the two sets. Furthermore the probability that the journal paper will receive more citations within a 5 year window than the “equivalent” conference paper is 0.64.

The 2007 data shows that for the windowed data, the journal paper and its prior have basically the same number of citations, and the same probability of receiving more citations than the other. For the non-windowed data, the prior conference paper has both a significantly higher median (p-value = 0.001415) and a higher probability (1 − 0.26 = 0.74) of receiving more citations than the corresponding journal paper.

The non-windowed citation count shows that in the short term (3–5 years), conference priors receive more citations than the following journal paper. After only 3 years of publication of the journal paper, the citation counts for the prior papers are significantly higher. After 5 years, they are essentially the same, and it is likely that sometime after 5 years, the journal paper would accumulate more citations than its prior. The windowed data, which should compensate for the bias against the journal paper citations, shifts the point of equivalence to 3 years after the journal publication, and after 5 years shows clearly that the journal papers accumulate more citations than their priors.

This result seems to indicate that there is inertia that lasts around 3 years in citation practices in CV during which one would prefer to cite the original paper in which research was published, as opposed to a more elaborate (and longer) follow-up paper. After this period, the journal paper starts being cited more frequently than its prior. These results qualify the results by Franceschet (2010) who found that CS conference papers generated much fewer citations than CS journal papers. We found that the citation count for those conference papers that are priors to journal papers is in fact higher than their follow-up papers in the short term (3–5 years), and only after that period do they follow the general pattern of journal papers receiving more citations than conference papers.

The second comparison is between journal papers with priors and without priors. The median number of non-windowed citations for the 2005 journal papers without priors is 28, while the median for the papers with priors is 64. The difference is statistically significant (p-value = 0.000418, Wilcoxon rank sum test). Furthermore a random paper with prior would have a probability of 0.66 of having more citations than a random journal paper without priors. The results for the 2007 data are similar: the median citation count for the journal paper with priors is statistically significantly larger (p-value = 0.02323) than the median for journal papers without priors, and papers with prior have a 0.58 probability of having more citations than papers without priors. Although the windowed citations data for this analysis is not as critical as it is for the first comparison, we report on the results in Tables 3 and 4 and they reaffirm the non-windowed analysis.

Table 3 2005 median citation counts and probability that first set has higher citation counts than second set using probability ranking
Full size table
Table 4 2007 median citation counts and probability that first set has higher citation counts than second set using probability ranking
Full size table

We also compared the citation counts of the priors themselves and the sampled conference papers that were not priors. The median non-windowed 2005 citations for the priors was 53, while the median for the non-priors was 29.5. The difference is significant (p-value = 0.002704, Wilcoxon rank sum test). The probability that a random prior would receive more citations than a random non-prior paper is 0.64. The results for the 2007 data are similar: the median citation count for the priors is statistically significantly larger (p-value = 0.0004677) than the median for the sample of non-priors, and priors have a 0.63 probability of having more citations than non-priors. The windowed citations data for this analysis was done as well, so that papers that were published earlier did not unduly influence the median within the set. However, between the sets, the windowed data is not as critical because the year of publication of the papers are distributed exactly the same in the non-priors set as the priors set. We report on the results in Tables 3 and 4 and they reaffirm the non-windowed analysis.

These two last results (journal papers without priors v. journal papers with priors and priors v. non-priors) seem to indicate that the prior papers were in some way “better” than the non-prior papers. It is likely that the authors realized that those papers were receiving many citations (or at least attention) and decided on improving those results for a second publication. The research reported in the prior papers was, in general, so important, that, as a conference paper they received many more citations than the other conference papers, and as journal papers, they received more citations than the other journal papers.

Analysis of time span between submission and acceptance/publication

The final comparison is the time span from submission to acceptance, of both 2009 journal papers with and without priors. We cannot measure rejection rates directly, but we can measure the time it took from submission to acceptance as a proxy of the difficulty of the whole revision process. If having a prior is some form of sift on the quality of the paper, and given that the research already received feedback, from the conference review process and from the researchers who attended the conference, one would expect that a paper with prior should have a faster acceptance period than one without.

The average number of days from submission to acceptance for papers with priors was 375.2, while the average for papers without priors was 357.1. The difference is not significant (p-value=0.4969, t-test).

Thus, against our expectation, publishing in a top-3 conference first does not reduce the review process time for the journal paper based on that prior paper. In other words, having a prior does not seem to make the acceptance of the journal paper any easier, or at least shorter.

Discussion and conclusions

This research started from certain assumptions. First, journals have higher impact factor than conferences, and that this is true for the top conferences and journals in an area. If that is true, why would researchers submit their work to very prestigious, but selective conferences? Based on the expected number of citations only, one should prefer submissions to journals, especially if they have similar rejection rates. Furthermore, conference submissions have the added complications of the costs of registering and participating in the conference, and writing the paper under severe page limits, which is very relevant in an area such as computer vision where, often, empirical validation is a necessary component of the paper.

This research started by exploring possible explanations for the question above. We heard from some researchers in the area that a previous publication in one of the top-3 conferences was, if not necessary, at least strongly recommended, for a submission to one of the top-3 journals. The survey showed that this is a strongly minority view in the community (5% of the surveyed authors believed the prior was necessary). And in fact “only” 30% of the papers in the top-3 journals had priors in one of the conferences. The number is not small, but it also indicates that it is not necessary nor strongly suggested that such a prior exists.

A second view is that the existence of a prior improves the chances of acceptance in one of the journals. That view is held by 30% of the community. We could not measure directly how “easy” the review process was for the papers with priors, but there is no significant difference between the acceptance time for papers with and without priors. Note that if reviewers for papers without priors had more questions and requested more changes in the paper, it would be likely that these added demands would reflect in the total time for acceptance.

Finally, some researchers (11% of the surveyed authors) wrongly believe that the conferences have higher impact factors than the journals.

We believe the survey data shows a divergence in the community—there are researchers for whom journal publication is the final goal, and a top-3 conference publication may not even be a goal to pursue. Of the 51 authors without priors, only 24% had submitted that research to a top-3 conference, and only 7% of them believe that conferences are “better” than journals. For different reasons, from an understanding of the higher impact factor of journals, or because, as one of the respondents mentioned, conference publications do not count as much or at all as scientific production in their evaluation metrics, these researchers seem to believe that the effort put into a top-3 conference submission is not cost-effective. The second group seems to value the conference more: 33% of the authors with priors believe that the conferences are “better” than the journals in some way. Among the reasons, as mentioned by the respondents, are: time for an outcome, professional networking, or constraints regarding students finishing their work and not being available for a long review process.

The most common answer for the reason why conferences are better than journals is the time for an outcome. Indeed, our data shows that for 2009, the average time for acceptance for the journals was very close to a year, which negatively compares to the usual 4-month period from submission to the camera-ready copy for conferences.

Many authors stated that it is better to first publish and present at a top-3 conference and then, after modifying their paper based on feedback to publish it in a top-3 journal.

We would like to note that 24% of the journal authors without priors said they submitted their paper to a top-3 conference and it was not accepted. That may indicate that publishable papers are not being accepted at conferences which unfortunately delays the dissemination of worthy ideas. On the other hand it may instead indicate that the review process of the conference ended up improving the quality of the paper significantly, therefore allowing to be published in a top-3 journal which indicates the high value of peer review for the scientific process.

The main conclusions from the percentage of journal papers that are extensions of conference papers, citations, and time-span analysis are:

  • about 30% journal papers were based on conference papers by the same authors;

  • within a short term after publication (3–5 years) journal papers with priors receive less citations than their priors, and after that period the journal papers receive more citations than the corresponding conference paper;

  • journal papers with priors are cited more than journal papers without priors;

  • conference papers that are priors are cited more than top-3 conference papers that are not priors;

  • the conclusions above imply that conference papers that are priors are cited more than journal papers without priors;

  • there is no significant difference in the review process time of top-3 journal paper submissions with prior v. those without prior.

The prior/follow-up paper pair is of a different nature than only the journal or the conference papers. The priors have more citations than similar conference papers, as the follow-up journal paper has more citations than the other journal papers. The most likely explanation is that the priors were “better” than the other conference papers, and that encouraged the authors to submit the extended version. If this explanation is correct, then journal editors requiring or strongly suggesting that a paper submitted to the journal should have already been published in one of the top-3 conferences (which is a somewhat naive conclusion from this research) would be a policy that would not work. It is not that the research has been already published in a top-3 conference that “causes” its journal paper to have more citations, it is because that research was “better” to start with.

Another result that requires some explanation is the delay in the journal paper citations to catch up with the citations of its prior. We believe that it takes time for the CV researchers to become aware that an extended version of a top-3 conference result has been published in a journal. The top-3 conferences are so well regarded among the CV community, that researchers will become aware of results published as soon as the proceedings become available. Furthermore, because of their prestige, a CV researcher may be reasonably sure that important research will be published in the top-3 conferences, and thus there is a low incentive to keep a closer look at new issues of the top-3 journals.

In very competitive research topics within CV, a researcher would have learned about a result from the prior conference publication, and accordingly, would have cited the prior. This researcher in his/her following research would likely keep citing the prior, even if there is a follow-up journal publication on the subject. Only a researcher that is newer to that research topic would have the option of learning about that result from either the prior or the journal publication, and this researcher would probably prefer the journal publication, and accordingly, cite it.

An interesting question to explore further is how similar or unique computer vision is among the subfields of computer science with respect to the relationships of publishing in the top journals v. the top conferences. We expect similar results. Another interesting thing to explore about journal papers without priors that were submitted first to a top-3 conference and were not accepted would be to determine if those papers were reviewed accurately the first time thereby either causing delay or improvement.