Introduction

When previously interbreeding populations are separated into allopatric ranges, reproductive incompatibilities may evolve over time as by-products of genetic differentiation (Allmon 1991; Ramsey et al. 2003; Fitzpatrick et al. 2008; Baack et al. 2015). If such divergent groups of populations secondarily come into contact, matings between them may be impaired, and hybrid offspring may suffer from reduced fitness due to outbreeding depression (Edmands 2002).

During allopatry, the diverging populations may be exposed to different biotic and abiotic conditions and adapt to these. For example, plants may be exposed to different herbivores and pathogens that select for different resistance and tolerance mechanisms (Thompson 2005). If such groups of populations come into secondary contact, the fate of their adaptations, whether they increase or decrease in the new combined range, may be constrained by co-evolved reproductive incompatibilities. If these are strong, movement of genes from one group of populations into the other may be hindered or delayed, depending also on the strength of selection on the traits (Barton and Hewitt 1985; Arnold 1997). If reproduction between such groups of populations is only weakly impaired, mixing and hybridisation may alternatively create genetic combinations that can feed evolution of novel traits and trait combinations (Hewitt 1988; Arnold 1997; Barton 2001; Soltis 2013).

Hybridisation barriers may affect several different life and reproductive stages, and involve processes mainly determined by the gametes, the maternal or paternal tissue, and interactions between the two (Arnold 1997). In plants, premating barriers often include different flowering times of the parental populations, and attraction of different pollinators (Cruzan and Arnold 1994; Dell’Olivo et al. 2011; Ishizaki et al. 2013; Whitehead and Peakall 2014; Ma et al. 2016). When pollen is deposited on stigmas, heterotypic pollen (i.e. from the other species, line or genotype) may germinate less frequently than contypic pollen (i.e. from the same species, line or genotype), their pollen tubes may grow slower and shorter, and they may be less successful in fertilizing ovules (Carney et al. 1994; Montgomery et al. 2010; Dell’Olivo et al. 2011; Hauser et al. 2012a; Sambatti et al. 2012; Swanson et al. 2016). Fruits containing many unfertilised ovules may develop less frequently than fruits with many fertilised ovules (Bac-Molenaar et al. 2015). After fertilisation, heterotypic (hybrid) zygotes may develop less frequently into mature seeds than contypic zygotes (Hauser et al. 1997; Ramsey et al. 2003; Sambatti et al. 2012; Ma et al. 2016), due to incompatibilities between the maternal and paternal contributions to the zygote and the endosperm (Bushell et al. 2003; Rebernig et al. 2015; Lafon-Placette and Köhler 2016; Oneal et al. 2016). F1 and later generation hybrids may additionally suffer from a low fitness, especially at reproduction where formation of viable gametes and offspring can be impaired (Hauser and Jørgensen 1998; Rieseberg 2000; Martin and Willis 2007; e Silva et al. 2012). A question that has been discussed in recent years is whether prezygotic or postzygotic processes play a larger role in the combined reproductive isolation between divergent evolutionary lineages and species (Lowry et al. 2008; Dell’Olivo et al. 2011; Sambatti et al. 2012; Scopece et al. 2013; Ma et al. 2016).

Barbarea vulgaris R.Br. (Brassicaceae) is a self-incompatible, short-lived perennial, which grows naturally in open and temporally disturbed habitats in Denmark and western Eurasia (Rich 1987; Al-Shehbaz 1988). The subspecies or variety arcuata (here called B. vulgaris for simplicity) includes two genetically divergent “plant types” that have different geographical distributions in Eurasia (Toneatto et al. 2010; Hauser et al. 2012b; Christensen et al. 2014). One of the plant types is resistant to several specialist insect herbivores (Nielsen 1997; Renwick 2002; Badenes-Perez et al. 2014), has glabrous leaves and is therefore named “G-type” (Agerbirk et al. 2003b). The other plant type is susceptible to the insect herbivores but predominantly resistant to an oomycete pathogen Albugo sp. that causes white blister rust (van Mölken et al. 2014; Heimes et al. 2015); it has pubescent leaves and is named “P-type”. The resistance to insect herbivores of the G-plants is caused by specific triterpenoid saponins, notably hederagenin cellobioside (Shinoda et al. 2002; Kuzina et al. 2009; Nielsen et al. 2010; Augustin et al. 2012; Badenes-Perez et al. 2014); susceptible P-plants also produce saponins but of different structures. The cause of Albugo resistance is unknown. Glucosinolates, which are the major defensive metabolites in Brassicaceae (Renwick 2002), also differ between the two plant types (Nielsen 1997; Agerbirk et al. 2003a). Genes for the saponin-based insect resistance, glucosinolates and pubescence are placed in different linkage groups (Kuzina et al. 2011), suggesting that the associations between these traits in the G and P-type is due to linkage disequilibrium evolved during a period of independent evolution.

We have previously suggested that the two plant types may have diverged in different geographical ranges during the ice age(s), based on their present geographical distributions with the G-type in western Europe and the P-type in eastern Europe (Hauser et al. 2012b; Christensen et al. 2014). In Denmark and Fenno-Scandia, the two plant types co-occur but grow predominantly in separate populations, whereas hybrid and mixed populations are rare (Hauser et al. 2012b; Christensen et al. 2014). A hybridisation barrier could have evolved during allopatry and prevented or delayed merging of the two plant types in their present sympatric range. In a previous study, we showed that fewer than expected hybrid seeds were produced in an open-pollinated field experiment including both G and P-plants (Toneatto et al. 2010); however, this experiment did not allow for a comprehensive analysis of the reproductive stages involved in the hybridisation barrier or a quantification of these.

We here investigate the sexual compatibility between the two plant types in more detail, to determine whether prezygotic, postzygotic or both types of processes are involved in the reproductive isolation, and to estimate the strength of the barrier. Mixtures of G and P-plants were grown in net tents and pollinated by flies, and the resulting seeds genotyped by microsatellite markers to determine paternity. Fruit and seed development was measured, and combined with the paternity data in a statistical model to estimate fitness of heterotypic pollen from deposition on stigmas to fertilization, and survival of heterotypic and contypic embryos from seed abortion. Finally, we evaluated the pollen development of heterotypic F1 offspring.

We thus asked the following questions: (1) is pollen from G-plants, when deposited on stigmas of P-plants, less successful in siring zygotes in developing fruits than pollen from P-plants, and vice versa? (2) are hybrid zygotes, produced by heterotypic fertilisations (G × P, P × G), less successful in developing into mature seeds than contypic zygotes (G × G, P × P)? (3) how strong is the combined reproductive barrier against heterotypic hybridisation, from pollen deposition to seed dispersal, compared to published estimates from interspecific hybridisations of closely related species?

Materials and methods

Plant material and pollination

Experimental plants were grown from seeds, originating from bulk collections from several different seed plants in two Danish G-populations (Ølst: north-eastern Jutland; Nørre Vedby: Falster) and two P-populations (Tollestrup: northern Jutland; Trundholm: west Sealand). Plants were initially grown in a greenhouse (18°, 16 h light/8 h darkness) until they reached the five-leaf stage in August, after which they were moved to an outside bench for vernalisation. When flowering in May the following spring, they were transplanted to net tents to avoid unintended pollinations during the experiment. Six plants were placed in each tent, in five different proportions: 6G:0P, 5G:1P, 3G:3P, 1G:5P, 0G:6P. Each proportion was replicated three times, with a total of 15 tents and 90 plants. Plants were matched to be at approximately the same size and stage of flowering within each tent; P-plants often flower one to 2 weeks earlier than G-plants, but with an overlap of several weeks. Already opened flowers were removed to ensure that only siliques pollinated during the experiment were included in subsequent analyses. Plants were pollinated by blow flies (Lucilla), adding 15–20 individuals per tent; these flies had previously been found to be good pollinators of B. vulgaris (S. Christensen, pers. obs.). When the first plant in a tent finished flowering (after on average 11 days), the experiment in that tent was terminated and all unopened flowers on the plants removed to avoid inclusion of non-experimental seed.

When fruits were ripe, the total number of pedicels set during the pollination period was counted on each plant to determine the number of flowers it produced. Siliques were sorted and counted in three categories: fully developed (more than 2 cm long and containing predominantly fully developed seeds), partially developed (<2 cm long and containing many aborted but at least one fully developed seed), and undeveloped (containing no full seeds). A sample of 10 siliques from each plant was randomly chosen (only 9 could be included for some plants) and the seeds within sorted and counted as fully developed (seeds of normal size and spherical shape), aborted and deformed (seeds that had started to develop but aborted before reaching full size), and undeveloped (empty funiculi or tiny primordia).

Paternity determination

Offspring seeds from each parental plant were sown in the greenhouse, and after germination three to nine seedlings were randomly selected and genotyped to determine their paternity (Table 1). Leaf samples were collected on ice, freeze dried, and DNA extracted using the CTAB method (Saghai-Maroof et al. 1984). No offspring from the 6G:0P and 0G:6P tents were included, as no hybrids could have been produced here.

Table 1 Number of offspring seedlings tested and determined to have been produced by contypic or heterotypic (hybrid) matings in the experimental plant mixtures

Three unlinked microsatellite loci developed for B. vulgaris (Kuzina et al. 2011) were used for the paternity determination, based on their ability to differentiate between G and P-plants (Christensen et al. 2014): Bv65 (5 alleles; frequencies in the G/P parents: (1) 0.27/0.02; (2) 0.02/0.0; (3) 0.71/0.15; (4) 0.0/0.72; (5) 0.0/0.11), Bv83 (2 alleles: (1) 0.98/0.04; (2) 0.02/0.96), Bv154 (6 alleles: (1) 0.73/0; (2) 0.27/0; (3) 0/0.71; (4) 0/0.08; (5) 0/0.14; (6) 0/0.08). For fluorescent labelling of PCR products, the three primer approach by Schuelke (2000) was used. Each PCR reaction contained 100 ng template DNA, 0.2 mM of each dNTP, 50 nM of forward primer with 5′M13 tail, 200 nM of reverse primer, 250 nM of fluorescently labelled M13 primer (with FAM, VIC or NED), 1 × standard reaction buffer, 16.5 mM MgCl2, and 0.25 U Amplicon Taq Polymerase (VWR) in a volume of 10 µl. Following initial denaturation for 4 min at 94 °C, 18 cycles were performed with 1 min at 94 °C, 30 s at 64 °C decreasing 0.5 °C every cycle, 1 min at 72 °C, then 20 cycles with 1 min at 94 °C, 1 min at 55 °C, 1 min at 72 °C, and a final extension step of 10 min at 72 °C. Amplification products were separated by capillary electrophoresis on an ABI 3130 xI Genetic Analyzer (Applied Biosystems) and scored using the GeneMapper software from the same company. All scores set automatically by GeneMapper were checked manually.

F1 pollen development

After paternity determination of the offspring plants, they were left to grow and flower. To determine if F1 hybrids between the G and P-type suffered from reduced pollen fertility, pollen was inspected in three newly opened flowers from offspring plants (28 heterotypic hybrids and 41 contypic plants). Two mature anthers per flower were shaken over a glass slide. One drop of MTT 1 % solution (3-(4.5-dimethylthiazolyl-2)-2.5-diphenyltetrazolium bromide; thiazolyl blue in 5 % sucrose (water solution) was added (Norton 1966; Khatun and Flowers 1995), and the slide kept at 37° for 30 min to dry. After that, a cover slide was mounted with a drop of glycerol, and 300 random pollen grains were scored as either fully developed, if deeply coloured pink/violet/brown and having a spherical oval shape, or as undeveloped, if not stained, only lightly stained, or small and deformed; 12.900 pollen grains were thus inspected in total.

Data analysis of seeds, siliques and paternity

To test whether siliques and seeds developed less frequently in tents with higher proportions of heterotypic plants, the proportions of fully developed siliques (to the number of flowers set during the experiment, as determined by numbers of pedicels) and the proportions of fully developed seeds (to the number of possible seeds, i.e. all seed categories summed) were tested against the proportions of heterotypic plants by logistic regression. Barbarea vulgaris is predominantly outcrossing and considered a self-incompatible species [97.5 %; Toneatto et al. (2010)], and the proportions of potential heterotypic pollen donors in the mixtures were therefore adjusted accordingly, i.e. to 0.0, 0.2, 0.6, and 1.0 for G-plants in the 6:0, 5:1, 3:3, and 1:5 mixtures (G:P), respectively, and likewise for P-plants in the 0:6, 1:5, 3:3, and 5:1 mixtures. Maternal plant type (G or P) was included as explanatory variable together with its interactions. An initial model including the experimental unit ‘tent’ as random effect was analysed and gave qualitatively the same results as the simpler model without, and is therefore not discussed further. For siliques, there was a highly significant interaction between maternal plant type (P or G) and proportions of heterotypic plants (likelihood ratio test, P = 1.126e−05), and the two maternal plant types were therefore analysed separately. Analyses were done with the glmer function from the lme4 package in R version 2.15.0 (R Development Core Team 2012).

Paternity of the seeds produced during the experiment was determined using the program Cervus (Marshall et al. 1998). Given the mother plant, all other plants in the respective tent were considered candidate fathers. The most likely father was selected as the one with the highest log-likelihood, and registered as being of either the same or other plant type as the mother (G or P). To determine the strength of determination, we calculated the difference in log-likelihood between the most likely father and the most likely potential father of the other plant type. We further compared the results from the paternity analysis, whether offspring resulted from contypic or heterotypic matings, with results from a population assignment with the software STRUCTURE version 2.3.4 (Pritchard et al. 2000), with K set a priori to 2 and including the parental plants.

The determined proportions of hybrid offspring were compared to those expected from the proportions of heterotypic pollen donors in the plant mixtures (adjusted for self-incompatibility; see above) if there were no barriers to hybridisation, using χ2 tests with Yates’ correction for continuity. These analyses assume that the plants had equal numbers of flowers, flowers contained equal amounts of pollen, and that pollinators did not discriminate between plant types. For the five plants in the 5:1 mixtures there were so few hybrid offspring that the number of conspecific and hybrid offspring was combined for P and G maternal plants.

Differences in pollen development between mature contypic and hybrid offspring were tested with a Kruskal–Wallis test in R version 2.15.0 (R Development Core Team 2012).

Estimation of prezygotic and postzygotic fitness

To estimate pre- and postzygotic fitness of a pollen grain landing on a stigma of the other plant type, we extended a probability model developed by Hauser et al. (1997). In summary, the proportions of heterotypic offspring developed by the maternal plants were compared to the proportions expected from the proportions of heterotypic pollen donors in the plant mixtures, adjusted for self-incompatibility, and the proportions of undeveloped siliques and aborted seeds. If more seeds abort with increasing proportions of heterotypic plants in the mixtures, this shows that heterotypic embryos and seeds are less fit than the contypic. If heterotypic pollen survive less than contypic pollen during germination, growth and fertilisation, this may result in lower numbers of fertilised ovules with increasing proportions of heterotypic plants in the mixtures, and thereby lower proportions of developing siliques. However, if a surplus of pollen is deposited on stigmas (compared to the number of ovules to fertilise), better fit pollen may replace dead pollen and fertilise the ovules. If so, a low relative fitness of heterotypic pollen will not be expressed as a corresponding decrease of silique development but as an “unexplained” deficit of hybrids once seed survival and silique development has been accounted for (see “Discussion”). Fitness parameters were estimated by maximum likelihood, based on the experimental proportions of heterotypic pollen donors in the mixture, the observed paternity proportions, and the proportions of undeveloped siliques and aborted seeds.

The proportions of heterotypic pollen (G × P or P × G) at fertilization in plant type i, after pollen germination and growth in plant mixture j, can be described bypollen fertilisation performance

$$q_{ij} = \frac{{p_{j} v_{i}^{h} }}{{p_{j} v_{i}^{h} + (1 - p_{j} )}},\quad i \, = 1, \, 2;\,\,j = \, 1, \, 2, \, 3,$$
(1)

where v h i is the relative performance of heterotypic pollen germinating and growing in styles of plant type i (v c i , the corresponding success of contypic pollen is set to 1), and p j is the proportion of heterotypic pollen donors in the G + P mixtures (assuming self-incompatibility, see above).

If only few ovules are fertilised within siliques due to low survival of pollen, siliques may not develop and mature; the proportion of fertilised heterotypic embryos in siliques that do develop (from which seeds could be harvested) can thus be described by

$$r_{ij} = \frac{{q_{ij} w_{i}^{h} }}{{q_{ij} w_{i}^{h} + (1 - q_{ij} )w_{i}^{c} }},$$
(2)

where \(w_{i}^{h}\) is the performance of heterotypic pollen in fertilising ovules in developing siliques of plant type i, and \(w_{i}^{c}\) is the corresponding performance of contypic pollen. The denominator, \(\bar{w}_{ij} = q_{ij} w_{i}^{h} + (1 - q_{ij} )w_{i}^{c}\), describes the weighted average development of siliques, which can be estimated from observed proportions of flowers that developed into siliques (partially + fully developed); this is used as a probability in the likelihood function below.

The two processes (1) and (2) may be confounded, depending especially on the amount of pollen deposited relative to the number of ovules in the ovaries, and thus whether dead pollen can be replaced by others; see above. For simplicity, we use the term total pollen performance for the combination of the two performance components (\(v_{i}^{.} * w_{i}^{.}\)), and the term pollen fertilisation performance for the second (w i ). The first component (v i ) thus designates the processes of pollen performance that are not manifested by decreased ovule fertilisation and silique development, such as pollen competition, incompatibility reactions, and death with replacement; we call this pollen relative growth performance; see also “Discussion”.

Heterotypic seeds in developing siliques may abort more frequently than the contypic due to incompatibilities of the parental genomes, between parental and maternal tissues, and endosperm failure; the proportion of mature heterotypic seeds can thus be described by

$$s_{ij} = \frac{{r_{ij} x_{i}^{h} }}{{r_{ij} x_{i}^{h} + (1 - r_{ij} )x_{i}^{c} }},$$
(3)

where \(x_{i}^{h}\) is the survival of heterotypic seeds from seed abortion in plant type i and \(x_{i}^{c}\) the survival of contypic seeds. The denominator, \(\bar{x}_{ij} = r_{ij} x_{i}^{h} + (1 - r_{ij} )x_{i}^{c}\), describes the weighted average seed survival, which can be estimated from observed proportions of fully developed seeds (to the sum of fully developed and aborted) in the siliques; this is used as a probability in the likelihood function below.

The number o h ijk of hybrid offspring out of O ijk offspring genotyped in total for plant mixture j in plant k (k = 1,…, n) of plant type i is binomially distributed with probability parameter s ij . The number m d ijk of fully and partially developed siliques out of \(M_{ijk}\) potential siliques (sum of full, partial and undeveloped ~ number of flowers) for plant mixture j in plant k of type i is binomially distributed with probability parameter \(\bar{w}_{ij}\). The number of fully developed seeds n d ijk out of N ijk embryos (sum of developed and aborted seeds) for plant mixture j in plant k of type i is binomially distributed with probability parameter \(\bar{x}_{ij}\).

The logarithm of the product of the three binomial probabilities gives the likelihood function

$$\ln L{\text{ }} = {\text{ }}\sum\limits_{{i = 1}}^{2} {\sum\limits_{{j = 1}}^{3} {\sum\limits_{{k = 1}}^{n} {\left\{ {\begin{array}{*{20}l} {\ln \left[ {\left( \begin{aligned} O_{{ijk}} \hfill \\ o^{h} _{{ijk}} \hfill \\ \end{aligned} \right)(s_{{ij}} )^{{o^{h} _{{ijk}} }} (1 - s_{{ij}} )^{{(O_{{ijk}} - o^{h} _{{ijk}} )}} } \right] + \ln \left[ {\left( \begin{aligned} M_{{ijk}} \hfill \\ m^{d} _{{ijk}} \hfill \\ \end{aligned} \right)(\bar{w}_{{ij}} )^{{m^{d} _{{ijk}} }} (1 - \bar{w}_{{ij}} )^{{\left( {M_{{ijk}} - m^{d} _{{ijk}} } \right)}} } \right]} \hfill \\ { + \ln \left[ {\left( \begin{aligned} N_{{ijk}} \hfill \\ n^{d} _{{ijk}} \hfill \\ \end{aligned} \right)(\bar{x}_{{ij}} )^{{n^{d} _{{ijk}} }} (1 - \bar{x}_{{ij}} )^{{(N_{{ijk}} - n^{d} _{{ijk}} )}} } \right]} \hfill \\ \end{array} } \right.} } } ,$$

assuming that these processes (1–3 above) are independent (see above and “Discussion”). The model was analysed in R version 2.15.0 (R Development Core Team 2012) to estimate the unknown parameters that describe pollen performance (v h i , w h i and w c i ), and survival from seed abortion (\(x_{i}^{h}\) and x c i ). Hypotheses on the magnitude of these unknown parameters were tested with likelihood ratio tests, as specified in Table 3. Only treatments with proportions of heterotypic plants <1 were used for model estimation; observations where p j  = 1 were not included as these were very few (6) and the effects of self-incompatibility uncertain if there is no contypic pollen available. The R codes for these analyses are available upon request.

Results

Seed and fruit set

The number of flowers produced by G and P parental plants during the pollination experiment did not differ (G: 256.3, SE = 23.1; P: 218.3, SE = 24.8; t 84.4 = 1.1216, P = 0.27).

The proportions of fully developed siliques decreased significantly with increasing proportions of heterotypic plants in the mixtures (Fig. 1a; Table 2), but to different degrees in G and P-plants. The proportions of partial siliques, in contrast, increased (Fig. 1a). Large (full) siliques contained significantly more fully developed seeds than smaller partially developed siliques (Kruskal–Wallis, K = 36.239, P < 0.0001).

Fig. 1
figure 1

Silique and seed development on G and P maternal plants in the plant mixtures, averaged across plants within each mixture. a Percentages of siliques that are either fully developed (>2 cm), partially developed (<2 cm) or undeveloped (no mature seeds). b Percentages of seeds that are either (1) fully developed, (2) deformed or aborted (stopped development before reaching full size), or (3) undeveloped or very small

Table 2 Logistic regression of the proportions of fully developed siliques and seeds as a function of the proportions of heterotypic plants in the experimental plant mixtures

The proportions of fully developed seeds also decreased significantly with increasing proportions of heterotypic plants (Fig. 1b; Table 2). Both the fractions of aborted and undeveloped seeds increased with increasing proportions of heterotypic plants.

Paternity

When examining the list of potential fathers for each offspring, as determined by Cervus, there was a large difference in log-likelihood between the most likely G and P-father (average LOD difference = 1.19e15); the assignment as contypic or heterotypic (hybrid) offspring is therefore unambiguous. The majority of offspring determined by paternity analysis to have contypic or heterotypic parentage were assigned likewise with Structure (Fig. 2, K = 2); a few of the G and P-parents were indicated by Structure to be admixed, suggesting that they may be advanced late-generation hybrids. Alternatively, rare polymorphisms may be retained at some of the marker loci used.

Fig. 2
figure 2

Re-assignment of offspring using Structure with K = 2. The paternity determined by prior analysis with Cervus (see text) is indicated on the first axis; parental plants are included for comparison. The two genetic clusters are indicated by different shades; more than one shade per individual column indicates admixture

The proportions of hybrids among the tested offspring (Table 1) were much lower than should be expected from the proportions of heterotypic pollen donors in the mixtures, if there was no reproductive barrier (all the χ 21 tests for the different mixtures have P < 0.005).

Pollen and embryo fitness

Heterotypic pollen was estimated to be only about 40 % as fit as contypic pollen during pollen germination and growth (“relative growth performance”: Table 3; Fig. 3), with no significant difference between G and P maternal plants (deviance = 0.78; P > 0.1). Heterotypic pollen was also significantly less likely to fertilise ovules in siliques that develop and mature, especially in G plants (“fertilisation performance”: appr. 80 % as likely as contypic pollen in G maternal plants, 92 % as likely in P; difference between G and P significant: deviance = 11.3, P < 0.001). The “total relative performance of pollen”, combining the two components above, was only 31 % for heterotypic pollinations on G maternal plants and 38 % on P maternal plants.

Table 3 Maximum likelihood estimates of pollen performance and seed survival to maturity for the experimental mixtures of G and P-type plants; standard errors in parentheses
Fig. 3
figure 3

Expected and observed proportions of hybrid offspring produced in the experimental mixtures of G and P-plants, as determined by the proportions of heterotypic pollen donors in the mixtures (adjusted for self-incompatibility) and the proportions of hybrids detected by paternity analysis of their offspring, respectively. Triangles indicate proportions for individual maternal plants, digits beside them the number of plants with that combination of proportions. The straight solid line indicates the proportions of hybrids expected if there was no barrier to hybridisation; broken lines indicate the cumulated deficit of hybrids due to low pollen performance from relative growth and fertilisation (upper dotted and middle hatched-dotted lines, respectively), and abortion of seeds (lower hatched line), as estimated by the statistical fitness model. See text for more explanations

Heterotypic F1 embryos developed significantly less frequently into mature seeds than contypic embryos (approx. 57 % as many, Table 3; Fig. 3); this did not differ between G and P maternal plants (deviance = 1.2; P > 0.1).

The observed proportions of hybrid offspring, in addition to the proportions predicted from the fitness model, are shown in Fig. 3.

F1 fertility

The proportions of deformed, unstained and partly stained pollen grains were much higher for the F1 hybrid offspring plants (17 %) than for contypic offspring (5 %; Fig. 4). This was the case both when comparing all the hybrid and contypic offspring combined (K 1 = 22.91, P = 1.699e−06) and when comparing only within maternal half-sib families that included both these offspring classes (K 1 = 13.22, P = 0.00028).

Fig. 4
figure 4

Proportion of malformed, partly and unstained pollen grains developed on contypic and heterotypic offspring plants when mature

Discussion

In a previous study, we found that fewer hybrids were produced than expected in a mixed open-pollinated planting of B. vulgaris G and P-individuals (Toneatto et al. 2010). Here, we analysed the hybridisation barrier between the two plant types in more detail to determine the reproductive stages involved and to estimate the strength of different components of the barrier. A significant deficit of hybrid offspring developed in the experimental mixtures of G and P-plants, and the hybridization barrier clearly affected both pre- and postzygotic processes and was rather strong. Upon maturity, heterotypic F1 offspring additionally produce less mature pollen than contypic offspring.

Prezygotic barriers

Performance of B. vulgaris pollen was severely depressed when pollinating plants of the other plant type (P-pollen on G-plants, G-pollen on P-plants). Siliques in tents with high proportions of heterotypic pollen donors developed and matured less frequently than siliques in tents with low proportions, suggesting lower successful fertilisation rates of ovules within (see also below for an alternative interpretation). Thus, our model estimated that heterotypic pollen was only 80–90 % as successful as contypic pollen in fertilising ovules. In addition, heterotypic pollen seemed to have a lower relative performance during germination and growth, approximately 40 % of the contypic pollen (Table 3), leading to shifts in paternity of the ovules that were fertilised. The latter fitness component (pollen relative growth performance: v i .) is estimated from the deficit of hybrids not accounted for by undeveloped siliques and seed abortion, and can therefore be interpreted as the results of pollen competition, incompatibility reactions and death, to the extent where less fit pollen is replaced with better fit pollen. If not replaced, pollen deaths will results in non-fertilised ovules, which will influence silique development; this is included in the “pollen fertilisation performance”.

The total pollen performance (combining the two components above) of heterotypic pollinations was only 31 % as high as for contypic pollinations in G maternal plants and 38 % in P maternal plants. Such a strong prezygotic barrier between the two plant types is surprising, as they are presently placed taxonomically in the same subspecies or even variety (Mossberg and Stenberg 2003; Frederiksen et al. 2012). Whereas a prezygotic barrier of that magnitude is not usually expected for intraspecific crosses, incompatibilities, competition during pollen germination, growth in styles, and ovule fertilization is known to severely reduce pollen success in most interspecific hybridizations (Carney et al. 1994; Gueritaine et al. 2003; Ramsey et al. 2003; Baack et al. 2015; Luca et al. 2015). In our study, the relative growth performance of heterotypic pollen did not differ between the two plant types of B. vulgaris, in contrast to many studies that have found asymmetric pollen fitness in crosses between different plant species (Carney et al. 1994; Hauser et al. 1997, 2012b; Nista et al. 2015). Such asymmetric pollen fitness has often been attributed to different style lengths of the species; styles of the two B. vulgaris plant types do not differ in this respect.

We here used a statistical approach to estimate pollen fitness, by assigning the deficit of hybrid offspring that could not be accounted for by seed abortion to prezygotic processes. Most other studies of heterotypic pollen fitness have measured pollen germination and tube growth by direct observations. Both methods have advantages and disadvantages. Selective processes acting on pollen may vary in different parts of the stigma and style, and direct measurements at a given position and time may not reflect the overall success (Walsh and Charlesworth 1992). Conspecific pollen may compete and interfere with heterospecific pollen and increase selection against hybridization (Hauser et al. 1997; Ramsey et al. 2003), which is difficult to measure by direct observations unless using differently marked pollen (Swanson et al. 2016). Our model, in contrast, estimates pollen fitness across these prezygotic processes; this approach, however, relies on interpretation of silique development and embryo survival, as discussed below.

Postzygotic barriers

Heterotypic (F1) embryos had a much lower survival to mature seeds than contypic embryos, due to increased probability of seed abortion. The relative survival to maturity of heterotypic seeds was thus estimated to be only approximately 57 % of the contypic seeds. As for the pollen fitness, this is a surprisingly strong barrier to hybridisation given that the two plant types are placed within the same subspecies or variety. Fewer seeds per fruit and lower average seed production are commonly found upon interspecific pollinations (Ramsey et al. 2003), even though this is not always the case (Rahme et al. 2009) and the magnitude may vary among sites (Aldridge and Campbell 2006).

In our statistical model, survival of the heterotypic and contypic seed was modelled with a single estimate across the different plant mixture proportions. This assumes that the survival probabilities of heterotypic embryos do not differ depending on the proportions of contypic embryos and seed within siliques (Hauser et al. 1997). This is most likely unrealistic, as developing embryos often compete for resources from the maternal tissues. Plants in the experiment aborted some seeds even in the pure contypic stands (6:0 and 0:6), probably due to lack of resources. If these resource-based abortions preferentially affect heterotypic seed in mixtures with high proportions of heterotypic plants but also contypic seed in mixtures with low proportions of heterotypic plants, heterotypic fitness would have been overestimated for the seed stage and underestimated for the pollen stage. This may be the case, but to test it would require that aborted seeds were scored for paternity, which would be difficult for especially the early abortions. Also, to estimate different heterotypic survival rates for the different mixture proportions would require quite large sample sizes, larger than used here. The estimates we obtained should thus be considered as weighted averages across the ranges of heterotypic plant proportions.

Combined barriers

When we combine the prezygotic and postzygotic fitness estimates into an estimate of total reproductive fitness from pollen deposition to mature seed (pollen relative growth performance x pollen fertilisation performance x seed survival), the relative fitness of P-pollen on G-plants is 0.18 and the relative fitness of G-pollen on P-plants 0.21 (or expressed by barrier strength (Lowry et al. 2008): 0.82 and 0.79, respectively). To evaluate the strength of this barrier we compared to published estimates from interspecific pollen mixture experiments, using their published ratios of heterospecific/conspecific seeds produced by 1:1 mixtures of parental pollen. Relative heterospecific pollen fitness calculated this way were 1.06 and 0.22 for Silene dioica and S. latifolia, respectively (Rahme et al. 2009), 1.26 and 0.22 for S. dioica and S. latifolia (Montgomery et al. 2010), 0.54 and 0.93 for Senecio chrysanthemifolius and S. aethnensis (Chapman et al. 2005), 0.08 and 0.1 for Hibiscus moscheutos and H. laevis (Klips 1999), 0.52 and 1.65 for I. aggregata and I. tenuituba (Carney et al. 1994), and 0.35 and 0.01 for Brassica napus and B. rapa (Hauser et al. 1997). From this non-exhaustive compilation, we conclude that the combined reproductive barrier between G and P-type B. vulgaris plants is as strong as, or stronger than, between several taxonomically recognised species pairs. In addition, mature F1 offspring produced significantly lower proportions of fully developed pollen than contypic plants, as is also often found for interspecific hybrids (Price and Rich 2007; Fuchs et al. 2011). Thus, F1 hybrids are negatively affected both at early life stages, e.g. during seed development, and at later stages during reproduction.

Lowry et al. (2008) suggested that prezygotic hybridisation barriers are stronger and more important than postzygotic, based on a review of available studies at that time. Our results seem to support this, as the prezygotic reproductive barriers are stronger than the postzygotic barrier from seed abortion in both the B. vulgaris plant types: relative prezygotic fitness of heterotypic pollen is 0.31 and 0.38 for G and P maternal plants, respectively, in comparison with 0.60 and 0.54 for the relative postzygotic fitnesses). Other studies, however, do not support that prezygotic barriers are stronger (Widmer et al. 2009; Sambatti et al. 2012; Scopece et al. 2013; Lindtke et al. 2014) and argue that multiple reproductive stages (pre- and post-mating as well as pre- and postzygotic), together with later hybrid fitness barriers, contribute to the combined barrier.

Our results on the relative strength of prezygotic and postzygotic barriers depend, however, on the interpretation of undeveloped siliques and seeds. We assumed that development of siliques is primarily determined by the number (or proportion) of fertilised ovules within, and not by seed abortion; we therefore include silique development as a prezygotic barrier. This may be correct, as has been suggested for A. thaliana (Bac-Molenaar et al. 2015); however, we do not know to what extent silique abortion in B. vulgaris is also affected by abortion of hybrid seeds. Our estimates of seed abortion included only the partly developed and clearly aborted seeds, and not the very small “undeveloped”. In fact, some of these could contain embryos that aborted early. If we alternatively consider undeveloped siliques to have “aborted” due to seed abortion within, and some of the “undeveloped” seeds to be early seed abortions, the relative magnitude of the postzygotic barriers would obviously increase. In any case, our results clearly show that both pre- and postzygotic barriers affect intercrossing between G and P-type B. vulgaris.

Our results are based on the assumption that G and P-plants produced approximately equal number of flowers and viable pollen during the experiment, which was delivered to the stigmas with even probabilities (i.e. no pollinator discrimination). Indeed, the number of flowers produced by G and P-plants during the experiment did not differ. A tendency for more flowers on G-plants may have contributed to the higher proportions of hybrid offspring and the slightly higher estimate of total fitness of heterotypic pollen on P-plants. Pollen numbers and development did not seem to differ between the two plant types (Fig. 4). In any case, as we harvested and tested seeds from both G and P-plants within each tent, a slight difference in pollen production, viability and delivery would have increased the estimate of heterotypic reproductive fitness (i.e. prezygotic and postzygotic combined) for one of the plant types but at the same time reduced it for the other plant type. As we found a lower combined reproductive fitness of heterotypic pollen on both G and P maternal plants, our conclusions, that the reproductive barrier is strong and affects both pre- and postzygotically, is robust to these potential biases.

Implications

The existence of a strong hybridisation barrier between the G and P-type of B. vulgaris strongly support our previous conclusion that they constitute two different evolutionary lineages that must have diverged for a substantial period of time (Toneatto et al. 2010; Christensen et al. 2014), long enough for reproductive incompatibilities to evolve. In their zone of co-occurrence in Scandinavia and Finland (Christensen et al. 2014), the hybridisation barrier probably counteract or delay mixing and introgression of traits from one plant type into the other. Thus, if the insect and pathogen resistance of the G and P-type increase plant fitness we would have expected them to spread into the range of the susceptible type if there was no or only a weak barrier (Barton and Hewitt 1985). However, this has not happened to any large extent, at least for the saponin-based insect resistance; in general the typical trait associations of the G and P-types are maintained in almost all B. vulgaris populations that have been tested from the range of co-occurrence (JK Nielsen, unpublished; Hauser et al. 2012b; Christensen et al. 2014).

Our results thus suggest that the distribution of resistances and probably other adaptive traits of B. vulgaris in the region of co-occurrence are constrained by the hybridisation barrier between the two plant types. Others studies have suggested post-glacial migration waves to have met and mixed in Scandinavia and Finland (Christensen et al. 2014), but few have asked whether the different histories of the populations affect their present ecology. De Carvalho et al. (2010) proposed that admixture between different migration lineages of aspen has contributed genetic variation for selection on bud set, and Bernhardsson et al. (2013) suggested that this admixture also influences plant defence traits and associated arthropod communities. Apart from these studies, it is an open question to what extent present day ecology of species is affected by historical ice age divergences.