Introduction

A widespread form of defence against predation is camouflage (also called crypsis), in which animals use their body colouration to remain undetected or unrecognised by predators (Thayer 1918; Cott 1940; Edmunds 1974; Stevens and Merilaita 2008; Merilaita et al. 2017; Cuthill 2019). One primary form of camouflage is background matching, in which the visual appearance of the animal (e.g. colour pattern) matches the appearance of its background (Endler 1978, 1984; Merilaita and Lind 2005; Michalis et al. 2017). Numerous studies have found support for this strategy in a variety of taxa ranging from insects (Feltmate and Williams 1989), amphibians and reptiles (Norris and Lowe 1964), fishes (Armbruster and Page 1996; Clarke and Schluter 2011), mammals (Vignieri et al. 2010), and also plants (Niu et al. 2017, 2018). However, unlike plants, most animals are not sedentary and need to move in search of food, mates and shelter (Merilaita et al. 1999). Besides, visual characteristics of the environment can also change over the lifetime of an organism, within a few seconds to hours (e.g. due to tidal cycles or water caustics: Fingerman et al. 1958; Penacchio et al. 2018; Matchette et al. 2020) or months (e.g. seasonal change in snow cover in temperate and polar regions: Litvaitis 1991, Zimova et al. 2014, 2018). Thus, organisms may encounter a range of substrates, and matching a single, visually distinct background may not be effective in such situations (Merilaita et al. 1999, 2001; Hughes et al. 2019).

Research over the past decades has identified ways in which camouflaging animals may decrease the risk of predation in a variable environment (i.e. heterogeneous in space or time). For instance, many animals are observed to change colour rapidly to match different environments (Stevens 2016; Duarte et al. 2017; Kjernsmo et al. 2020) or behaviorally prefer a background that better matches their appearance (Kang et al. 2014; Stevens and Ruxton 2019). Alternatively, animals can adapt colourations to resemble multiple backgrounds simultaneously—termed ‘compromise’ or ‘generalist’ strategy. In this case, the animal cannot strongly match any background (i.e. an animal can only partially match multiple backgrounds). This is because organisms have to compromise the extent of matching one background at the cost of mismatching the other (Merilaita et al. 1999; Houston et al. 2007; Sherratt et al. 2007; Michalis et al. 2017; Toh and Todd 2017). This is in contrast to the ‘specialist’ strategy, where individuals best resemble one of the many backgrounds (Merilaita et al. 1999).

Several theoretical and experimental studies have identified situations under which generalists and specialists might be favoured (Merilaita et al. 1999; Houston et al. 2007; Sherratt et al. 2007; Toh and Todd 2017). Experiments employing multiple, visually distinct backgrounds are difficult to undertake. Therefore, most studies have considered the optimization of background matching based on differential survivorship in two distinct backgrounds (but see Nokelainen et al. 2020). This definition of heterogeneity refers to a situation where the scale of spatial or temporal differences in visual features of the backgrounds are many orders of magnitude larger than the size of the animal—termed ‘disjunct heterogeneity’ (Bond and Kamil 2006). The occurrence of dark and light coated animals that are found on rocks made of the dark lava flow and surrounding light-coloured regions is a good example of such heterogeneity (Hoekstra et al. 2004).

The outcome of selection in such heterogeneous environments can be depicted in the form of a trade-off curve (Fig. 1a), which theoretically can be of three shapes, (i) concave, (ii) convex or (iii) linear (Levins 1968; Merilaita et al. 1999; Sherratt et al. 2007; Hughes et al. 2019) or include combinations of the three shapes (Merilaita et al. 1999). A concave trade-off curve suggests that the net survivorship (i.e. combined survivorship on both backgrounds) is higher for specialists than for generalists. This scenario can occur when the visual difference, i.e. heterogeneity, between the two background patches is relatively high, and the generalists have low survival in both the patches, whereas the specialists have high survival on their respective matching backgrounds (Merilaita et al. 1999; Toh and Todd 2017). On the other hand, a convex curve indicates that the generalists have higher net survivorship when compared to a specialist that has higher survivorship only on the matching background. Therefore, a convex trade-off curve represents a scenario where a generalist strategy is likely to evolve, whereas a specialist strategy is favoured under the concave scenario (Merilaita et al. 1999; Ruxton et al. 2004; Sherratt et al. 2007; Hughes et al. 2019). Alternatively, under a linear trade-off scenario, both the generalist and specialist strategies have equal net survivorship, and therefore, both strategies may theoretically co-exist (Merilaita et al. 1999). This is one of the conditions favouring local maintenance of polymorphism (Bond 2007). Support for the concave (Sandoval 1994; Sherratt et al. 2007), convex (Macedonia et al. 2003; Toh and Todd 2017) and linear trade-off curves (Merilaita et al. 1999; Houston et al. 2007; Toh and Todd 2017) have been found in different studies that mainly focused on the extent of visual difference between the backgrounds (Merilaita et al. 1999, 2001; Toh and Todd 2017) or the movement range of predators (Houston et al. 2007). Apart from these, it is possible that other factors, such as those that influence the detection of background matching prey, may affect the optimal strategy, but such factors largely remain untested (Hughes et al. 2019).

Fig. 1
figure 1

(a) Trade-off curves representing possible optimal background matching strategies expected based on differential fitness (i.e. survivorship or latency to attack) on two different backgrounds (A and B); adapted from (Sherratt et al. 2007; Hughes et al. 2019). (b–d) The possible survivorship scenarios of prey on complex background (solid coloured lines) for the given trade-off curve in the simple backgrounds (dotted/broken black line) to be (b) concave (specialists have higher net survivorship), (c) linear (both specialists and generalists have equal net survivorship) or (d) convex (generalists have higher net survivorship). Green lines represent higher fitness gain for specialists, orange lines indicate equal fitness gain for both specialists and generalists, and blue lines indicate higher fitness gain for the generalist in a complex background

One factor that influences target detectability is the amount of visual information present in the background, i.e. the visual complexity of the background (Duncan and Humphreys 1989; Merilaita 2003). Visually complex backgrounds contain high visual noise (i.e. a low signal to noise ratio Merilaita et al. 2017) and, therefore, are cognitively challenging for the predators to process the information (Dukas 2004), leading to increased search time (Dimitrova and Merilaita 2009, 2011). Background complexity can arise due to a variety of sources and can be of many forms based on the properties of background features and its spatial arrangement (Dimitrova and Merilaita 2011, 2014; Xiao and Cuthill 2016; Merilaita et al. 2017). Predation experiments have altered background complexity by varying the (i) diversity of visual information, for e.g. orientations of elements (Kjernsmo and Merilaita 2012), (ii) the geometric complexity of element shapes, for e.g. the ratio of perimeter to the square root of area (Dimitrova and Merilaita 2011) or (iii) based on ‘visual clutter’ (Xiao and Cuthill 2016). These studies found that background complexity increases prey detection time and, therefore, suggested that selection to maximally match the background might be relaxed on a complex background (Dimitrova and Merilaita 2009, 2011; Kjernsmo and Merilaita 2012; Xiao and Cuthill 2016). However, it is unclear whether the visual complexity of the background can affect the optimal background matching strategy in terms of generalisation and specialisation of prey colouration in heterogeneous environments.

In this study, we explored how one aspect of background complexity—defined based on the complexity of pattern elements (Dimitrova and Merilaita 2011)—influences the survivorship trade-off curve at varying levels of background heterogeneity using two independent setups of heterogeneity. We adopted a virtual predation experiment as in many previous studies (reviewed in Karpestam et al. 2013), where humans acted as ‘predators’ searching for artificial targets on a computer screen. While background heterogeneity can occur over spatial (e.g. two different habitats) or temporal scales (e.g. seasonal changes), we examined a scenario where visual backgrounds vary spatially (Cuthill 2019) and the prey adopting camouflage is of many orders of magnitude smaller than the spatial variation in background features (Bond and Kamil 2006; Sherratt et al. 2007; Toh and Todd 2017). If background complexity affects detection differently for the specialists and generalists, there are three possible detectability (or survivorship) trade-off scenarios, assuming that there are three major types of trade-off in heterogeneous environments (Merilaita et al. 1999, Sherratt et al. 2007; Fig. 1a). In other words, while the trade-off curves can take many possible shapes (Merilaita et al. 1999), for simplicity, we here predict scenarios where generalists and specialists are expected to be significantly different from each other in terms of survival. The three scenarios are (i) specialists gain more net survivorship (i.e. are more difficult to detect) due to increased complexity than do generalists and, therefore, the trade-off curve in a complex heterogeneous environment becomes more concave than in a comparable simple heterogeneous environment (green curves; Fig. 1b–d). On the contrary, (ii) generalists gain relatively more survivorship than do specialists, resulting in a more convex trade-off curve in complex heterogeneous than in a simple heterogeneous environment (blue curves; Fig. 1b–d). Finally, (iii) both generalists and specialists may gain equal net survivorship, and the shape of the trade-off curve is the same in simple and complex environments (orange curves; Fig. 1b–d). We show that the possibility of each scenario depends on the extent of the visual difference between the backgrounds.

Methods

General procedure

The experimental protocol followed the Declaration of Helsinki, and participants signed an informed consent form before taking part in the experiment. All participants were drawn from the student population of Indian Institute of Science Education and Research Thiruvananthapuram, represented both sexes and aged from 17 to 27 years, with normal or corrected to normal vision. Participants were not informed about the hypotheses being tested. However, it was not possible to employ a double-blind procedure as each participant received both simple and complex treatments (see below). The experiments involved participants performing tasks through a custom Graphical User Interface (GUI) presented on a 21.5″ touch-sensitive gamma-corrected LCD monitor (DELL S2240Tb, refresh rate of 60 Hz) (Fig. 2f; Supplementary Fig. S1). The GUI was created in MATLAB v. R2017a (MATLAB and Statistics Toolbox Release 2017a, The MathWorks, Inc., Natick, Massachusetts, USA). Participants viewed the presentation from a distance of ca. 65 cm and were given printed instructions describing how to perform the task. The GUI included a square-shaped ‘target’ on a ‘background’ (both created in MATLAB) (Fig. 2f), and each participant was presented multiple unique target-background combinations.

Fig. 2
figure 2

(a) Elements used in the backgrounds and targets in all the experiments (elements were of different achromatic shades in the actual experiments). Upper row: simple elements; lower row: complex elements. Backgrounds and targets used in colour heterogeneity setup (including the generalist targets) when the level of heterogeneity was high for (b & d) simple and (c & e) complex element shape. Note: background, targets and elements are not scaled. See Supplementary Fig. S2–S6 for other heterogeneity setups and levels. (f) Screenshot of the GUI presented to the participants (see Supplementary Fig. S7–S12 for more examples). Note that the target was not highlighted in the actual experiments

The task involved discriminating the target from the background and ‘attacking’ it as quickly as possible by touching it. Participants were asked to search for a square-shaped target that either had the same or similar elements of the background and were informed that the target may occlude some part of elements in the background (e.g. Fig. 2f). Participants were instructed to attack the target more than once if they were unsuccessful in their attempts. Before the start of each experimental session, for practice, one of the randomly chosen target-background combinations was shown to the participant highlighting the actual position of the target (Fig. 2f). We proceeded with the trials only after the participants confirmed verbally that it was clear to them what they were searching for in the background. After the target was successfully attacked in the actual trials, the participant was informed that the target had been successfully attacked (a uniform white screen with a ‘Next’ button was shown; Fig. 2f), and the time was recorded automatically (attack latency in seconds with a precision of 10 ms). If participants did not find the target within 60 s, the target was recorded as having survived for 60 s, and the next target was presented without revealing the position of the target that the participant had failed to find. A uniform white screen was shown until they pressed the ‘Next’ button, and the next target-background combination was presented (Fig. 2f). Each participant was presented with 30 such target items (3 replicates of 5 target types across two background types presented in random order; full details of target types and background types are in the subsequent sections). We acknowledge the limitation that the cognitive processes in the visual search task might differ in humans and other animals. However, this method is advantageous to test hypotheses that are difficult to investigate using experiments with real predators and prey.

Outline of experimental design

The study comprised two ‘setups’ based on how heterogeneity was defined—i.e. size and colour heterogeneity (Fig. 3). Each setup consisted of three levels of heterogeneity—low, intermediate, and high. Thus, we had a total of 6 experiments, 3 per setup representing different levels of heterogeneity. Each experiment included a comparison between backgrounds made of simple and complex elements. All targets and backgrounds of the complexity type ‘simple’ consisted, solely of ‘simple’ elements, while the target and backgrounds of ‘complex’ type consisted only of ‘complex’ elements (Fig. 2a). Here, we define the complexity of elements based on the element shape (see below). The study involved a total of 180 participants, with 30 participants per experiment. While it would have been preferable to have the same participant searching for targets of different heterogeneity levels, during our pilot experiments, many participants terminated the experiment prematurely when they were asked to search targets for more than one level of heterogeneity, possibly due to fatigue. In order to avoid participants from terminating the experiment prematurely, we used six groups of unique participants across the six experiments.

Fig. 3
figure 3

(a) Infographic depicting hypothetical backgrounds with varying level of heterogeneity and complexity in nature. An organism selected to match these backgrounds may have pattern elements similar to the characteristics of pebbles. Heterogeneity is depicted here based on the pebble size and complexity with pebble shape. (b–c) Summary of target properties. Each panel represents a heterogeneity-complexity combination, with the target label represented on top of each square target. For simplicity, only specialist (a4 and b4) and neutral generalist (a2b2) targets are shown (See Fig. 2 and Supplementary Fig. S2–S6 for a full set of targets and backgrounds). (b) size heterogeneity: the size of each individual element (in pixels) is given under each target. The difference in element size between the targets are given above arrows. (c) colour heterogeneity: the ratio of grey element colour is given under each target (black:dark grey:light grey:white). The difference in the number of the unique coloured grey element between the targets is given above the arrows. For a full set of element size and grey element ratios, see Supplementary Table S1–S4

Elements

The targets and backgrounds consisted of multiple achromatic grayscale elements and were presented upon a uniform neutral grey (R=G=B=128; luminance: 72.9 cd/m2) base. The elements were of four achromatic grey shades—white (R=G=B=255; luminance: 139.9 cd/m2), light grey (R=G=B=192; luminance: 106.9 cd/m2), dark grey (R=G=B=64; luminance: 40.1 cd/m2) and black (R=G=B=0; luminance: 5.9 cd/m2). In all experiments, the background and target contained elements that had eight unique shapes (Fig. 2a).

The element complexity index in Dimitrova and Merilaita (2011) was defined as perimeter-to-√area ratio; an element was considered ‘simple’ if the index was ≤4, and ‘complex’ if the index was ≥6. In our study, we categorised an element as simple if the index was ≤4, and complex if the index was ≥6.5 (Fig. 2a). These limits were chosen based on previous studies (Dimitrova and Merilaita 2009, 2011, 2014), who found that varying the perimeter-to-√area ratio of elements within these limits markedly affected the detectability of targets. This method of classifying the complexity of the element shape has been used in multiple studies involving both humans and birds (Dimitrova and Merilaita 2009, 2011, 2014; Toh and Todd 2017), and offers the advantage of controlling background features objectively. However, we acknowledge the limitation in classifying background complexity based on element shape and that in nature, the complexity of background need not necessarily occur due to element shape (see Xiao and Cuthill 2016; Fig. 3a for an example).

Backgrounds and target type

For each experiment, there were two background types (250 replicate each), A and B. The extent of heterogeneity was defined as the visual difference between background A and B, which was categorised into three levels—low, intermediate, and high. The difference between background A and B was characterized either based on element size (size heterogeneity setup) or the proportion of elements with a particular grey shade (colour heterogeneity setup) (Fig. 2b–d; Fig. 3b–c). The backgrounds were generated using a custom-written MATLAB script that saturated the neutral grey base by placing the elements at random positions and orientations, following procedures in Toh and Todd (2017). To retain a constant density of elements, the centres of any two elements were separated by a minimum distance of 1.305 times the length of the elements. Thus, in each experiment, all replicates (pre-generated backgrounds: n=250 for each A and B; targets: n=100) of a particular background type (A or B) had the same total number of elements and hence density. For each presentation, a random combination of target and background from the replicates was presented. The target was always square-shaped and was placed at a random position without rotation within the background (i.e. with its edges parallel to that of the rectangular background). The effective background-size was 314 mm in breadth and 224 mm in height on the monitor screen. The size of the targets and background was held constant in all the experiments.

In the experiments, for each background pair A and B, there were three target types a4, a2b2, and b4. Target types a4 and b4 were specialists of background A and B respectively, a2b2 was the generalist on both backgrounds. Target a4 matched background A better (contained characteristics of elements same (100%) as that in background A) and therefore had a lower match with the background B (0% of elements from background B). A 100% match denotes (theoretically) that the target contained elements that were identically shaped and in the same proportion and density as the background. Similarly, b4 had 100% matching elements with B, but 0% with A. Target a2b2 had 50% matching elements with both backgrounds. For example, in the colour heterogeneity setup, the ratio of the elements with different grey shades was intermediate to the element ratios of the backgrounds (see below for details). In addition, these experiments also included two more generalists which we call a3b and ab3 that better matched the backgrounds A and B, respectively. Target a3b shared 75 % of its elements with background A, more than it did with background B (25%). Likewise, the target ab3 shared 75 % of its elements with background B (75%), more than it did background A (25%). We present the results of these intermediate generalist targets in the Supplementary Materials (Figs. S13–S16).

Size heterogeneity setup

All backgrounds in this setup had equal numbers of elements with a specific grey shade. The elements in both the targets and backgrounds had an equal probability of being white, light grey, dark grey, and black in colour (see Elements section above for actual RGB values). However, we varied the size of the elements in the backgrounds to achieve heterogeneity. Since the overall shape of the element is varied, we defined the size of the element as a square containing the element shape, with the width of the square equivalent to the length or width of the element, whichever was greater. Thus, for a circle (a simple element) the size was scaled by the radius, which was the same as the width of the containing square. The width of the square, which defined the element size, was varied for each background and target based on the extent of heterogeneity. For the low heterogeneity experiment, the size of elements used in background A was 125 x 125 pixels, and B was 175 x 175 pixels (Fig. 3b). Therefore, the size difference of elements between the two backgrounds was 50 pixels. For the intermediate and high heterogeneity setup, the size of the elements in background A was 100 x 100 and 75 x 75 pixels, respectively, whereas, for background B, this was 200 x 200 and 225 x 255 pixels. Thus, for intermediate and high heterogeneity, the element size differences were 100 and 150 pixels, respectively (Fig. 3b). However, it took a greater number of elements to saturate background A because the elements of background A were smaller than that of background B. While it has been shown that a high number of elements in a given background may increase the detection time (Dimitrova and Merilaita 2014), we minimised this effect by using the same density of elements across backgrounds and targets, following procedures in Toh and Todd (2017). The exact size of the elements for all the heterogeneity levels is presented in Supplementary Table S1, S2.

The square-shaped target was 17.9 × 17.9 mm2 in size on the computer screen. For each level of heterogeneity, irrespective of the complexity of element shape, the specialist target a4 was made of the element of the smallest size (i.e. same as background A), whereas target b4 had elements of the largest size (same as background B). Irrespective of the level of heterogeneity, the generalist target a2b2 had a constant element size of 150 pixels because the element size of other target types and the backgrounds differed symmetrically from 150 pixels (Supplementary Table S1). More details on the size of targets for each setup are presented in Supplementary Table S1, S2 and Fig. S2–S4.

Colour heterogeneity setup

The proportion of different grey elements in the background was varied to achieve different levels of heterogeneity. There were more white and light grey elements in background A and a greater number of dark grey and black elements in B (Supplementary Table S3; Fig. 3c). For the low heterogeneity experiment, the ratio of the grey shade of elements (black:dark grey:light grey:white) in backgrounds A and B were 2:2:8:8 and 8:8:2:2, respectively (Table S3; Fig. 3c). That is, out of every 20 elements, a minimum of 8 elements (2 of each grey shade) were common to both backgrounds (i.e. based on the number of the grey shade of elements), and 12 were different between the backgrounds. For each heterogeneity level, the element ratio was symmetric in either direction from an equal proportion of grey shades (i.e. 5:5:5:5) for the backgrounds (A and B). That is, for intermediate and high heterogeneity, the number of unique elements was 14 (for A 1:2:8:9 and B 9:8:2:1) and 16 (for A 1:1:9:9 and B 9:9:1:1) respectively, making the backgrounds appear more different from each other. The exact element ratios for each heterogeneity level are given in Fig. 3c, and Supplementary Table S3 and also see Fig. S5–S6. In cases where the ratio of grey shade values happened not to be a whole number, we generated two subsets of replicate backgrounds or targets with different amounts of lighter or darker grey elements so that the average unique grey elements in the replicate background images is maintained. For example, 1:2:8:9 and 2:1:9:8 ratios represent the specialist target ratio 2:2:8:8 for intermediate heterogeneity, and each had a 0.5 probability of occurrence in the replicate population (Fig. 3c).

The target size was 20.14 mm in both breadth and height. Each target consisted of 20 elements (Fig. 3c). The specialist target a4 was made of the grey element at the same ratio as that of background A. Target b4 had the same grey element ratio as background B. The intermediate level of background matching can be achieved in two ways for the generalist targets—either by altering the actual matching property of the element on a continuous scale or by having a proportion of elements from the extreme backgrounds. Here, we follow procedures as in previous studies by varying the proportion of grey colour elements present in the background (Bond and Kamil 2006; Sherratt et al. 2007; Toh and Todd 2017). The element ratio 5:5:5:5 represents the generalist target (a2b2) which had an equal number of all grey elements. Exact element ratios (black:dark grey:light grey:white) of targets for each heterogeneity level are given in Supplementary Table S4 and also see Fig. 3c.

Statistical analyses

We performed a Bayesian implementation of generalized linear mixed model analysis in R (R Core Team 2019) via RStudio (RStudio Team 2015) using the function MCMCglmm in R package MCMCglmm (Hadfield 2010, 2019). Since many targets were either left undetected or attacked quickly, attack latency values tended to be bimodally distributed, especially when elements were complex. Hence, we fitted the attack latency data using a censored Gaussian distribution (cengaussian family) following procedures in Hadfield (2010). The attack latency values were first scaled before the analysis to obtain a standardised metric of effect size (Nakagawa and Cuthill 2007). In the MCMCglmm model, Participant ID was included as a random effect term and target type as the predictor variable. The MCMCglmm model was specified as MCMCglmm (cbind (attack_latency,S) ~ target, random= ~ participant_ID, family=‘cengaussian’). In this model, attack_latency is the scaled attack latency (s) while S is a binary vector indicating whether the target was attacked (attack latency < 60s) or not attacked (attack latency = 60s).

We specified an uninformative inverse-Wishart distribution (with variance, V=1 and nu= 0.002) prior following (Hadfield 2010) in our MCMCglmm analysis, and the results were not sensitive to the priors used (not shown). We ran two chains of 510,000 MCMC iterations with a burn-in of 10,000 and a thinning interval of 500, so the effective sample size was above 200. Convergence within chains was checked using diagnostics in the coda package (Plummer et al. 2006). Convergence between chains was ensured using the Gelman-Rubin statistic (Gelman and Rubin 1992), where all the fitted models had potential scale reduction factor less than 1.1. From the fitted models, we present the mean and 95% confidence interval of the posterior estimates (β coefficient) from the planned pairwise comparisons between the specialist targets (a4 or b4 as base level) and generalists (a2b2) (Ruxton and Beauchamp 2008). We considered a comparison significant if the 95% credible intervals of the posterior distribution of estimate (β) did not overlap with zero. A significantly lower attack latency for one target over another indicates that the former is a better strategy to cope with background matching in a heterogeneous environment, whereas the lack of a significant difference between target types indicates that both strategies are likely to co-exist. Thus, a positive value of the estimate indicates that attack latency was higher for the generalist target in comparison to the specialist, and a negative value indicates lower attack latency for the generalist in comparison to the specialist. All analyses were conducted separately for each complexity type (simple or complex) for all experiments. We present results based on the censored attack latency data (hereafter, ‘attack latency’) in the main text. Similar results were obtained when we categorised our data as a binary variable, i.e. when a target that was not attacked was considered to have survived (i.e. attack latency = 60 s), and targets with other latency values were considered to have not survived (Supplementary Fig. S17, S18).

Results

In general, the attack latency of specialist targets on their matched background (i.e. a4 on A, and b4 on B) was higher than that for generalist targets, and this difference between specialists and generalists increased with the level of background heterogeneity (Supplementary Fig. S13). Further, attack latency was higher in complex than in simple backgrounds (Supplementary Fig. S14, S15). However, the level of heterogeneity modulated the influence of background complexity, which we explore in detail below for each level of background heterogeneity and setup (size and colour) separately.

Size heterogeneity setup

Overall, 927 out of 2700 targets were not attacked by participants. The success of participants attacking the targets increased with the level of heterogeneity. The number of targets successfully attacked was 460, 627 and 686 for low, intermediate, and high heterogeneity respectively out of 900 targets each (Supplementary Fig. S17).

Low heterogeneity

No comparison between the generalist and specialist target types was significant when the element shape was simple (Fig. 4). For complex elements, the attack latency for generalists a2b2 was significantly higher than that for both specialists a4 and b4 (Fig. 4).

Fig. 4
figure 4

Posterior mean and 95% credible intervals (CI) of estimates (β) from MCMCglmm model comparing attack latency of specialist targets (a4 and b4) against generalists (a2b2). For simplicity, only specialist (a4 and b4) and neutral generalist (a2b2) targets are shown. See Supplementary Fig. S14 for full results including the intermediate generalists a3b and ab3. Posterior estimates not overlapping with zero (red asterisks) are considered significant. A positive value of estimate indicates that attack latency was higher for the generalist target in comparison to the specialist one, and a negative value indicates lower attack latency for the generalist in comparison to the specialist. Point and line colour indicate whether the background and target elements were simple (orange) or complex (grey). The strength of background heterogeneity is represented on top of the panel and setup (size or colour) across rows

Intermediate heterogeneity

All comparisons between the generalist and specialist target types were non-significant irrespective of element shape (Fig. 4), except for the comparisons between the generalists a2b2 and a4 for simple elements. The generalist a2b2 had significantly lower attack latency than the specialist a.4

High heterogeneity

All comparisons between the generalist and specialist target types were significant irrespective of element shape (Fig. 4). In general, the attack latency of specialists (a4 or b4) was greater than the of generalists (a2b2).

Colour heterogeneity setup

Overall, 839 out of 2700 targets were not attacked by participants. The success of participants attacking the targets increased with the level of heterogeneity. The number of targets successfully attacked was 578, 619 and 664 for low, intermediate, and high heterogeneity respectively out of 900 targets each (Supplementary Fig. S17).

Low heterogeneity

No comparison between the generalist and specialist targets was significant when the element shape was simple (Fig. 4). However, when elements were complex, generalists (a2b2) had higher attack latency than specialists (a4 or b4). That is, the comparisons between the generalist a2b2 with both a4 and b4 were significant.

Intermediate heterogeneity

None of the comparisons between the generalist and specialist targets was significant (Fig. 4). Both generalist and specialist targets had similar attack latency irrespective of the element shape.

High heterogeneity

The generalist target (a2b2) had lower attack latency than the specialist a4 irrespective of element shape. However, for the comparison against b4, the generalist target a2b2 had significantly lower attack latency when the element shape was complex and there was no difference on simple backgrounds (Fig. 4).

Discussion

Since searching for a target is challenging in backgrounds with a high amount of visual features (Merilaita 2003; Dimitrova and Merilaita 2009, 2011; Xiao and Cuthill 2016), we asked whether the optimal background matching strategy in heterogeneous environments is affected by visual complexity. Heterogeneity and background complexity can be defined (and experimentally manipulated) in multiple ways. Here, we defined background complexity in terms of pattern element shape and heterogeneity as the visual difference between backgrounds (Fig. 3a). Using attack latency as a proxy for fitness, we show that the shape of the fitness trade-off curve (i.e. optimal background matching strategy) is determined by a complex interaction between the extent of visual difference between the background (i.e. heterogeneity) and visual complexity. First, with the increase in heterogeneity, we found specialists were harder to find than generalists, indicating that the visual distinctiveness of the backgrounds has a strong influence on the optimal background matching strategy (Merilaita et al. 1999; Houston et al. 2007; Sherratt et al. 2007; Toh and Todd 2017). Second, in line with previous studies, we found that targets took a longer time to be attacked when presented on a complex background than on a simple background (Dimitrova and Merilaita 2009, 2011, 2014; Xiao and Cuthill 2016), suggesting that background complexity can have a strong influence on the outcome of selection for prey to match a particular background, probably due to the constraint on information processing by predators (Dimitrova and Merilaita 2011). Finally, we found complexity increased attack latency only when there was a threshold level of background matching. Generalist targets took a longer time to be attacked than specialist targets in a complex, less heterogeneous background. In contrast, irrespective of complexity, specialist targets are better than generalists at avoiding attack in highly heterogeneous backgrounds.

When backgrounds were simple and heterogeneity was low, the specialist target had similar attack latency as generalists. However, the generalist targets had a higher attack latency than the specialist targets when elements were complex. These results are similar to the scenario depicted by the blue line in Fig. 1c, where the generalist target had higher survival relative to the specialists, here, due to complexity. Since the generalist target is visually intermediate between the two backgrounds, the increase in attack latency due to complexity is likely to be equal across both backgrounds. On the other hand, for specialist targets, the increase in attack latency due to complexity was higher in the matching background than in mismatching backgrounds (Supplementary Fig. S13), resulting in a convex trade-off curve. Therefore, our results suggest that a complex, less heterogeneous environment favours the evolution of a generalist background matching strategy. Furthermore, we found the trade-off curve to be linear and not convex as expected when the element shape was simple, contradicting results from previous studies (Merilaita et al. 1999; Sherratt et al. 2007; Toh and Todd 2017). However, a convex trade-off curve that has been shown at low heterogeneity levels in previous studies (Merilaita et al. 1999; Sherratt et al. 2007; Toh and Todd 2017) may be due to high complexity of the background, as reported in the current study, or because of asymmetric survival of generalists in the two backgrounds (Toh and Todd 2017).

For intermediate heterogeneity, the trade-off curve was linear irrespective of whether the background contained simple or complex elements (Fig. 1c, dotted and orange lines), suggesting that both generalists and specialists can co-exist at intermediate heterogeneity levels. Thus, polymorphic crypsis can evolve when a prey animal encounters multiple distinct backgrounds but when the backgrounds are not largely discernible. Another cognitive process—‘search image formation’—may also be responsible for the existence of polymorphism in nature (Pietrewicz and Kamil 1979; Bond and Kamil 1998, 2002, 2006; Punzalan et al. 2005) reviewed in Bond (2007). In our experiments, all possible target-background combinations were presented randomly to prevent participants from encountering target-background combinations in the same order. Further, the arrangements of elements in the background and targets were also unique, probably making it hard for participants to develop a consistent search image, and therefore, it is unlikely that search image formation would have influenced our results. Nevertheless, further work is needed to understand how search image formation is different in simple and complex backgrounds, and how it affects the optimal strategy in the heterogeneous environment.

In the simple background, we found a concave trade-off curve when heterogeneity was high, which suggests that a specialist strategy is likely to evolve in an environment with multiple, highly distinct backgrounds (Sherratt et al. 2007; Toh and Todd 2017). This is likely because the generalist target, which is a compromise between the two backgrounds, can be discriminated easily from both backgrounds at a very high level of heterogeneity. On the other hand, specialists had a greater increase in attack latency due to complexity compared to generalists in the matching backgrounds (similar to the green line in Fig. 1b). These results indicate that for high heterogeneity, complexity does not benefit the generalist target and, in general, indicates that visual complexity does not necessarily benefit highly mismatching prey. Experiments on least killifish indicate that female fish with striped patterns prefer to stay on a background containing randomly oriented and overlapping stripes compared to a background with stripes matching the orientation of stripes in the fish (Kjernsmo and Merilaita 2012). Therefore, it is possible that in nature, preference for complex backgrounds could be selected for in a variable environment, which awaits further investigation. Another area for research is to test whether the preference for complex backgrounds is stronger for a generalist or a specialist prey.

In experiments testing the role of complexity on background matching, Dimitrova and Merilaita (2009) used common elements between the simple and complex backgrounds to maintain the basal level of matching between the corresponding targets. In our experiments, we did not control for such an effect as our initial pilot experiments indicated that the effect of complexity is less pronounced (at least in humans) if the background contained some simple elements. We also acknowledge that the area covered by elements was overall less in complex than in simple backgrounds because complex elements, on average, have relatively lower area than simple ones. Thus, our results have to be interpreted with caution because the level of background matching may have been different between the corresponding complex and simple backgrounds. The difference in element area between simple and complex elements could have further rendered the level of heterogeneity for corresponding simple and complex backgrounds incomparable (i.e. low, intermediate, and high heterogeneity may not be the same for simple and complex backgrounds). Further, high contrast markings (or edges) play a key role during the initial stages of image segmentation and detection (Srinivasan et al. 1987; Merilaita et al. 2017). In our experiments, targets occluded background elements, which may have altered the original shape of the elements and sometimes creating prominent edges. Occlusion by the target could have made the occluded background elements easily detectable—especially for simple elements as they have a low perimeter-to-√area ratio. Therefore, we cannot rule out the possibility of participants using different search strategies on simple and complex backgrounds. Future studies that employ a definition of complexity that is based upon natural conditions would be of interest (e.g. Xiao and Cuthill 2016).

The size and colour of elements appear to be the main features under strong selection in background matching (Endler 1978, 1981, 1984; Troscianko et al. 2016; Baling et al. 2020). Therefore, our results should be widely applicable to natural systems where background matching is achieved through diverse element features. However, we modified only a single attribute of heterogeneity (i.e. either colour or size—Fig. 3b, c) at a time in our experiments and in nature, the optimal strategy might be influenced by matching several features of the background (e.g. colour over texture; Michalis et al. 2017). We also acknowledge the limitation that we defined complexity based on element shape, and under natural conditions, many aspects of complexity may be at play (Xiao and Cuthill 2016). However, we emphasise that our experimental setup is not completely artificial, and parallels can be drawn with natural systems. For instance, the shape of elements that make up a background, such as gravel in streams (Fig. 3a), may be complex (Endler 1980) and thus make the search task more difficult for a predator.

Summary and conclusions

Previous studies have suggested that background complexity relaxes the selection to match a background accurately (Merilaita 2003; Dimitrova and Merilaita 2009, 2011, 2014; Xiao and Cuthill 2016), but how it affects the optimal background matching strategy on heterogeneous backgrounds remained unclear (Hughes et al. 2019). In this study, we found that specialists benefited more due to complexity when background heterogeneity was high, while generalists benefited more at low heterogeneity. Hence, the effect of complexity on optimal background matching strategy is conditional on the level of background heterogeneity. Ours is the first study to show that the heterogeneity of the habitat and visual complexity impact the optimal camouflage strategy of organisms.