## Abstract

The desert ant *Cataglyphis* exhibits a strong tendency toward returning to its previous foraging direction when the last foraging run was successful. This behavior is called sector fidelity. A very simple behavioral rule, the τ-rule, has previously been identified as the possible underlying mechanism. Up to now, sector fidelity has been considered a means of facilitating navigation by exploiting familiar landmark information. We propose that sector fidelity enhances the foraging success of ants or other animals in an environment with a heterogeneous resource distribution. We tested the τ-rule and another promising behavioral rule in a very simple environment and modeled the foraging success of these strategies. For each condition, the parameters of the 2 heuristics were optimized using an evolutionary algorithm. The results of our simulations show that the τ-rule performs very well under different resource distributions and availabilities. It can therefore function as a very general adaptive foraging strategy for finding prey in landscapes with heterogeneous resource distributions.

All animals require food for growth and reproduction. Because acquiring food resources is an important factor for fitness, the ability for finding prey should be under strong selection. Therefore, animals should have effective search strategies with an outcome close to the optimum and adaptation to the environmental conditions the animals are exposed to. Optimal foraging models try to predict optimal behavior in different foraging situations, generally assuming that all relevant information about the environment is known (e.g., Charnov 1976; Green 1984). In many cases though, animals do not have complete but only limited knowledge about resource availability and distribution in their environment. To make fast and correct decisions, animals can be expected to use heuristics (also called rules of thumb). The approach of bounded rationality states that animals are equipped with simple behavioral rules, so-called heuristics, that perform well for specific tasks and in the environment in which the animal lives (Gigerenzer and Selten 2001). Heuristics often exploit key information about the environment and allow for fast decisions that usually yield good results close to the optimum (Todd and Gigerenzer 2000).

A number of studies on animal foraging behavior discuss the possibility of simple strategies being able to generate the observed behavior in situations, where animals’ choices depend on their recent experience. Krebs et al. (1978) analyzed the foraging behavior of great tits in a 2-armed bandit situation with one option being more profitable than the other. Shettleworth et al. (1988) conducted experiments with pigeons in a 2-armed bandit situation with one option being stable, the other one being variable. The proposed heuristic did not predict the observed behavior in every detail, though this does not mean that the animals did not use a heuristic. Seth (2002) proposes a rule of thumb that can very well describe the foraging behavior of bees when offered differently profitable options (see also Thuijsman et al. 1995).

The North African desert ant *Cataglyphis* lives in a habitat that could promote the usage of a heuristic as a foraging strategy. *Cataglyphis bicolor* is a single-prey loader and a solitary central place forager (Schmid-Hempel 1984). Due to the high surface temperatures in the desert, the ants do not lay odor tracks (Ruano et al. 2000). *Cataglyphis* foragers mainly carry arthropod carcasses back to the nest, but they also take desert isopods and berries. The latter ones certainly occur clumped (Schmid-Hempel 1984), and the other ones probably also have a heterogeneous distribution.

At least in some sites, predation pressure on desert ants is very high, so that foragers only have a very short lifetime during which they can make an average of about 30 foraging runs (Schmid-Hempel P and Schmid-Hempel R 1984). One study conducted in the desert in Tunisia has shown that *Cataglyphis* foragers are successful in about 40–45% of their foraging runs (Wehner et al. 2004). Considering the short lifetime, any successful foraging strategy should rely on little information that can be acquired without much effort.

*Cataglyphis bicolor* workers exhibit a strong tendency toward returning to a certain feeding direction, called sector fidelity. This behavior is also found in other individually foraging ant species, such as *Cataglyphis fortis* (Wehner et al. 2004) and *Pachycondyla apicalis* (Fresneau 1985). Wehner et al. (2004) demonstrated that this foraging behavior can be described by a very simple behavioral rule, the “τ-rule.” According to the τ-rule, an ant returns to search in the same direction if it was successful on the previous run. If a run was not successful, the ant changes direction with an exponentially decreasing tendency depending on the number of previously successful runs in the direction it last visited. Wehner et al. (2004) concluded that the τ-rule can serve as a model for the underlying proximate mechanism. However, they did not study the functional significance of this mechanism. Wehner et al. (2004) suggested that sector fidelity could be related to navigational benefits resulting from exploiting familiar landmark information. However, *Cataglyphis* ants possess excellent navigational skills (Wehner 2003), and *C. fortis* also develops sector fidelity in a landmark-free environment (Wehner et al. 2004). Thus, his behavior does not seem to be restricted to facilitating navigation but could also provide other advantages. We investigated whether this behavioral mechanism could serve as a heuristic to enhance the foraging success of *Cataglyphis* ants and whether such benefits could explain why the ants exhibit the observed sector fidelity. Our hypothesis is that the τ-rule is a very efficient search strategy in the ants’ natural environment and leads to an increased foraging success compared with random search, both in constant and changing environments. We also predict that the τ-rule performs better under the modeled circumstances than another heuristic, the ϵ-greedy strategy (for a detailed account, see next section). We also hypothesize that the τ-rule is robust against changes of its parameter because we expect a good heuristic to still perform well even if its parameters are changed slightly. Also, ants with a long lifetime should have a greater advantage by using this heuristic compared with ants with a short lifetime because they are able to exploit a high-rewarding food source for longer.

## FRAMEWORK OF THE MODEL

We treated the situation the ants face in an abstract way as a very simple instance of the multiarmed bandit problem. For a gambler, the multiarmed bandit problem is to decide which arm of a *K*-slot machine to pull to maximize his total reward in a series of trials. Each lever is associated with a certain reward distribution. Choosing one lever does not only bring in a certain reward but also results in gathering information about this lever. The optimal strategy for this problem is an optimal trade-off between exploration and exploitation (Robbins 1952; Vermorel and Mohri 2005). For our situation, each lever translates into one foraging direction the ants can choose. In our case, each foraging direction is not associated with a certain reward distribution but with a certain probability of finding a prey item. A strategy that exploits cue information about which of the possible directions is most profitable would be beneficial to the ants’ foraging success. We chose 2 different strategies, both of them heuristics, which seemed to be appropriate for the foraging task. Note that we hypothesize a nondepleting resource in our model. This is a reasonable simplifying assumption because the ants would not survive if food resources were not replenished regularly. This assumption distinguishes our model from many other optimal foraging models, especially those concerning parasitism and superparasitism by parasitoids (e.g., Iwasa et al. 1981; Spataro and Bernstein 2007) but is in accord with models concerning the ideal free distribution (Milinski 1984; Hakoyama 2003; Ohashi and Thomson 2005).

As a first heuristic, we chose the τ-rule. This heuristic was proposed as the underlying behavioral mechanism to generate sector fidelity by Wehner et al. (2004). The τ-rule can be expressed as the following (the original rule was modified slightly):with *P* being the switching probability to a different sector; τ being the scaling factor for *S*, determining how fast the ant commits itself to one sector (this parameter is being optimized by an evolutionary algorithm); and *S* being the number of successful runs in a row on the currently chosen sector (this is the only input the τ-rule uses).

Select first sector randomly.

If successful, stick to the current sector.

If not successful, switch to a different, randomly selected sector with probability

*P*= e^{−τ*S}

Note that Wehner et al. (2004) adopted this formula with *S* being the number of all successful runs in the ant's life up to the current timestep. As this original version of the τ-rule is not very well able to discriminate between high- and low-rewarding sectors, we adapted it to the environmental requirements of a heterogeneous resource distribution. A comparison of experimental results and results of the simulations showed that the degree of sector fidelity the ants exhibit can be achieved by a τ value of 0.69–2.77 (Wehner et al. 2004).

The second heuristic we tested is the ϵ-greedy strategy. This strategy was originally proposed as an algorithm for coping with a multiarmed bandit situation. Vermorel and Mohri (2005) tested this strategy besides others and found this simple strategy that resembles a heuristic to score surprisingly well. Because of the parallels between the desert ants’ situation and a multiarmed bandit situation, we tested the ϵ-greedy strategy in our very simple environment in order to compare the performance of the τ-rule to that of a different strategy. The ϵ-greedy strategy reads like following:

Choose the first sector randomly.

Choose a random sector with probability

*P*= ϵ [0 < ϵ < 1].Otherwise, choose the sector with the highest estimated (arithmetic) mean, based on the rewards received so far.

We tested both heuristics in a very simple environment to calculate the foraging success of each heuristic for different resource distributions regarding the probability of finding a prey item that will be described in detail in the next paragraphs. To achieve the maximum success of the heuristics, we optimized the parameters τ and ϵ with an evolutionary algorithm.

The model is not spatially explicit. The food distributions shown in Figure 1 therefore only give information about the overall distribution of food of the whole searching area, but not about the spatial distribution of sectors. Each of the 12 possible directions the ants can run to is represented by a probability of finding a prey item. We chose the number of foraging directions to be 12 because this number is supported by a statistical analysis of ant foraging runs by Wehner et al. (2004). Nevertheless, the number of possible foraging directions is not critical for the outcome of the simulations. In additional simulations with 8 and 16 sectors, the results differed only slightly. These differences encompassed small quantitative changes but no qualitative differences.

The ants' lifetime is modeled as an exponential decay function to resemble the natural distribution of the age of the foragers. Ants survive for at least 13 foraging runs and 54 at the most and make 26 runs on average—the average number of foraging runs in nature is about 30 (Schmid-Hempel P and Schmid-Hempel R 1984). For each run, the ant chooses a foraging direction according to the heuristic it uses. Whether the ant is successful or not is decided by chance. Therefore, at each timestep, a random number is generated and compared with the probability of finding a prey item in the corresponding sector. If the ant is successful, it receives exactly one prey item.

We calculated the foraging success of the 2 heuristics in a heterogeneous food environment. Wehner et al. (2004), who investigated whether the τ-rule could be an appropriate mechanism to describe the foraging behavior of *Cataglyphis* workers, assumed prey items to be homogeneously distributed. In such an environment, the 2 heuristics cannot benefit from information about more profitable sectors and therefore do not yield a higher foraging success than random choice. We consider it reasonable to assume that prey are distributed heterogeneously in the natural environment of *Cataglyphis*. Due to local irregularities of the desert ground, arthropod carcasses could accumulate at certain sites, so that the probability of finding food is higher in some directions than in others. Under such conditions, the ants can benefit from exploiting the information they acquire with the τ-rule or the ϵ-greedy strategy by using it for an effective search.

The distribution of prey in nature is not known. We chose 5 different types of distributions for our simulations. The chosen distributions cover several possibilities that how prey items could be distributed in nature (Figure 1). The probabilities of finding prey for each sector are allotted according to these 5 distributions. To cover the whole spectrum of mean finding probabilities, the sector with the highest probability of finding prey was varied between 0.1 and 1 (in steps of 0.1) by multiplying the values on the *y* axis of Figure 1 with 1/10 to 10/10. For the distributions A, B, and C, the sector with the highest finding probability is twice as good as the mean finding probability. Distributions D and E have different mean finding probabilities.

We tested the τ-rule both in a static and in a changing environment. In the static environment, all probabilities of finding prey remained associated with the same sector over time. In the changing environment, the heterogeneity of the distributions remained constant but finding probabilities changed over time. In 1/6 of the runs, the finding probability of the sector the ant currently visited switched place with another finding probability. In nature, it is possible that environmental conditions such as prey locations could change suddenly, which is simulated by our program. The ϵ-greedy strategy was only tested in the static environment. Because this strategy calculates the fraction of successful runs for every foraging direction, the estimates would become equal for every sector after some runs in the changing environment. Under these conditions, it would not be advantageous to use such a strategy. This disadvantage could be compensated by integrating a leaky memory into the strategy.

## RESULTS

### Performance of the τ-rule

#### The τ-rule in a static environment

The τ-rule yields higher success rates than random choice in a static environment for all food distributions and all mean finding probabilities that we simulated (Figure 2). However, it does not yield as high success rates as an omniscient forager that knows the precise distribution of food. The success rates lie about halfway in-between those of a randomly searching forager and an omniscient forager. When the mean finding probability of food distributions is equal, the variance influences the performance. In an environment with a higher variance in the food distribution, the τ-rule yields higher success rates. Using the success rate of random search as a yardstick, the success rate of the τ-rule increases with mean finding probabilities (Figure 3). The optimal τ values determined by the evolutionary algorithm are negatively correlated with the mean finding probabilities. The relationship is nearly linear. All optimal τ values lie between 0 and 4. When the optimal τ value is 0, the ant does not rely on successful runs made in the past, and the τ-rule is reduced to the strategy “Stay on the same sector as before when you have just been successful there, otherwise switch to a different direction.” This strategy should always be optimal when there is one sector with a finding probability of 100%. A high optimal τ value signifies a high reliance on past experiences, which is optimal for low mean finding probabilities.

#### Comparison with ϵ-greedy strategy

Both heuristics, τ-rule and ϵ-greedy strategy, perform better than random choice (Figure 3). The τ-rule performs much better than the ϵ-greedy strategy, though. We performed a *t*-test to test for the difference in means of 2 samples. Each sample consisted of 10 populations, using either the τ-rule or the ϵ-greedy strategy as a behavioral strategy for foraging. Each population consisted of 120 individuals who foraged in a static environment. We then calculated the mean foraging success of each population on food distribution A for each mean finding probability shown in Figure 3. For all finding probabilities, the means differed significantly from each other (*P* < 0.01 in all cases; Figure 3 also shows the standard error of the mean for both heuristics and each mean finding probability we tested).

The ϵ-greedy strategy has optimal ϵ values between 0.3 and 0.4 for almost all tested distributions, which means that a player performs optimally when choosing sectors randomly in 30–40% of the runs. Unlike the τ-rule, the ϵ-greedy strategy has a very distinct maximum in the fitness landscape. The results of simulation runs of different ant lifetime durations show that the success of the ϵ-greedy strategy approaches that of the τ-rule for a duration of lifetime of 2000 runs.

#### The τ-rule in a changing environment

The performance of the τ-rule in a changing environment is not as good as in a static environment. The τ-rule still yields a higher foraging success than random choice in a changing environment for mean finding probabilities higher than 0.1, but the success rate reaches only between ∼75% (distribution E) and ∼94% (distribution D) of the success rate in a static environment. The foraging success for distributions A–C in a changing environment is about 87% of the foraging success of the τ-rule in a static environment.

The optimal τ values for all distributions lie between 0 and 2.45 in a changing environment. When only considering those distributions that yield a foraging success between 40% and 45% as found in nature, the optimal τ values for these distributions are not in accordance with the estimate of Wehner et al. (2004) (0.69–2.77) but are below this estimate (<0.5), which reflects a low reliance on past experiences in the changing environment.

### Robustness of the τ-rule

#### Fitness landscape

In their model, Wehner et al. (2004) estimated the value of the realized τ to be between 0.69 and 2.77 in *Cataglyphis* by comparing the results of their model and experimental data, assuming that the ants used the τ-rule as a foraging strategy. Wehner et al. (2004) experimentally found out that the average foraging success of *Cataglyphis* ants, which is defined as prey items taken to the nest per run, lies between 40% and 45% at the Tunisian site. We were able to reproduce such an average foraging success with all 5 assumed food distributions. Some of these distributions are shown in Figure 4. The maxima of the fitness landscapes are not very distinct, so that the foraging success should be very robust against changes in τ for values above 0.75.

#### Dependence of the optimal τ value on duration of lifetime

Optimal τ values are not the same for lifetimes of different durations (Figure 5). If an ant lives only long enough to make 5 foraging runs, the optimal τ value is higher than that of an ant that makes 50 foraging runs in its lifetime. In short, the optimal τ value decreases with the duration of the lifetime. This means that an ant that can expect to perform many foraging runs should not commit itself to one sector as fast as an ant that survives only a few runs. The mean foraging success increases with lifetime duration. Also, the optimum of the fitness landscape becomes more distinct as ants live longer (Figure 5).

## DISCUSSION

Our argument that the τ-rule functions as a heuristic search strategy that enhances foraging success in *Cataglyphis* is based on the assumption that prey items are distributed heterogeneously in the ants’ natural environment. A homogeneous distribution of prey items would not promote any search strategy because an ant would find the same amount of prey items no matter where it searched. Up to now, it has not been studied how prey items are distributed around *Cataglyphis* nests. The distribution of prey items could, in fact, be random and therefore homogeneous. There is some evidence, though, that this is not the case. Some results of experimental data support our assumption of a heterogeneous prey distribution. Schmid-Hempel (1984) shows that *Cataglyphis* workers exploit different kinds of food sources, and at least one of these (berries on shrubs and herbs) has a clumped distribution. Data collected by Wehner et al. (1983) show that the azimuthal distribution of foragers around a *Cataglyphis* nest was not random on 2 different days. This result can only be explained with a heterogeneous food distribution if the ants use the τ-rule as a foraging strategy. Assuming a homogeneous food distribution around the nest would produce a random distribution of foragers. Further evidence comes from the study from Wehner et al. (2004). They found that foraging success increases with lifetime. Ants are successful in 19% of their runs during the first 3 foraging runs and have a 70% success rate during runs 28–30 (see also Schmid-Hempel P and Schmid-Hempel R 1984). Although the increase of foraging success with lifetime could be explained with longer durations of the foraging runs at the end of an ant's life, one could also argue the other way around that ants increase searching time per run because their chance to be on a high-rewarding sector increases during their forager career and thus the extra effort in searching time will be rewarded with a high probability of a successful run. Simulations show that the increase in success rate with forager lifetime could be explained with a heterogeneous prey distribution. Simulations with a homogeneous resource distribution do not produce the result that foraging success increases with lifetime but assuming a heterogeneous food distribution in the model does. In conclusion, the assumption of a heterogeneous prey distribution is in our view likely to represent the natural conditions.

As shown in the results, the τ-rule yields higher success rates than random choice for all tested food distributions. This shows that the application of the τ-rule is adaptive in an environment with a heterogeneous resource distribution. Therefore, the τ-rule is not only a possible mechanism to generate sector fidelity but also can be seen as a strategy for improving foraging success in *Cataglyphis*. Because the circular search area around the nest of *Cataglyphis* is not naturally divided into discrete sectors or patches, it is more plausible that the ant chooses approximate directions instead of sectors and deviates at least 30° from the previous direction when it switches to another sector. This way the division of the search area is not static but dynamic and can differ among ants. The study by Wehner et al. (2004) indicates that foragers prefer neighboring sectors over a completely randomly chosen new foraging direction when switching. This might be an advantage if information about adjacent sectors can be inferred from information about the current foraging sector. This point can only be addressed with a spatially explicit model.

In the heterogeneous food landscape assumed for the model, the τ-rule is well suited to discriminate between differences in the distribution of prey items between sectors and react accordingly, although it uses only one type of information, which is the number of successful runs in the last sector chosen. Therefore, the τ-rule can be called ecologically rational, meaning that it fits the demands and structure of the particular environmental niche (Seth 2002). We suggest that the τ-rule is the possible underlying mechanism for sector or site fidelity not only in *Cataglyphis* ants but also in other central place foragers, especially other ant species. Workers of the European ant *Lasius fuliginosus*, which use trunk trails radiating from the nest, show a high trail and site fidelity of individual ants as well (Quinet and Pasteels 1996). The western harvester ant, *Pogonomyrmex occidentalis* also shows fidelity in search site (Crist and MacMahon 1991).

The other heuristic we tested, the ϵ-greedy strategy, performs less well than the τ-rule. The choices that are made according to this strategy are not well adapted to the environment postulated in our model. The reason for the comparatively poor performance of the ϵ-greedy strategy does not seem to be the foragers’ short life span, though. Simulations with 2000 runs show that even with that many tries, the ϵ-greedy strategy does not perform quite as well as the τ-rule. Even if the ϵ-greedy strategy performed better than the τ-rule, it would not be a biologically plausible strategy to solve the ants’ problem of maximizing foraging success. The ϵ-greedy strategy implies a division of the foraging ground into several discrete options from which the ant chooses. Also, it requires the storage of much more information than the τ-rule and is therefore not an efficient mechanism of improving foraging success.

As demonstrated in the results (see Figure 4), the τ-rule is very robust in the sense that the parameter τ can take almost any value above 0.75 to make the strategy perform well with all different types of resource distributions and different overall availabilities of food. Many distributions, some are shown in Figure 4, fall within the bounds of values for the parameter τ that were estimated by Wehner et al. (2004) to be the realized τ values in *Cataglyphis* foragers. Schmid-Hempel (1984) shows that persistency in searching behavior at a previously visited site differs between individuals. This suggests that the realized τ value is not the same for every individual ant.

The τ-rule also yields higher success rates than random choice in a changing environment. However, the estimated optimal τ values do not fall within the bounds of τ values that seem to be used by the ants in the field (Wehner et al. 2004). This implies that the change of high- and low-rewarding food locations is not as fast as assumed in the model, although the differences in τ values might also be a result of the changes we made in the τ-rule compared with the original version by Wehner et al. (2004). It is reasonable to assume that prey availability changes its spatial pattern over time, although such a change should be rare enough to not affect a single ant repeatedly.

Although the τ-rule can explain the ants’ decisions about where to forage, it does not consider aspects of the foraging ecology such as worker size, territory size, and time spent searching. Thomas (1988) gives a theory for optimal durations of search periods dependent on prey density and patch variability for a central place forager. Goss et al. (1989) modeled the effect of the size of the foraging area and the number of foragers at the colonial level and the effect of body size and the forager's ability to remember a food source location on the individual level for noncooperative foraging social insects. Only a synthesis of these different aspects can probably give a comprehensive insight into the mechanisms affecting foraging behavior.

The advantage of modeling the situation in an abstract way is that this abstraction allows us to draw general conclusions that can be applied to any situation that fits our assumptions. As a conclusion, we summarize the conditions under which the τ-rule is an adaptive strategy: 1) The animal has several options to choose from. 2) Each option gives a reward with a certain probability. If the situation does not give a zero-or-one success, but varies in amount of reward, the τ-rule can be extended easily to such a situation. 3) The animal has no initial knowledge about the profitability of the options. 4) The quality of the options is not equal so differences can be detected. 5) The success experienced up to a certain timestep correlates with future success. This means that the τ-rule can only be superior to random search if fluctuations regarding the availability of food in time are not too strong. Also, exploitation of individual sites has to be limited, so that the food availability in one sector should not be altered much as the result of an individual finding food. This premise is fulfilled if overall prey availability is high in relation to the number of exploiting individuals or if prey is replenished regularly.

We think that these 5 premises can be applied to many foraging situations animals are faced with in nature. Usually animals are confronted with 2 or more foraging options between which they can choose. These options can be different flower types, trees, prey types, or locations, commonly termed patches in the literature. Premise 5 seems more unlikely to represent natural conditions in many situations, especially when patches are depleted. When depletion is important and resources on depleted patches are not replenished, other patches or options should replace the depleted option. This situation is similar to our simulations in a changing environment. The results show that even when changes occur frequently, the τ-rule performs well under these conditions. Nevertheless, further investigations will help to understand how animals could deal with this situation.

## FUNDING

Research Center for Mathematical Modelling in Bielefeld.

We thank Thomas Hoffmeister, Munjong Kolss, Ruediger Wehner, and York Winter for their helpful comments on previous versions of the manuscript.

## References

*Lasius fuliginosus*(Latreille) (Hymenoptera, Formicidae)

*Cataglyphis bicolor*(Hymenoptera, Formicidae)

*Cataglyphis bicolor*(Hymenoptera, Formicidae)

*Cataglyphis bicolor*(Hymenoptera: Formicidae)

*Cataglyphis bicolor*