## Abstract

Monogamy is associated with sibling-directed altruism in multiple animal taxa, including insects, birds and mammals. Inclusive-fitness theory readily explains this pattern by identifying high relatedness as a promoter of altruism. In keeping with this prediction, monogamy should promote the evolution of voluntary sterility in insect societies if sterile workers make for better helpers. However, a recent mathematical population-genetics analysis failed to identify a consistent effect of monogamy on voluntary worker sterility. Here, we revisit that analysis. First, we relax genetic assumptions, considering not only alleles of extreme effect—encoding either no sterility or complete sterility—but also alleles with intermediate effects on worker sterility. Second, we broaden the stability analysis—which focused on the invasibility of populations where either all workers are fully sterile or all workers are fully reproductive—to identify where intermediate pure or mixed evolutionarily stable states may occur. Third, we consider a broader range of demographically explicit ecological scenarios relevant to altruistic worker non-reproduction and to the evolution of eusociality more generally. We find that, in the absence of genetic constraints, monogamy always promotes altruistic worker sterility and may inhibit spiteful worker sterility. Our extended analysis demonstrates that an exact population-genetics approach strongly supports the prediction of inclusive-fitness theory that monogamy promotes sib-directed altruism in social insects.

## 1. Introduction

Altruism among animals is epitomized by the workers of eusocial insect societies, who sacrifice their personal reproductive success to promote their siblings’ welfare [1]. This remarkable self-abnegation—seemingly at odds with the ‘survival of the fittest’—is traditionally explained by kin selection: a gene causing workers to share provisions or defend the communal nest can spread if the workers’ sacrifice increases the survival of their siblings, who are likely to carry copies of the same gene [2–4]. Higher genetic relatedness between the altruist and her beneficiaries would therefore—all else being equal—promote selection for altruism [2]. Accordingly, monogamy is often highlighted as a key promoter of sibling altruism, because maternal promiscuity decreases relatedness between siblings, diminishing the inclusive-fitness benefits of sib-rearing [5–12]. A wealth of empirical evidence supports this view, revealing a strong association between monogamy and sib-directed altruism in insects [8,13,14], birds [15] and mammals [16].

Worker sterility in the social Hymenoptera (wasps, bees and ants) appears to be a conspicuous example of sib-directed altruism. In many hymenopteran species, female workers lay unfertilized eggs in their natal colony, which develop into males on account of their haplodiploid mode of sex determination. But in some species, workers have partly or entirely stopped making sons in order to focus their efforts on helping instead [4]. Accordingly, worker sterility may represent a trade-off between personal reproduction and sibling welfare, similarly to how the evolution of eusociality involves individuals forgoing dispersal in order to join an unmated worker caste [8–10]. As both empirical [13] and theoretical [12,17,18] studies have demonstrated that high relatedness promotes the evolution of a sterile worker caste, a standard account of inclusive-fitness theory might predict that—as with other forms of sibling altruism—monogamy should promote voluntary worker sterility.

However, this prediction has recently been challenged by Olejarz *et al.*’s [19] mathematical analysis of worker sterility in haplodiploid insect colonies, which uses an explicit population-genetics model to derive exact conditions for the invasion and stability of a worker-sterility allele. Surprisingly, this analysis could not identify a consistent effect of monogamy on the evolution of non-reproductive workers. Here, we revisit this analysis, exploring alternative assumptions concerning the genetics, evolution and ecology of worker sterility. We find that a more-comprehensive investigation of Olejarz *et al.*’s [19] exact population-genetics approach strongly supports the view that monogamy promotes altruistic worker sterility in insect societies and corroborates inclusive-fitness theory more generally.

## 2. Model and results

Olejarz *et al.* [19] investigated the spread of an allele that renders workers carrying the allele—who would otherwise produce sons through arrhenotokous parthenogenesis, substituting them for the queen’s sons—completely sterile. As the proportion *z* of sterile workers in a colony increases, the proportion *p*_{z} of surviving males produced by the queen rather than by workers also increases, while overall colony productivity *r*_{z} may increase or decrease. Reproductive females are assumed to mate *n* times before colony founding, such that varying *n* allows alternative scenarios of monogamy versus promiscuity (i.e. single versus multiple insemination) to be explored. Following these assumptions, Olejarz *et al.* [19] found that—in a seeming challenge to inclusive-fitness theory—voluntary worker sterility sometimes invades under single mating (*n*=1) only, sometimes under double mating (*n*=2) only, sometimes under both single and double mating, and sometimes under neither, suggesting no clear effect of monogamy on the invasion of sterility.

To explore the generality of this unexpected finding, we take up a suggestion by Olejarz *et al.* [19], p. 13 and extend their analysis to consider alleles with intermediate effects on worker sterility (as was done for a similar model by Olejarz *et al.* [20]). Intermediate-effect alleles may exhibit incomplete penetrance (such that each carrier has some intermediate probability of being sterile), or may encode intermediate phenotypes (such that each carrier divides her resources between colony tasks and personal reproduction); these scenarios are mathematically equivalent, but for ease of comparison with Olejarz *et al.* [19], we focus on the former interpretation. This suggested extension seems particularly apt, as the incomplete penetrance of sterility has been shown to be important for the evolution of reduced worker reproduction both in theory and in empirical practice [6,21–23]; indeed, the model of Olejarz *et al.* [19] assumes that sterility alleles are expressed only in workers, not in queens, so it is conceivable that sterility alleles may arise that are only expressed in a fraction of the workers who carry them. Accordingly, we have derived exact conditions for the invasion of a recessive or dominant sterility allele with arbitrary penetrance *v*, where *v*=1 represents full penetrance and 0<*v*<1 represents incomplete penetrance (see Methods).

Before continuing, we will clarify some assumptions and details of terminology. First, we adopt the assumption of Olejarz *et al.* [19] that worker sterility is voluntary—i.e. controlled by genes present in the worker herself. However, reduced worker reproduction could instead result from policing by other workers [21,24–26] or from manipulation by the queen [12,27,28–32]. The question of who controls worker sterility is critically important, because while monogamy ought to promote voluntary sterility [12,13,17,18], it should have no effect on maternally manipulated sterility [32,12], and is known to inhibit policing of worker reproduction by other workers [26,22].

Second, we focus on the case where worker sterility is altruistic, i.e. where workers sacrifice their personal reproduction such that the queen and any other laying workers can reproduce more. The alternative is that worker sterility involves spite [33] rather than pure altruism, such that in giving up her own reproduction, a worker reduces the fitness of the queen or of other workers. The model of Olejarz *et al.* [19] allows spiteful worker sterility to be analysed, which is a strength of their model so long as the fundamental difference between spiteful and altruistic sterility is acknowledged. We focus on non-spiteful sterility in the main text. In the Methods, we provide a mathematical definition of spiteful worker sterility and show how spiteful worker sterility may be inhibited, rather than promoted, by monogamy—an already well-established result in the inclusive fitness literature, where workers investing in suppressing other workers’ reproduction is known as worker policing [21,22,24–26].

Finally, we are focusing on the evolution of sterility among workers, and therefore we are assuming that a non-dispersing, unmated worker caste already exists. Olejarz *et al.* [19] set their results in contrast with Boomsma’s [8] ‘monogamy hypothesis’, which holds that monogamy promotes eusociality. But this contrast is potentially misleading [34,35], because the evolution of sterility among workers and the evolution of eusociality *per se*are separate things. We focus on the evolution of worker sterility as an elaboration—rather than as an inseparable feature—of eusociality, but briefly analyse the impact of monogamy on the evolution of an unmated (and sterile) worker caste at the end of the Model and results section.

### 2.1. Unconstrained allelic effects: monogamy promotes worker sterility

In this section, we analyse the invasion of voluntary worker sterility into a population with fully reproductive workers. In their analysis, Olejarz *et al.* [19] found that sterility can sometimes invade under promiscuity but not under monogamy, depending on how worker sterility affects colony productivity and the queen’s share of male production. This finding seems to contradict inclusive-fitness theory, because it apparently identifies cases where monogamy inhibits sibling-directed altruism instead of promoting it. We argue here that this conclusion is premature: sometimes because it rests on unjustified assumptions concerning the genetics of worker sterility, and sometimes because it confuses altruism with spite. In our extended invasion analysis, we allow worker-sterility alleles exhibiting incomplete penetrance or intermediate effects to arise, and we focus on altruistic worker sterility, rather than assuming that worker sterility may be spiteful. Accordingly, we find that there are no conditions under which altruistic worker sterility can invade under promiscuity and not under monogamy, and that monogamy is sometimes required for altruistic worker sterility to invade. In this sense, we show that monogamy always promotes the invasion of altruistic worker sterility relative to promiscuity.

We begin by considering the invasion of recessive worker-sterility alleles; we show that monogamy is always more favourable to the invasion of altruistic worker sterility than promiscuity (in the sense explained above), and we explain why allowing alleles of intermediate penetrance to arise overturns the result of Olejarz *et al.* [19] that double mating can be more favourable to the invasion of altruistic worker sterility than single mating. Then, we perform a similar analysis for dominant worker-sterility alleles, showing that monogamy is usually—but not always—more favourable to the invasion of altruistic worker sterility than promiscuity. Finally, we show that under the most general assumptions—namely, when we assume that worker sterility alleles could be dominant, recessive or incompletely dominant—monogamy is always more favourable to the invasion of altruistic worker sterility than promiscuity.

To facilitate comparison of our results with those of Olejarz *et al.* [19], in this section, we only consider whether—starting with a population in which no workers exhibit sterility—it is possible for a ‘sterility allele’ to invade, thereby rendering some workers sterile. Olejarz *et al.* [19] do not consider the equilibrium level of sterility that is expected to evolve in monogamous versus promiscuous populations, but focus on whether a sterility allele can invade from rarity to any non-zero frequency. We address the same question here, performing a more extensive analysis in the next section.

#### 2.1.1. Recessive worker-sterility alleles only

When we assume that worker-sterility alleles are necessarily recessive, and require all mutant worker-sterility alleles to show full penetrance (i.e. *v*=1), our analysis exactly recovers Olejarz *et al.*’s [19] results (figure 1*a*). However, when we assume that recessive mutant worker-sterility alleles may arise with any level of penetrance (i.e. 0<*v*≤1), we find that—strikingly—monogamy always promotes the invasion of worker sterility (figure 1*b*). To be specific, we mean that if a series of worker-sterility alleles were to arise in a non-sterile population, with each allele exhibiting a randomly selected penetrance, there are no *r*_{z} and *p*_{z} curves such that at least one allele could invade under promiscuity, but no allele could invade under monogamy (provided that worker sterility is non-spiteful; see Methods). Conversely, there are an infinite number of *r*_{z} and *p*_{z} curves for which at least one recessive worker-sterility allele could invade under monogamy, but no recessive worker-sterility alleles could invade under promiscuity.

Why does allowing incomplete penetrance—or intermediate effects more generally—make such a categorical difference? The population genetics of invasion from rarity is the key. Specifically, whether a recessive sterility allele invades depends upon what happens in colonies founded by a heterozygous female who has mated with one mutant male and *n*−1 wild-type males. Other colony types featuring the mutant allele occur, but are either comparatively rare (because they require more copies of the rare mutant allele among mating partners), or exhibit exactly the same phenotype as wild-type colonies (because sterility is expressed only when both parents pass the recessive mutant allele to their daughters). Therefore, sterility can only invade if these ‘mutant’ colonies—in which a proportion *z*=*v*/2*n* of workers are sterile—succeed in spreading the sterility allele. If we only permit alleles with full penetrance (*v*=1) to arise, this allelic constraint may overpower the altruism-promoting effect of higher relatedness: for example, double mating (*n*=2) may facilitate the invasion of sterility relative to single mating (*n*=1) if colony efficiency is relatively high when *c*). By contrast, if we permit alleles with incomplete penetrance (0<*v*≤1) to arise, mutant colonies may exhibit any one of a range of phenotypes, depending on *v* (namely, *d*).

#### 2.1.2. Dominant worker-sterility alleles only

If we assume that worker-sterility alleles are necessarily dominant, then there are two ‘mutant’ mating types which determine whether sterility can invade: a heterozygous mutant female mating with *n* wild-type males, and a wild-type female mating with one mutant male and *n*−1 wild-type males. These mating types produce colonies with a proportion *z*=*v*/2 and *z*=*v*/*n* of sterile workers, respectively. Hence, under single mating (*n*=1), it is the relative success of colonies with a fraction *v*/2 or *v* of sterile workers which determines whether a dominant sterility allele can invade, while under double mating (*n*=2), only the relative success of colonies with *v*/2 sterile workers determines whether a dominant sterility allele can invade. Therefore, if the relative success of colonies with a fraction *v*/2 of sterile workers is low, it is possible for single mating to disfavour the invasion of a worker-sterility allele relative to double mating. Nonetheless, for the scenario investigated by Olejarz *et al.* [19], fig. 8, we find that single mating always promotes the invasion of dominant sterility relative to double mating (figure 2).

#### 2.1.3. Any worker-sterility alleles

Above, we have considered the invasion of recessive and of dominant worker-sterility alleles as separate cases to facilitate comparison with the analysis of Olejarz *et al.* [19]. However, there is no biological reason to restrict our analysis to the cases where *either* all possible worker-sterility alleles must be recessive *or* all possible worker-sterility alleles must be dominant. If we simply make the assumption that both dominant and recessive worker-sterility alleles may arise, then—again assuming worker sterility is non-spiteful—it is not possible to construct *r*_{z} and *p*_{z} such that at least one sterility allele can invade under promiscuity, and yet no sterility allele can invade under monogamy (table 1). (The invasion of a worker-sterility allele with incomplete dominance *h*=*v* is mathematically equivalent to the invasion of a dominant worker-sterility allele with penetrance *v*, so the case of additivity or incomplete dominance does not need to be considered separately.) Hence, when arbitrary constraints on allelic variation are lifted, monogamy always promotes the invasion of worker sterility relative to promiscuity.

### 2.2. Beyond invasion: monogamy promotes worker sterility

We have shown that, by relaxing the strong genetic constraints imposed by the analysis of Olejarz *et al.* [19], monogamy always promotes the invasion of non-spiteful worker sterility relative to promiscuity. But to only consider whether sterility alleles can invade may be misleading, for two reasons. First, that a sterility allele spreads from rarity says little about its equilibrium frequency, which may be a more-relevant measure of monogamy’s impact upon worker altruism than mere invasion. Indeed, although as Olejarz *et al.* have shown promiscuity sometimes promotes sterility’s invasion *per se* under full penetrance, we find that monogamy typically increases the equilibrium level of sterility under the same conditions. Interestingly, we find that the ‘numerical experiments’ of Olejarz *et al.*, which identified more cases in which only double mating promoted the invasion of sterility than cases in which only single mating promoted the invasion of sterility, are highly sensitive to the method used to construct the colony productivity function *r*_{z} (figure 3).

Second, if we do allow intermediate-effect alleles, then considering only whether a single invasion occurs is inadequate, because long-term evolution is likely to involve multiple successive invasions (cf. [36]). How can we predict the outcome without knowing in advance which alleles may arise, and when? The solution is that, over the long term, populations exposed to sufficient genetic variation will converge on an evolutionarily stable strategy (ESS; [37])—a level of sterility that cannot be invaded by an allele encoding any other level of sterility. To identify a candidate ESS for sterility, we further extend Olejarz *et al.*’s [19] population-genetics analysis to derive an exact condition for the invasion of an allele encoding a small increase to average sterility, *z*:
*r*′_{z} and *p*′_{z} are the slopes of the *r*_{z} and *p*_{z} functions at *z*, respectively. Remarkably, this exact condition holds for both recessive and dominant genetics. Using this condition and a global stability analysis, we find that the ESS for sterility is always at its highest under single mating (figure 4; see Methods).

Intuition for this exact population-genetics result may be obtained by recasting condition (2.1) in terms of inclusive fitness [2]. Accordingly, natural selection favours an increase to average sterility, *z*, when
*R*_{neph}=(2+*n*)/8*n*, *R*_{sis}=(1+*p*_{z})((2+*n*)/8*n*) and

The left-hand side of condition (2.2) can be interpreted as the inclusive-fitness effect experienced by a focal worker who stops laying male eggs. The ‘sacrifice effect’ captures the direct cost of her sterility, in that she forfeits her relative share (1−*p*_{z})/(1−*z*) of all worker-laid males. The ‘efficiency effect’ captures her impact on colony efficiency, which increases by a relative amount *r*′_{z}/*r*_{z}, augmenting the production of her sisters and of colony-produced males, a proportion *p*_{z} of whom are her brothers and a proportion 1−*p*_{z} of whom are her nephews. And the ‘male production effect’ captures her impact on the proportion of male eggs produced by the queen versus workers: her relative gain of brothers is *p*′_{z}, while her relative gain or loss of nephews exactly balances her forfeited sons and gained brothers.

Condition (2.2) clarifies the impact of monogamy upon worker sterility: by increasing a worker’s relatedness to her nephews and sisters, monogamy increases her inclusive-fitness benefit of promoting colony efficiency, and by increasing a worker’s relatedness to her nephews, it increases her inclusive-fitness benefit of augmenting her fellow workers’ production of sons. Hence, overall, monogamy promotes non-spiteful worker sterility. Note that if sterility either reduces colony efficiency (*r*′_{z}<0) or reduces the reproduction of other workers ((1−*p*_{z})/(1−*z*)−*p*′_{z}<0), then worker sterility is spiteful and may be relatively promoted by promiscuity (see Methods). Condition (2.2) also clarifies how Olejarz *et al.*’s [19] model differs from Boomsma’s [8–10] model for the evolution of eusociality: in Boomsma’s model, females trade away offspring for siblings as dispersers evolve into a non-totipotent worker caste, while in Olejarz *et al.*’s model, an existing non-totipotent worker caste trades away sons for brothers and nephews. Conditions (2.1) and (2.2) are exactly equivalent, are valid for recessive, dominant or additive genetics, and can be obtained using standard kin-selection methodology (see Methods).

Conditions (2.1) and (2.2) can be derived using either the simplifying assumption that genetic variation for worker sterility is at a single locus and that new allelic variants arise via mutations of vanishingly small effect (see appendix A), or using the more general assumption that worker sterility is a quantitative trait (see appendix B). However, it is important to note that their utility in predicting an equilibrium level of worker sterility extends beyond these cases. Using an individual-based model (see Methods) to analyse a population in which mutant sterility alleles of any penetrance 0≤*v*≤1 may arise—not just those exhibiting incremental differences in penetrance—we find that the ESS predicted by conditions (2.1) and (2.2) is still reached and that monogamy still promotes the evolution of worker sterility relative to promiscuity (figure 5).

### 2.3. Alternative ecological scenarios: monogamy promotes worker sterility

Finally, we consider some alternative scenarios for the evolution of worker non-reproduction, using a demographically explicit model of queen-worker competition over egg-laying (see Methods). This yields a functional form for *p*_{z} which explicitly accounts for the relative egg-production capabilities of workers relative to the queen, which we substitute for the more hypothesis-free linear forms of the *p*_{z} function analysed by Olejarz *et al.* [19]. We then use standard neighbour-modulated-fitness methodology [38] to consider four alternative scenarios. First, we consider the original scenario of Olejarz *et al.* [19], in which workers’ sons compete only with the queen’s sons. Second, we consider a scenario in which workers’ sons compete equally with the queen’s sons and daughters, which requires analysis of sex ratio evolution because the queen is selected to adjust her sex allocation in response to workers’ sons potentially replacing her daughters. Third, we consider the evolution of soldier sterility in claustral inbreeders, such as the gall-forming thrips [39]. Fourth, we consider the evolution of a sterile worker caste via female non-dispersal, i.e. a possible scenario for the evolution of eusociality [8–10]. In all four cases, we find that monogamy always promotes non-spiteful worker sterility relative to promiscuity. Strikingly, these more-realistic scenarios identify large parameter ranges over which monogamy is critical for the evolution of worker sterility or of a worker caste (figure 6; see Methods). This conclusion also holds if we alternatively consider a diploid mode of inheritance, as exhibited by termites (figure 7; see Methods).

## 3. Discussion

In seeming contrast with the predictions of inclusive-fitness theory, Olejarz *et al.*’s [19] exact population-genetics analysis could not identify a consistent effect of monogamy on the evolution of voluntary worker sterility. This surprising result, if robust, would have not only overturned a considerable theoretical consensus, but would also have left a number of empirically described patterns bereft of a predictive, explanatory framework. Happily, we have shown that by relaxing constraints on genetic variation (figures 1 and 2 and table 1), considering the consequences of invasion rather than just its occurrence (figure 3), describing long-term evolutionarily stable states (figures 4 and 5), and exploring a wide range of ecological scenarios (figures 6 and 7), a clear sterility-promoting effect of monogamy consistently emerges. Moreover, we have shown that the long-term evolutionary outcome is readily described, conceptualized and explained by standard inclusive-fitness theory. In sum, a more comprehensive analysis based on Olejarz *et al.*’s [19] exact population-genetics approach supports inclusive-fitness theory and its prediction that monogamy promotes the evolution of altruistic worker sterility.

We have found that a distinction needs to be made between non-spiteful and spiteful worker sterility. Worker sterility may be spiteful if it either decreases colony productivity (i.e. if *r*′_{z}<0) or if, by giving up her own reproduction, a worker reduces the reproductive fitness of other workers (i.e. if *p*′_{z}>(1−*p*_{z})/(1−*z*); see Methods). When worker sterility is spiteful, monogamy may inhibit worker sterility relative to promiscuity. However, this is not because inclusive-fitness predictions for the evolution of worker sterility are wrong: on the contrary, it is a straightforward consequence of condition (2.2), an exact population-genetics result that was derived without reference to inclusive fitness, but which has a clear and intuitive interpretation in terms of a worker’s inclusive fitness. This is exactly analogous to how kin-selection methodology makes diametrically opposite predictions as to patterns of social sterility in polyembryonic parasitoid wasps depending on whether the soldiers have a family-benefit or within-family-conflict function [40,41]. It is generally understood that a worker allocating resources to egg-laying will be less able to allocate resources to colony tasks. Moreover, under the simplest assumptions, a worker abstaining from male production should, in doing so, increase the relative contribution to male production of both other workers and of the queen, which would yield a non-spiteful *p*_{z} function (see Methods) and would lead to monogamy promoting worker sterility. A potential example of worker spite is proposed by Olejarz *et al.* who suggest that—in the context of the queen policing worker reproduction—if ‘too many workers reproduce, then the queen could be overwhelmed, and her effect on removing worker-laid eggs is diminished’ [19], p. 6. This could indeed yield a spiteful *p*_{z} function if, for example, the queen were so ‘overwhelmed’ by the production of an additional worker egg that she lost track of more than one elsewhere. This is not impossible, but it does seem unlikely to be generally true, and in the absence of a concrete model or empirical support for this scenario, the assertion that spiteful worker sterility is an ‘equally plausible scenario’ [19] is difficult to accept.

Crucially, we did not derive our analysis by assuming beforehand that the evolution of worker sterility is determined by a specific condition of the form *rb*>*c* (i.e. a Hamilton’s [2] rule). Instead, we began with an explicit population-genetics model which contains no ‘built-in’ assumptions about inclusive-fitness effects. Our findings differ from those of Olejarz *et al.* [19] not because we have interpreted them using inclusive-fitness theory, but fundamentally because we have relaxed the genetic assumptions made by Olejarz *et al.* and focused on the long-term outcome of evolution rather than on the success or failure of a single invasion by a worker-sterility allele of specific effect. We then presented the results of this explicit population-genetics analysis (condition (2.1)) using an inclusive-fitness interpretation (condition (2.2)) because this form is more intuitive. This underlines that the role of inclusive-fitness theory is not usually to provide the starting point for a formal mathematical analysis, but rather to provide synthesis of—and facilitate generalization beyond—the results obtained by a diversity of different analyses undertaken using a diversity of different methodologies [42].

Although our analysis demonstrates that monogamy typically promotes worker sterility even when strong genetic constraints are assumed (figure 3*c*), we focus on the result that monogamy always promotes non-spiteful worker sterility in the absence of such genetic constraints (table 1 and figures 4–7). Formally, this analysis makes the assumption of ‘weak selection’, i.e. that allelic variation is small in magnitude so that the effect of large fitness differences between genotypes can be ignored. Does this mean that we are replacing one set of unrealistic genetic assumptions (full penetrance only) with another (weak selection)? No, because weak-selection results represent the limiting case of long-term evolution under a variety of different assumptions. Indeed, our main results are robust under a variety of evolutionary scenarios. First, they can be derived using an explicit population-genetics analysis that assumes that worker sterility is controlled by infinitesimal variation appearing at one locus at a time and that worker-sterility alleles are either dominant or recessive (appendix A). Second, they can also be derived using standard kin-selection methodology [38] which assumes additive, heritable genetic variation potentially at many loci (appendix B). Finally, we have shown that these ESS predictions are reached when we assume that allelic variation may arise at one or at many loci and that mutations typically have large effects on phenotype in a finite population subject to stochastic effects (figure 5).

The approach of Olejarz *et al.* [19] gives exact results for the invasion of worker sterility, but under extraordinary genetic constraints, namely that sterility is determined by a single locus with either recessive or dominant alleles of full penetrance. Olejarz *et al.* point out that, under these conditions, the mating number and a few points from the *r*_{z} and *p*_{z} curves are sufficient to predict whether sterility will invade. However, we rarely have this much information about any particular population of interest, let alone for all populations for which we would intend such theory to apply. It is much more likely that we will be presented with a pattern in the natural world—e.g. that voluntary sterility tends to be more common in species with monogamous mating (figure 8)—which may well be noisy. The goal of evolutionary analysis should be, first and foremost, to provide an intuitive explanation for these broad patterns, rather than trying to provide exact but difficult-to-interpret results for an idealized scenario that will never be encountered in the real world (cf. [43]). Needless to say, ecological factors—i.e. the costs and benefits of worker sterility—play a crucial role. But relatedness is also important, and we have found that monogamy promotes altruistic worker sterility across a broad range of scenarios.

## 4. Methods

### 4.1. Spiteful worker sterility and policing

In the model of Olejarz *et al.* [19], worker spite may occur via two routes—one operating through colony efficiency, *r*_{z}, and one operating through the queen’s production of males, *p*_{z}. The first case occurs when an increase in average worker sterility decreases colony efficiency (i.e. when *r*′_{z}<0)—for example, if the sterility allele has a pleiotropic effect on worker condition which results in less-efficient work. In such a case, monogamy will inhibit the evolution of worker sterility relative to promiscuity, because promiscuity decreases relatedness between relatives, thereby lessening the harmful impact of sterility upon a worker’s inclusive fitness via colony efficiency.

The second case occurs when an increase in a focal worker’s sterility harms the reproductive success of other workers. In the main text, we assume that when a worker becomes sterile, her forfeited sons are replaced partly by the queen’s sons and partly by her sisters’ sons, such that by forfeiting sons she gains both nephews and brothers, or at least does not lose nephews. But if, due to the shape of the *p*_{z} function, the queen gains a larger proportion of sons than the worker forfeits (that is, when *p*′_{z}>((1−*p*_{z})/(1−*z*))), this ‘outsized gain’ by the queen must be balanced by *decreased* male production by other workers, such that, by becoming sterile, the focal worker loses nephews overall. If the focal worker loses nephews by becoming sterile (i.e. when (1−*p*_{z})/(1−*z*)−*p*′_{z}<0; see condition (2.2)), then promiscuity, by decreasing the worker’s relatedness to nephews, may promote this spiteful form of worker sterility relative to monogamy, unless this relative cost of sterility is countered by a colony efficiency benefit of sterility, which would be largest in magnitude under monogamy.

This second form of spiteful worker sterility is connected with worker policing [24,25]. Specifically, both worker policing and this form of worker spite involve workers investing in reducing the reproduction of other workers in order to increase colony productivity. Standard inclusive-fitness theory [21,24,25] and empirical evidence [22,26] have emphasized that promiscuity promotes worker policing, so the result that this form of worker spite may be promoted by promiscuity is not at all surprising.

For non-incremental increases in sterility, the condition for spiteful worker sterility becomes (*p*_{v}−*p*_{u})/(*v*−*u*)>(1−*p*_{u})/(1−*u*), where *u* is the level of worker sterility in the monomorphic population before the mutant allele is introduced, and *v* is the level of worker sterility encoded by the mutant allele.

### 4.2. Explicit population-genetics analysis

In appendix A, we extend the methods of Olejarz *et al.* [19] to consider the invasion of an allele with an arbitrary effect on worker sterility; the results of this analysis are presented here. We find that a recessive allele encoding worker sterility *v* can invade a population monomorphic for sterility *u* when
*v* can invade a population monomorphic for sterility *u* when

Note that conditions (4.1) and (4.2) give both the invasion and stability of a given level of sterility: that is, if a sterility allele with effect *v* can invade a population monomorphic for sterility *u*, then this is the same as saying that a population monomorphic for sterility *u* is not stable to invasion by a sterility allele with effect *v*. For example, substituting *n*=1, *u*=0, *v*=1 into condition (4.1) yields the condition for the invasion of a recessive sterility allele under single mating from Olejarz *et al.* [19], their condition 1, while substituting *n*=1, *u*=1, *v*=0 into condition (4.2) yields the condition for the stability of a recessive sterility allele under single mating from Olejarz *et al.* [19], their condition 3.

To find when natural selection will favour a small increase in sterility *δz*, we make the substitution *v*=*u*+*δz* into conditions (4.1) and (4.2) above. Then, by linearizing *r*_{z} and *p*_{z} around the point *z*=*u*, we can recast these conditions in terms of the value and slope of *r*_{z} and *p*_{z} at this point. More specifically, for a recessive sterility allele, substituting *v*=*u*+*δz* into condition (4.1) yields
*r*_{z} and *p*_{z} around *z*=*u*, we replace *r*_{u+δz/2n} with *r*+(*δz*/2*n*)*r*′, where *r*=*r*_{u} and *r*′=*dr*/*dz*|_{z=u}. Similarly, we replace *p*_{u+δz/2n} with *p*+(*δz*/2*n*)*p*′, where *p*=*p*_{u} and *p*′=*dp*/*dz*|_{z=u}. This yields
*δz*^{2} or higher, substituting *z* for *u* and simplifying yields

Similarly, for a dominant sterility allele, substituting *v*=*u*+*δz* into condition (4.2) yields
*r*_{z} and *p*_{z} around *z*=*u* as above, we obtain
*δz*^{2} or higher, substituting *z* for *u* and simplifying yields

### 4.3. Numerical experiments

Olejarz *et al.* [19] performed numerical experiments to see whether sterility was more likely to invade under single mating or double mating. To do so, they constructed randomly generated *r*_{z} functions according to one of two procedures. Here, we add to these procedures, bringing the number of possible methods for constructing the *r*_{z} function to five (figure 3*a*). Each involves drawing four random variates—here, notated as *a*, *b*, *c* and *d*—from a normal distribution with mean 0 and standard deviation *σ*. In all cases, we assume *r*_{0}=1, and use the random variates to generate *r*_{1/4}, *r*_{1/2}, *r*_{3/4} and *r*_{1}, which suffice to numerically integrate the evolutionary dynamics of worker sterility using the system of ODEs described by Olejarz *et al.* [19]. We restrict our attention here to the invasion of an allele encoding full sterility in its carriers, under either recessive or dominant genetics.

The first procedure, ‘random noise’, is equivalent to Procedure 1 in Olejarz *et al.* [19]. Here, we set *r*_{1/4}=*r*_{0}+*a*, *r*_{1/2}=*r*_{0}+*b*, *r*_{3/4}=*r*_{0}+*c* and *r*_{1}=*r*_{0}+*d*. Note that the four values are completely uncorrelated with each other; sequential values of *r*_{z} are independent from previous values, which is why we have named this procedure ‘random noise’. This procedure might generate plausible *r*_{z} functions for a population where every colony-level increase in worker sterility were to completely erase the effect of any previous increase in worker sterility, replacing it with a new, random effect. That is, it is not particularly plausible.

The second procedure, ‘plateau’, is equivalent to Procedure 2 in Olejarz *et al.* [19]. Here, the values *r*_{1/4}, *r*_{1/2}, *r*_{3/4} and *r*_{1} are drawn from a correlated multivariate normal distribution. This can be simulated by transforming four uncorrelated normal variates; one way of doing this is by using the matrix
*ρ* is the desired correlation between each variate. By multiplying the vector of uncorrelated variates by the Cholesky decomposition of this matrix, one obtains four correlated variates

Now, we set *r*_{1/4}=*r*_{0}+*a*′, *r*_{1/2}=*r*_{0}+*b*′, *r*_{3/4}=*r*_{0}+*c*′ and *r*_{1}=*r*_{0}+*d*′. Note that, because the variables are correlated, the first ‘step’ (from *r*_{0} to *r*_{1/4}) tends to be larger in magnitude than subsequent ‘steps’ (i.e. from *r*_{1/4} to *r*_{1/2}, *r*_{1/2} to *r*_{3/4} or *r*_{3/4} to *r*_{1}), which is why we have named this procedure ‘plateau’. This procedure might generate plausible *r*_{z} functions for a population in which worker sterility brings diminishing returns to colony productivity, where these diminishing returns happen to set in near

Note that both the ‘random noise’ and ‘plateau’ procedures tend to produce *r*_{z} functions that disadvantage single mating relative to double mating. For the ‘random noise’ procedure, this is because although the procedure is just as likely to produce a peak at *p*_{1/2}≥*p*_{1/4}), yet, on average, they are receiving the same expected increase in productivity; hence, single mating is relatively disfavoured without a clear biological rationale. And as the ‘plateau’ procedure tends to produce colony efficiency functions with diminishing returns on worker sterility for colonies with *r*_{z} function with a relative peak at

The third procedure, ‘random steps’, sets each point in *r*_{z} to the value of the previous point plus a random perturbation: *r*_{1/4}=*r*_{0}+*a*, *r*_{1/2}=*r*_{1/4}+*b*, *r*_{3/4}=*r*_{1/2}+*c* and *r*_{1}=*r*_{3/4}+*d*. This procedure might generate plausible *r*_{z} functions if each increase in worker sterility had a random increasing or decreasing effect on colony productivity. The fourth procedure, ‘increasing steps’, is similar, except steps are constrained to be positive: *r*_{1/4}=*r*_{0}+|*a*|, *r*_{1/2}=*r*_{1/4}+|*b*|, *r*_{3/4}=*r*_{1/2}+|*c*| and *r*_{1}=*r*_{3/4}+|*d*|. This procedure might generate plausible *r*_{z} functions if each increase in worker sterility added a random increase to colony productivity. The fifth procedure, ‘linear’, uses a single normal variate to establish a constant step size for *r*_{z}: *r*_{1/4}=*r*_{0}+*a*, *r*_{1/2}=*r*_{1/4}+*a*, *r*_{3/4}=*r*_{1/2}+*a* and *r*_{1}=*r*_{3/4}+*a*. This procedure might generate plausible *r*_{z} functions if each increase in worker sterility had a consistent increasing or decreasing effect on colony productivity. For each of these new procedures, later points in *r*_{z} depend on earlier points, but there is no tendency for ‘steps’ between points in *r*_{z} to change in average magnitude, which arguably makes them less biased in favour of particular mating-number regimes than the old procedures.

In figure 3, we test each of these five procedures to see whether single or double mating is more favourable to the invasion (figure 3*b*) or equilibrium level of sterility (figure 3*c*), for recessive versus dominant sterility. The form of *p*_{z} we use (*p*_{z}=*k*+(1−*k*)*z*, with *k*=0.2), chosen for comparison with the numerical experiments of Olejarz *et al.* [19], their table 1, prevents worker sterility from resulting in a net loss of nephews (see Spiteful worker sterility and policing, above). Beneath the bar charts in figure 3*b*, we show the percentage of experiments for which the exclusive invasion of sterility under either single or double mating occurred with an initially decelerating *r*_{z} (i.e. where *r*_{1/2}−*r*_{1/4}<*r*_{1/4}−*r*_{0}). Note that, for these values of *p*_{z}, double mating only promotes the invasion of sterility relative to single mating when *r*_{z} is initially decelerating. In figure 3*c*, error bars show bootstrapped 95% confidence intervals for average worker sterility.

For the analysis presented in table 1, *r*_{z} functions are constructed using *σ*=0.25, and intermediate values (i.e. any *r*_{z} for *p*_{z} functions are constructed using random variates as follows: *p*_{0} is drawn from a uniform distribution between 0 and 1; *p*_{1/2} is drawn from a uniform distribution between *p*_{0} and (*p*_{0}+1)/2 (for non-spiteful worker sterility) or between (*p*_{0}+1)/2 and 1 (for worker spite); *p*_{1}=1; and all other values are linearly piecewise-interpolated between these three points.

### 4.4. Evolutionarily stable strategy analysis

By setting the left-hand side of condition (2.2) to zero, it is possible to find a convergence-stable point [12] for worker sterility. At these points, natural selection will not favour the invasion of an allele encoding either a small increase or a small decrease to worker sterility (i.e. convergence-stable points are stable to small perturbations); moreover, for a population playing a strategy that is close to a convergence-stable point, natural selection will favour the invasion of strategies between the population strategy and the convergence-stable point (i.e. convergence-stable states are reachable from nearby states). However, a convergence-stable point is only an ESS if *no* alternative allele can invade at this point. Therefore, in order to find a true ESS, we treat convergence-stable points as ‘candidate ESSs’, then use conditions (4.1) and (4.2) to determine whether any alternative allele can invade a population monomorphic for the candidate ESS under the appropriate regime of dominance or recessivity. If no alternative allele can invade, the candidate ESS is a true ESS. In figure 4, true ESSs are shown.

Note that it is possible for an ESS to *not*be convergence-stable, and this method will not identify such states. However, we are only interested in ESSs that are reachable, i.e. both convergence-stable and evolutionarily stable. Such strategies are called ‘continuously stable strategies’ (CSSs; [44]).

### 4.5. Demographically explicit ecological scenarios

In appendix B, we develop a general kin-selection model for the evolution of worker sterility. This analysis can be used to investigate a variety of ecological scenarios. Here, we present four such scenarios for the evolution of worker sterility.

#### 4.5.1. Scenario A. Workers’ sons replace queen’s sons

In this scenario, we assume that non-sterile workers replace the queen’s sons with their own sons, as in the model of Olejarz *et al.* [19]. Following these assumptions, we find that natural selection will favour an increase to worker sterility, *z*, when
*R*_{neph}=(2+*n*)/8*n*, *R*_{sis}=(1+*p*_{z})((2+*n*)/8*n*) and *p*_{z})/(1−*z*) of all worker-laid males. The ‘efficiency effect’ captures her impact on colony efficiency, which increases by a relative amount *r*′_{z}/*r*_{z}, augmenting the production of her sisters and of colony-produced males, a proportion *p*_{z} of whom are her brothers and a proportion 1−*p*_{z} of whom are her nephews. And the ‘male production effect’ captures her impact on the proportion of male eggs produced by the queen versus workers: her relative gain of brothers is *p*′_{z}, while her relative gain or loss of nephews exactly balances her forfeited sons and her gained brothers.

Similarly, natural selection favours an increase to the queen’s sex allocation, *x* (her proportion of resources allocated to daughters), when

#### 4.5.2. Scenario B. Workers’ sons compete with all queen’s offspring

It is also possible to assume that, rather than only displacing the queen’s sons, workers’ sons compete with the queen’s sons and daughters equally. This scenario may apply if workers do not discern between fertilized and unfertilized eggs when they replace the queen’s eggs with their own; alternatively, it may apply if rather than replacing the queen’s eggs, the workers simply lay their eggs in the communal nest, and all queen-produced and worker-produced offspring have the same expected survival. Following these assumptions, we find that natural selection will favour an increase to worker sterility, *z*, when
*p*_{z} is the proportion of all offspring on the patch that are produced by the queen, *R*_{neph}=(2+*n*)/8*n*, *R*_{sis}=((1+(1−2*x*)*p*_{z})/*xp*_{z})((2+*n*)/8*n*) and *x*)*p*_{z})/*xp*_{z} (the product of the relative reproductive value of all females compared to that of all males, (1+(1−2*x*)*p*_{z})/(1−*xp*_{z}), and the number of females relative to the number of males, (1−*xp*_{z})/*xp*_{z}), which comes into the expression for *R*_{sis}. Similarly to condition (4.3), the left-hand side of condition (4.5) can be interpreted as the inclusive-fitness effect experienced by a worker who stops laying male eggs. Here, the ‘sacrifice effect’ captures the direct cost of her sterility, in that she forfeits her relative share (1−*p*_{z})/(1−*z*) of all worker-laid males. The ‘efficiency effect’ captures her impact on colony efficiency, which increases by a relative amount *r*′_{z}/*r*_{z}, a proportion *xp*_{z} of which goes towards sisters, (1−*x*)*p*_{z} towards brothers, and 1−*p*_{z} towards nephews. And the ‘offspring production effect’ captures her impact on the proportion of eggs produced by the queen versus workers: her relative gain of sisters is *xp*′_{z}, and her relative gain of brothers is (1−*x*)*p*′_{z}, and hence her relative gain of nephews exactly balances her lost sons, less her gained brothers and sisters.

In this scenario, queen sex allocation is not independent of worker sterility. We find that natural selection favours an increase to the queen’s investment in daughters, *x*, when
*p*_{z}=1), the queen favours an even sex ratio (*x**=(1+*p*_{z})/(1+3*p*_{z}), resulting in a population sex ratio of *X**=*p*_{z}(1+*p*_{z})/(1+3*p*_{z}), which is male-biased for all *p*_{z}<1.

#### 4.5.3. Scenario C. Worker sterility among claustral inbreeders

Here, we assume that the queen produces a first brood of female and male soldiers, who mate among themselves; the second brood of female and male dispersers is partly produced by the queen and partly produced by the soldiers, as in the gall-forming social thrips. For simplicity, we assume here that queens and soldiers produce an even sex ratio for the second brood, but allowing sex ratio evolution does not change the results qualitatively (not shown). Following these assumptions, we find that natural selection favours an increase to the sterility of female soldiers, *z*, when
*R*_{dau}=(5+*p*_{z})/6, *R*_{son}=(3+*p*_{z})/6, *R*_{niece}=(3+6*n*+*p*_{z})/12*n*, *R*_{neph}=(3+2*n*+*p*_{z})/12*n*, *R*_{sis}=(3+2*n*+*p*_{z})/6*n* and *R*_{dau}=*R*_{son}=(11+*p*_{z})/16 and *R*_{niece}=*R*_{neph}=*R*_{sis}=*R*_{bro}=(1+*n*)/4*n* (figure 7*a*). Similarly to condition (4.5), the left-hand side of condition (4.7) can be interpreted as the inclusive-fitness effect experienced by a worker who stops laying male eggs; but in condition (4.7), the female worker’s ‘sacrifice effect’ involves giving up both daughters and sons; the ‘efficiency effect’ involves an increase in both niece and nephew production as well as sister and brother production; and the ‘offspring production effect’ involves the focal worker gaining both sisters and brothers, while her gain or loss of nieces and nephews balances her forfeited offspring and her gained siblings.

#### 4.5.4. Scenario D. The evolution of eusociality

Here, we assume that the queen produces and provisions a first brood of females, and then produces a second batch of female and male eggs. Each first-brood female can either disperse—leave the nest, mate, and produce female and male offspring on her own—or work—stay in the nest and help to raise the queen’s second-brood offspring without producing any offspring of her own. We assume that each worker can raise *b* siblings, on average, in her natal nest, and that each disperser can raise *b*(1−*c*) offspring, on average, in her newly founded nest, where *c* represents the cost of dispersal; and, additionally, that workers may synergistically or antagonistically interact according to the parameter *s*, such that if the total number of female workers is *Kz*, then in total workers can raise *Kzb*(1+*sz*) of the queen’s second-brood offspring. This model is conceptually similar to the one considered by Boomsma [8–10] for the evolution of eusociality. Following these assumptions, we find that natural selection will favour an increase to worker sterility, *z*, when
*R*_{sis}=(2+*n*)/4*n* and *R*_{sis}=*R*_{bro}=(1+*n*)/4*n* (figure 7*b*). When *z*=0, this condition reduces to
*n*=1), any marginal benefit of rearing siblings rather than offspring (for example, any non-zero cost of dispersal, mating or nest founding) suffices to favour the invasion of sterile workers, regardless of the level of worker synergy, *s*; but with any level of multiple mating (*n*>1), a threshold dispersal cost of at least (*n*−1)/2*n* is required for natural selection to favour the invasion of sterile workers (figures 6*d* and 7*b*). In other words, only marginal efficiency gains are needed for worker sterility to invade under strict monogamy [8–10].

#### 4.5.5 Explicit forms for *r*_{z} and *p*_{z}

Scenarios A, B and C above are independent of the particular *r*_{z} and *p*_{z} functions used. However, for preparing figures 6–8, we used the explicit forms

The *r*_{z} function above has three components: a baseline efficiency of 1; *bz*, representing a linear fitness benefit for each sterile worker; and *sz*^{2}, representing an ‘interaction effect’ of worker sterility. We use the parameter *s* to examine scenarios where multiple sterile workers result in either accelerating (*s*>0) or diminishing returns (*s*<0) to colony productivity.

The *p*_{z} function given above corresponds to a model in which the queen and *k*(1−*z*) reproductive workers each take an equal share of offspring production. Alternatively, *k* can capture not only the total number of workers but also their ability to control offspring production relative to the queen; for example, halving *k* could represent either a halving in the number of workers or a halving of their relative ability to control offspring production, keeping the number of workers constant.

A function of this form can also model more complicated demographic processes: for example, if we assume that there are *N* workers, each of whom replaces a random egg with their own at rate *W*, while the queen can replace a worker’s egg with her own at rate *Q*, then the form above gives the proportion of eggs produced by the queen at equilibrium when *k*=*NW*/*Q*. In models where worker-laid and queen-laid individuals compete equally, regardless of their sex, production of eggs and replacement of eggs will often be equivalent processes: that is, the form given above for *p*_{z} also holds if workers, rather than replacing the queen’s eggs, simply lay their own eggs in the communal nest without replacement. In that case, the *r*_{z} function would capture both the overall production and survival of eggs.

#### 4.5.6. Stable level of sterility

For figures 6 and 7, we determine the convergence-stable point [12] for sterility by numerically integrating the selection gradients for sterility and sex allocation (left-hand sides of conditions (4.3)–(4.8)). First, we set the sex ratio to

### 4.6. Stochastic individual-based model

To verify the results of our kin-selection analysis (figures 4, 6 and 7), we implemented a stochastic individual-based model (figures 5 and 8) in C++. Here, each individual comprises a locus encoding their breeding value for worker sterility, *Z*. The locus comprises one or two genes, depending on whether the individual is haploid or diploid, and each gene is represented by a real number *γ*∈[0,1]. Breeding values are determined by averaging genic values: hence, a haploid individual with genotype *γ* has breeding value *Z*=*γ*, while a diploid individual with genotype *γ*_{1},*γ*_{2} has breeding value *Z*=(*γ*_{1}+*γ*_{2})/2.

At the beginning of each generation, *M* mated females each produce *K* female workers on their home patch. Each worker has a probability *Z* of being sterile. The patch average sterility *z* determines the colony productivity *r*_{z} and the proportion of males produced by the queen *p*_{z}. The next generation of breeders is then produced: first, a patch is randomly selected from the population with probability proportional to its colony efficiency, *r*_{z}, and a female is produced by the queen on that patch; then, another *n* patches are randomly selected with replacement, with probability proportional to their colony efficiency, and each of these *n* patches produces a male (from the queen with probability *p*_{z}, or from a random reproductive worker on that patch with probability 1−*p*_{z}); the female mates with these *n* males, and this process is performed *M* times, at which point all the *M* mated females replace the foundresses of existing patches. All other individuals on each patch die, returning the population to the beginning of the life cycle.

Simulations start with a monomorphic population in which all *γ*=0, and hence *Z*=0 for each individual. A gene in a newly produced individual has a 1% probability of mutating, in which case its genic value changes from *γ* to *δ* is drawn from a normal distribution with mean 0 and standard deviation 0.01. We validated this stochastic individual-based model by using it to verify the analytical conditions of Olejarz *et al.* [19], not shown.

For figure 8, we make the following assumptions. The mating number *n* is either fixed at 1 (species 1–10) or drawn randomly from 1 to 5 (species 11–60). Each species’ *p*_{z} function uses the form *p*_{z}=1/(1+*k*(1−*z*)) (see Explicit forms for *r*_{z} and *p*_{z}, above), where *k* for each species is drawn randomly from 1 to 5. Finally, each species’ *r*_{z} function is of the form *r*_{z}=1+*bz*+*sz*^{2} (see Explicit forms for *r*_{z} and *p*_{z}, above), with *b* and *s* chosen such that *r*_{1} follows a uniform distribution between 1.5 and 2.5 and such that the slope *r*′_{0} is between 50% and 150% of the slope of the line between (*z*=0,*r*_{0}) and (*z*=1,*r*_{1}). In this way, the colony productivity function is equally likely to be concave or convex.

## Data accessibility

Data and code are deposited at Dryad: https://doi.org/10.5061/dryad.gt8b5 [46].

## Competing interests

The authors declare no competing interests.

## Authors' contributions

N.G.D. and A.G. designed the study and wrote the manuscript.

## Funding

A.G. is supported by the Natural Environment Research Council (A.G., NE/K009524/1). Funders were not involved in study design, interpretation or the decision to submit the work for publication.

## Acknowledgements

We thank Koos Boomsma, Peter Nonacs, Kevin Foster, James Marshall, Sam Levin, Carl Veller and David Queller for helpful comments.

## Appendix A: Explicit population-genetics analysis

Here, we analyse the invasion of a sterility allele into a wild-type population. The population is initially monomorphic for an allele *A* encoding sterility with penetrance 0≤*u*≤1, and a rare mutant allele *a* is introduced which encodes sterility with penetrance 0≤*v*≤1. Throughout, we closely follow the approach of Olejarz *et al.* [19], whose analysis is equivalent to ours with the assumptions that *u* and *v* are restricted to either 0 or 1.

We denote colony types by the genotype of the queen and the genotypes of her mating partners. Hence, *X*_{AA,m} is the frequency of colonies with an *AA* queen, *m* mutant (*a*) males and *n*−*m* wild-type (*A*) males; similarly for *X*_{Aa,m} and *X*_{aa,m}. At any given time step, we also keep track of the number of reproductive females of each genotype—*x*_{AA}, *x*_{Aa} and *x*_{aa}—and the number of reproductive males of each genotype—*y*_{A} and *y*_{a}. Matings between reproductives lead to the establishment of new colonies; hence, the evolutionary dynamics of colony types are captured by
*AA*,*m* colonies is proportional to the frequency of reproductive *AA* females, multiplied by their probability of mating with exactly *n*−*m* wild-type males and *m* mutant males; similarly for *Aa*,*m* and *aa*,*m* colonies.

The death rate of existing colonies, *ϕ*, is defined as

**A.1. Reproductive offspring if the mutant allele is dominant**

When the mutant allele is dominant, the production of each type of reproductive female (*x*_{AA}, *x*_{Aa}, *x*_{aa}) and male (*y*_{A}, *y*_{a}) is

These equations can be understood as follows. First, note that in an *AA*,*m* colony, a fraction *z*=((*n*−*m*)/*n*)*u*+(*m*/*n*)*v*=((*n*−*m*)*u*+*mv*)/*n* of workers will be sterile (*AA* workers with probability *u*, and *Aa* workers with probability *v*); in an *Aa*,*m* colony, a fraction *AA* workers with probability *u*, and *Aa* and *aa* workers with probability *v*); and in an *aa*,*m* colony, a fraction *z*=((*n*−*m*)/*n*)*v*+(*m*/*n*)*v*=*v* of workers will be sterile (*Aa* and *aa* workers with probability *v*). That is why these values of *z* as subscripts to the *r*_{z} and *p*_{z} functions are always associated, above, with their associated colony frequencies, *X*_{AA,m}, *X*_{Aa,m} and *X*_{aa,m}, respectively.

For female reproductives, each separate term within the curly braces above combines three elements; we will take the first term in curly braces in the *x*_{AA} line,
*X*_{AA,m}); the productivity of that colony type, as a function of the fraction of sterile workers within colonies of that type (i.e. *r*_{(nu+m(v−u))/n}); and the fraction of females and/or males produced by that colony type with the corresponding genotype (i.e. a fraction (*n*−*m*)/*n* of females produced in *AA*,*m* colonies have genotype *AA*, which is why they add to the quantity *x*_{AA}). Each term within equation (A 12) can be broken down in this way.

Accordingly, the production of female reproductives can be understood as follows: *AA*,*m* colonies produce (*n*−*m*)/*n* *AA* females and *m*/*n* *Aa* females; *Aa*,*m* colonies produce (*n*−*m*)/2*n* *AA* females, *Aa* females and *m*/2*n* *aa* females; and *aa*,*m* colonies produce (*n*−*m*)/*n* *Aa* females and *m*/*n* *aa* females.

Male production is more complicated, because both queens and workers produce males, but the principle is the same. We will take the first term in curly braces in the *y*_{A} line,
*AA*,*m* colonies (i.e. *r*_{((n−m)u+mv)/n}*X*_{AA,m}) goes towards the production of both the queen’s sons and workers’ sons. In particular, the queen is *AA*, so all her sons have genotype *A*, and the queen produces a fraction *p*_{((n−m)u+mv)/n} of males in the colony. Simultaneously, the workers—whose sons comprise a fraction 1−*p*_{((n−m)u+mv)/n} of colony male production—are (*n*−*m*)/*n* *AA* and *m*/*n* *Aa*; in the former group, workers are reproductive with probability 1−*u*, while in the latter group, workers are reproductive with probability 1−*v*; and all the sons of the first group will be *A*, while only half of the sons of the second group will be *A*. Hence, overall, a fraction (((*n*−*m*)/*n*)(1−*u*)+(1/2)(*m*/*n*)(1−*v*)((*n*−*m*)/*n*)(1−*u*)+(*m*/*n*)(1−*v*))(1−*p*_{((n−m)u+mv)/n}) of males produced in *AA*,*m* colonies are *A* males produced by workers. Note that the expressions for *y*_{A} and *y*_{a} can be further simplified, but we have left them in the form above to maximize clarity.

Accordingly, the production of male reproductives can be understood as follows. In *AA*,*m* colonies, the queen’s sons are all *A*; all of the sons of *AA* workers and half of the sons of *Aa* workers are *A*, while the other half of the sons of *Aa* workers are *a*. In *Aa*,*m* colonies, the queen’s sons are half *A* and half *a*; all of the sons of *AA* workers and half of the sons of *Aa* workers are *A*, while the other half of the sons of *Aa* workers and all of the sons of *aa* workers are *a*. Finally, in *aa*,*m* colonies, the queen’s sons are all *a*; half of the sons of *Aa* workers are *A*, while the other half of the sons of *Aa* workers and all the sons of *aa* workers are *a*.

**A.2. Reproductive offspring if the mutant allele is recessive**

Along similar principles, when the mutant allele is recessive, the production of each type of reproductive female and male is

These equations can be understood similarly to equation (A 12); in fact, they are identical, except for two general changes. First, the subscripts to *r*_{z} and *p*_{z} are different, because the mutant allele is recessive instead of dominant, which results in different proportions of sterile workers in colonies of each type: in an *AA*,*m* colony, a fraction *z*=((*n*−*m*)/*n*)*u*+(*m*/*n*)*u*=*u* of workers will be sterile; in an *Aa*,*m* colony, a fraction *aa*,*m* colony, a fraction *z*=((*n*−*m*)/*n*)*u*+(*m*/*n*)*v*=((*n*−*m*)*u*+*mv*)/*n* of workers will be sterile. Second, because of these differing proportions of sterile workers, the production of sons by workers is different, so the coefficients of 1−*p*_{z} in the fourth and fifth lines are different.

**A.3. Condition for invasion of a dominant mutant sterility allele**

Continuing to follow the approach of Olejarz *et al.* [19]: for a dominant mutant sterility allele, whether the allele increases in frequency from rarity is governed by the behaviour of *AA*,0, *AA*,1 and *Aa*,0 colonies. Colony types with more copies of the mutant allele are rarer, and hence will have a negligible effect on invasion. Therefore, from equation (A 9), we need only consider

We start with a wild-type population (*X*_{AA,0}=1) and introduce a small perturbation of magnitude *ϵ*≪1. Considering the density constraint (equation (A 11)), and only keeping terms up to order *ϵ*, this gives

Substituting (A 15) into (A 12), and keeping terms only up to order *ϵ*, gives

Finally, substituting (A 10), (A 16) and (A 17) into (A 14) and discarding powers of *ϵ*^{2} or higher gives

If the dominant eigenvalue of the above matrix is greater than zero, then a dominant sterility allele with penetrance *v* can invade a population monomorphic for sterility with penetrance *u*. This condition, after simplification, is

**A.4. Condition for invasion of a recessive mutant sterility allele**

For a recessive mutant sterility allele, whether the allele increases in frequency from rarity is governed by the behaviour of *AA*,0, *AA*,1, *Aa*,0, *AA*,2, *Aa*,1 and *aa*,0 colonies. Colony types with more copies of the mutant allele are rarer, and hence will have a negligible effect on invasion. Therefore, from equation (A 9), we need only consider

We start with a wild-type population (*X*_{AA,0}=1) and introduce a small perturbation of magnitude *ϵ*≪1. Considering the density constraint (equation (A 11)), and only keeping terms up to order *ϵ*^{2} (because terms of order *ϵ* alone are not sufficient to determine whether the recessive allele invades), this gives

Substituting equation (A 20) into equation (A 13), and keeping terms only up to order *ϵ*^{2}, gives

Substituting equations (A 10), (A 21) and (A 22) into equation (A 19) and discarding powers of *ϵ*^{2} or higher gives, in matrix form,

The dominant eigenvalue is 0, and its corresponding eigenvector is *X*_{AA,0} over the first-order perturbations to *X*_{AA,1} and *X*_{Aa,0}.)

Substituting equations (A 10), (A 21), (A 22) and (A 23) into equation (A 19), and keeping terms up to order *ϵ*^{2}, gives

The equation for

The same can be done for

Equation (A 26) can be used to solve for

The equations for

We solve for *t* to become relatively large, such that all the time-dependent terms in equations (A 25)–(A 28) approach zero. Accordingly, the sign of *v* will invade a population monomorphic for sterility with penetrance *u* if

## Appendix B: Kin-selection analysis

Here, we develop a general model of the evolution of wholly or partly non-reproductive workers using standard kin-selection methodology [38,47]. In this model, a mated queen founds a colony by producing an initial brood of females and/or males. Depending on the model scenario, first-brood females may either mate with first-brood males—from their own or from a different colony—or remain unmated. Then, according to the level of worker sterility *z*, a focal first-brood female (i.e. a worker) invests a proportion of her resources into helping to raise the colony’s next brood—which consists partly of queen-produced offspring (queen-laid females, notated f, and queen-laid males, notated m) and partly of worker-produced offspring (worker-laid females, notated *φ*, and worker-laid males, notated *μ*)—and a proportion of her resources into producing her own offspring. Individuals of the second brood disperse and mate, with each female mating with *n* males, and mated females then found new patches, restarting the cycle.

In this model, we denote a focal worker’s sterility by *Z*, the average sterility on a focal patch by *z* and the average sterility in the population by *x*, and the average sex ratio strategy among all queens in the population is denoted by *f*=*f*(*z*,*x*); the production of queen-laid second-brood males on a focal patch is *m*=*m*(*z*,*x*); the production of worker-laid females by a focal worker is *ϕ*=*ϕ*(*Z*,*z*,*x*); and the production of worker-laid males by a focal worker is *μ*=*μ*(*Z*,*z*,*x*). We denote by

For a gene increasing worker sterility to spread, its carriers, on average, should leave more descendants than other members of the population. Accordingly, natural selection will favour an increase in worker sterility, *z*, when

Above, *R*_{sis}, *R*_{bro}, *R*_{dau}, *R*_{niece}, *R*_{son} and *R*_{neph} are the (life-for-life) relatedness between a focal female worker and her sister, brother, daughter, niece, son and nephew, respectively, and all derivatives are evaluated at

Each term on the left-hand side of condition (B 1) captures how a small increase in worker sterility impacts upon the fitness of different individuals in the population, weighted by the life-for-life relatedness between those individuals and a focal worker, which combines both (i) the reproductive value of those individuals (i.e. their capacity for projecting genes into future generations) and (ii) the extent to which those individuals themselves carry the gene increasing worker sterility. Alternatively, each term can be read as an inclusive-fitness effect experienced by a focal worker who gives up reproduction to become sterile. These interpretations are mathematically equivalent, but we focus on the inclusive-fitness interpretation here, as it is conceptually simpler.

Similarly, natural selection will favour an increase in the queen’s sex allocation strategy (her investment in daughters), *x*, when

Above, *R*_{dau | Q} is the relatedness between a focal queen and her daughter, *R*_{son | Q} is the relatedness between a focal queen and her son, *R*_{gdau | Q} is the relatedness between a focal female and her granddaughter (her daughter’s daughter), *R*_{gson | Q} is the relatedness between a focal female and her grandson (her daughter’s son), and all derivatives are evaluated at

For scenario A, the production of queen-laid females, queen-laid males, worker-laid females and worker-laid males is *f*=*xr*_{z}, *m*=(1−*x*)*r*_{z}*p*_{z}, *ϕ*=0 and *μ*=(1−*x*)*r*_{z}(1−*p*_{z})((1−*Z*)/(1−*z*)), respectively. For scenario B, we use *f*=*xr*_{z}*p*_{z}, *m*=(1−*x*)*r*_{z}*p*_{z}, *ϕ*=0 and *μ*=*r*_{z}(1−*p*_{z})((1−*Z*)/(1−*z*)). For scenario C, we use *f*=*xr*_{z}*p*_{z}, *m*=(1−*x*)*r*_{z}*p*_{z}, *ϕ*=*yr*_{z}(1−*p*_{z})((1−*Z*)/(1−*z*)) and *μ*=(1−*y*)*r*_{z}(1−*p*_{z})((1−*Z*)/(1−*z*)). And for scenario D, we use *f*=*x*(*z*+*sz*^{2}), *m*=(1−*x*)(*z*+*sz*^{2}), *ϕ*=*y*(1−*Z*)(1−*c*) and *μ*=(1−*y*)(1−*Z*)(1−*c*). Substituting these definitions into conditions (B 1) and (B 2) recovers conditions (4.3)–(4.8) above.

**B.1. Relatedness calculations**

The life-for-life relatedness of individual A to individual B is *R*_{AB}=(*F*_{AB}/*F*_{AA})(*c*_{B}/*c*_{A}), where *F*_{AB} is the consanguinity of individual A and individual B, *F*_{AA} is the consanguinity of individual A to herself, *c*_{B} is the class reproductive value of individual B and *c*_{A} is the class reproductive value of individual A [48]. Note that as individual A is always the same individual within a given condition above, we can instead use *R*_{AB}=*F*_{AB}*c*_{B} or any multiple thereof without affecting the resulting conditions.

Accordingly, consanguinities needed for the conditions above can be found in table 2. The consanguinities for a female worker under claustral inbreeding are obtained by first calculating the coefficient of inbreeding for a foundress in this mating system (the probability that her two genes at a given locus are identical by descent). Suppose that an offspring is foundress-laid with probability *Q*, and soldier-laid with probability 1−*Q*. If foundress-laid, her coefficient of consanguinity is zero, because patch founders are unrelated. If worker-laid, then her paternally inherited gene comes from her grandmother, and her maternally inherited gene comes, with equal probability, either from her grandfather—who is unrelated to her grandmother—or from her grandmother; in the latter case, her two genes are either copies of the ‘same’ gene in her grandmother, in which case they are identical by descent with probability 1, or are copies of ‘different’ genes from her grandmother, in which case they are identical by descent with probability *G*, where *G* is the offspring’s grandmother’s coefficient of inbreeding. That is, overall, the probability that these two genes are identical by descent is *G*=*F*, which gives *F*=(1−*P*)/(3+*P*). A similar argument gives the same result under diploidy.

**B.2. Class reproductive values**

To determine the class reproductive value of each of the four dispersing offspring classes (queen-laid females, class f; queen-laid males, class m; worker-laid females, class *φ*; and worker-laid males, class *μ*), we first solve for the total reproductive value of all dispersing females, *c*_{F}=*c*_{f}+*c*_{φ}, and the total reproductive value of all males, *c*_{M}=*c*_{m}+*c*_{μ}. Defining *c*_{F}=(*Q*/2+(3(1−*Q*))/4)*c*_{F}+(*P*+(1−*P*)/2)*c*_{M}, with the constraint that *c*_{M}=1−*c*_{F}, can be solved to give *c*_{F}=2(1+*P*)/(3+2*P*+*Q*) and *c*_{M}=(1+*Q*)/(3+2*P*+*Q*). As an individual’s mating success is not affected by whether they are queen- or worker-laid, we have *c*_{f}=*Qc*_{F}, *c*_{φ}=(1−*Q*)*c*_{F}, *c*_{m}=*Pc*_{M} and *c*_{μ}=(1−*P*)*c*_{M}, which, overall, gives

When all second-brood offspring are queen-laid (*P*=*Q*=1), this yields the expected result that *c*_{φ}=0 and *c*_{μ}=0; when all second-brood offspring are worker-laid (*P*=*Q*=0), this yields the expected result that *c*_{f}=0, *c*_{m}=0,

It is illustrative to examine a special case. When all second-brood females are queen-laid (*Q*=1), this reduces to
*P*=1, we have the expected result that the total value of dispersing females is *P*=0, the total value of dispersing females is

- Received December 13, 2017.
- Accepted April 3, 2018.

- © 2018 The Authors.

Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited.