1.1. What Is Agent-Based Modeling

Agent-based computational modeling (ABM) is an approach to modeling complex social dynamics that has developed in recent decades, facilitated by increased computational power. In an ABM, actors in a system are represented as autonomous individuals in a computer program. They are given rules that govern their behavior, including adaptation and interaction with each other and with their environment through time, and a starting configuration. The ABM then simulates¹ both individual trajectories and population-level patterns or outcomes, which are generated from the bottom up by the decentralized actions and interactions of the agents. Such a model provides mechanistic mapping from individual-level assumptions to coevolving population-level dynamics. Assumptions can be informed by data or theory, and outcomes at both the individual and population levels can be compared with data statistically. ABM allows enormous flexibility in assumptions, and agents can be modeled at any level (or multiple levels) of scale.

1.2. Why Agent-Based Modeling

Like other modeling methods, this technique has both advantages and important limitations. The particular advantages of ABM come from its flexibility, which can help model designers and users to manage three particular challenges that complexity poses for researchers and policy makers alike: heterogeneity, spatial structure, and adaptation.

1.3. A Brief Overview of Agent-Based Modeling in Various Fields

Application of ABM first proliferated in biology and in social science and initially focused on important uses of the technique for theory and hypothesis development. Early examples in social science included work on cooperation (Axelrod, 1997b), electoral and bureaucratic dynamics (Bendor et al., 2003; Bendor and Moe, 1985; Kollman et al., 1992, 1997; Laver, 2005), conflict (Bhavnani and Miodownik, 2009; Epstein, 2002), and segregation (Bruch and Mare, 2006; Schelling, 1971; Xie and Zhou, 2012). Early work using ABM in evolutionary biology (Axelrod et al., 2004; Hammond and Axelrod, 2006a; Holland, 1992; Nowak, 2006; Ohtsuki et al., 2006) and ecology (DeAngelis and Mooij, 2005; Heckbert et al., 2010) also contributed in important ways to theory development. Many of these efforts leveraged the ability of ABM to capture heterogeneity, spatial structure, and adaptation to generate important new insights. With growing computing power, application of ABM expanded to such fields as education (Maroulis et al., 2014), anthropology (Axtell et al., 2002), economics and finance (Dawid and Fagiolo, 2008; Dawid and Neugart, 2011; Farmer, 2000; Farmer and Foley, 2009; LeBaron and Winkler, 2008; Tesfatsion and Judd, 2006), marketing (North et al., 2010; Rand and Rust, 2011), and land use (Berger et al., 2007; Berger and Troost, 2014; Brady et al., 2012; Brown et al., 2005a; Brown et al., 2005b; Guzy et al., 2008; Happe et al., 2008; Happe et al., 2006; Heckbert, 2011; Magliocca et al., 2014; Sun et al., 2014). ABM also began to be applied in a broader set of ways, including models engaged with large data sets (Axtell et al., 2002; Bruch and Mare, 2006; Farmer and Foley, 2009) and models designed to engage with or inform policy and to address policy resistance (Berger et al., 2007; Brown et al., 2005a; Brown et al., 2005b; Dawid and Fagiolo, 2008; Farmer, 2000; Guzy et al., 2008; Happe et al., 2008; Happe et al., 2006; Heckbert, 2011; LeBaron and Winkler, 2008; Magliocca et al., 2014; Schlüter and Pahl-Wostl, 2007; Sun et al., 2014).

A very recent, but rapidly growing, application area for ABM is in public health. Initial applications of ABM to public health focused on the epidemiology and control of communicable disease (Burke et al., 2006; Epstein, 2004; Epstein, 2009; Eubank et al., 2004; Ferguson et al., 2006; Germann et al., 2006; Lee et al., 2010; Longini et al., 2005; Longini Jr et al., 2007; Yang et al., 2009). A large network of modelers (MIDAS²) funded by the National Institutes of Health (NIH) has had substantial scientific and policy impact using ABM among other modeling approaches (see section 3.1 below). The last 5 years have seen growing recognition of the potential for ABM to yield new insights on a wide array of topics in public health, particularly in light of the importance of heterogeneity, spatial structure, and adaptation that have been informed by other fields (Brown et al., 2005a; Brown et al., 2005b; Magliocca et al., 2014; Sun et al., 2014), and the approach was highlighted in three recent Institute of Medicine reports (IOM, 2010, 2012, 2013). This broader awareness has led to the recent proliferation of work, including the creation of two additional NIH-funded modeling networks that use ABM: one focused on obesity (NCCOR Envision³) and one on health disparities (NICH⁴). Initial ABM studies in these areas include (COSSA, 2014; Hall et al., 2014; Hammond, 2009; Hammond and Ornstein, 2014; Hammond et al., 2012; Zhang et al., 2014).

2.1. Policy as a Specific Use for Agent-Based Modeling

Computational or mathematical models (including ABM) offer a number of potential advantages to a decision maker. By making explicit the assumptions, key pathways, and uncertainties involved (along with the mapping of all three of these onto potential outcomes), models can help decision-makers to revisit and discuss implicit mental models that may be driving the decision process. Explicit models are more easily tested, both for internal consistency and for external fidelity. Models can also be especially useful tools when fielding real-world experiments to inform policy choice is difficult, overly expensive, time-consuming, unethical, or impractical. An additional advantage offered by models such as ABM lies in their ability to uncover potentially unanticipated adaptive system responses that a policy or intervention might trigger (see section 1.2). ABM can also help a decision maker understand the implications heterogeneity (across individuals, contexts, or time) may have for the impact of a policy in the longer term or in contexts differing from those for which empirical evidence is available. Finally, models can sometimes be of particular use when processes of policy implementation or even policy making itself are the focus.

2.2. Three Specific “Modalities” for Informing Policy with Agent-Based Modeling

The use of ABM to inform policy or decision making comes with its own particular set of considerations. ABMs that inform policy fall into three distinct categories: prospective policy models, retrospective policy models, and indirect policy models.

Prospective policy models (also sometimes called ex ante models) help to inform the design of policies or interventions by elucidating their potential effects. Such models contain representations of key dynamic mechanisms in a system, along with explicit representations of one or more policy choices, and they allow comparison of policy options within the simulated system. This process can aid in the design of policies or interventions by:

• Identifying leverage points where small shifts induced by targeted policies can generate large shifts in systemic outcomes or dynamics (such as “tipping points”). This may help to identify previously unnoticed opportunities or strategies for intervention.
• Elucidating potential linkages (trade-offs or synergy) between multiple policies or intervention elements in a complex system. This may help to facilitate coordination across “silos” in government or society, as needed for “systems” interventions (Huang et al., 2011; Nader et al., 2012).
• Allowing experimentation “in silico” to understand full potential consequences (intended or unintended) of interventions, which may include counterintuitive or unexpected impacts. This is of particular use when “in vivo” or “in vitro” experimentation (for example, through a randomized clinical trial) is not practical.
• Anticipating a variety of possible future scenarios that may unfold, incorporating both uncertainty and policy choices, or helping to elucidate how an intervention design might “scale,” translate to a novel context, or play out in the long term.

Models (including ABM) are most effective as one input into a multifaceted decision-making process; they generally cannot eliminate uncertainty or the need for judgment in weighing difficult trade-offs. They can, however, be of substantial help to decision makers in managing both complexity and uncertainty.

Retrospective policy models help to understand the underlying reasons for (retrospectively observed) success or failure of a policy or intervention that is already in place. To do this, they leverage the ability of ABM to provide insight into complex and dynamic mechanisms that are at work in a system which may not be directly observable (see section 1.2 above). In some settings, data may not exist (or might even be impossible to collect) to disentangle multiple simultaneously occurring mechanisms. ABM can help with causal inference in such circumstances. In the context of an intervention evaluation, this type of model can help evaluators to understand why and how elements of the intervention may have succeeded or failed. By facilitating consideration of heterogeneity (see section 1.2), ABM can also help to understand differential success of a policy or intervention across sub-populations or contexts. This may be critical for consideration of scaling and translation of successful interventions. In practice, retrospective modeling may often be combined with subsequent prospective modeling that leverages lessons learned from existing data to design improved interventions.

Indirect influence on policy or decision making may also come from models that are not explicitly aimed at consideration of policy choices. ABM offers extensive capabilities for understanding etiology, bidirectional relationships between system structure and individual behavior over time, and the operation of pathways that cross levels of scale. This type of model generally does not contain any explicit representation of policies or interventions and thus does not directly simulate the potential impacts of policy choices. Nonetheless, discoveries derived from this type of model may have important implications for policy—including identification of key leverage points, mechanisms, or windows of opportunity for intervention. Application of such insights within a policy-making process must be done with care and may require further simulation modeling that explicitly contains representation of the policy choices under consideration.

3.1. Examples of “Prospective” Agent-Based Modeling to Inform Policy Design

3.2. Examples of “Retrospective” Agent-Based Modeling

Retrospective use of ABM to understand differential success of policies and interventions in public health has only recently begun to emerge,⁵ but examples of this use are more widespread in social science. One illustrative example comes from political science consideration of real-world electoral systems and their implications for party competition and bureaucratic politics (Laver, 2005; Laver and Sergenti, 2011). In this work, agent-based models of multiparty competition (in which political party leaders and voters are types of agents) are applied to understand the historical trajectories of party policies and vote shares in 10 European countries. Another example is the recent use of ABM to study the economics of systemic risk in the housing market (Geanakoplos et al., 2012). This work looks retrospectively at policies that were in place during the housing boom and bust of 1997–2009 and develops an ABM of the underlying mechanisms (individual-level incentives and behavioral adaptations to the policies) that produced the observed outcome.

3.3. Examples of “Indirect” Policy Implications from Agent-Based Modeling Focused on Mechanisms

Early applications of ABM in obesity research have focused on elucidating complex etiology. Obesity results from a multiscale system, with behaviors and outcomes driven by interacting mechanisms that sometimes cross levels of scale. ABM has the potential to offer new insights into mechanisms and to connect research focused on “below the skin” with that focused on “above the skin” (Hammond, 2009). Some of these models are beginning to offer insights that may have important indirect policy implications. For example, recent work focused on understanding preference formation builds on existing evidence in neuroscience on key brain systems that are involved in controlling eating behavior (Hall et al., 2014), making use of ABMs’ ability to cross the “skin barrier” by embedding dynamic reward-learning processes within agents while exposing agents to different external sequences of environmental food exposures. The resulting model (Hammond et al., 2012) illustrates how early food exposures can strongly shape food preferences in ways that have substantial inertia in the face of subsequent changes in food opportunities; the model also shows how preference formation can be path dependent in the sequence of experience exposures (see section 1.2). Although this model does not directly consider any specific real-world policy, it offers potential implications for both targeting and timing of interventions to prevent obesity by encouraging the formation of preferences for healthy food. Another group of recent ABM papers focuses on the role of social networks and social influence in obesity, elucidating potential dynamic mechanisms through which social influence occurs (and which may potentially be harnessed for interventions) (Bahr et al., 2009; Hammond and Ornstein, 2014; Zhang et al., 2014). Some of these models include empirically grounded biological processes (Hall, 2010) that interact with the social level (Hammond and Ornstein, 2014); others begin to explore modalities for interventions to harness social forces while stopping short of prospectively modeling any specific real-world policy in detail (Bahr et al., 2009; Zhang et al., 2014).

In social science, examples of ABM with important indirect implications for policy include canonical work on the drivers of segregation (Bruch and Mare, 2006; Xie and Zhou, 2012) and work on the underlying mechanisms that may explain the ubiquity of ethnocentrism (Hammond and Axelrod, 2006b). In both cases, the models do not directly simulate specific policy or intervention choices—but they elucidate powerful pathways (and sometimes specific levers in the form of key variables) that could be harnessed for policy or intervention purposes (see, for example, Axelrod, 2004, which discusses potential application of insights from the ethnocentrism model to security issues of central Asia).

4.1. Key Building Blocks of an Agent-Based Model

Although ABMs are quite diverse as a group, reflecting diversity in topic (see section 1.3) and goal (see section 4.2)—and understanding the number of design choices implicit in these building blocks helps to motivate some of the best practices discussed below (see section 4.3).

The elements of an ABM may be organized according to the “PARTE” framework: Properties, Actions, Rules, Time, and Environment. The first three elements (Properties, Actions, Rules) define the agents, while the next two (Time and Environment) define the context (see Figure A-1).

FIGURE A-1

PARTE framework.

Properties are characteristics of individual agents (such as sex, age, disease state, wealth, and body mass index). An agent property can be:

• Mutable or immutable over time (within the simulation). This is a design choice: a model focusing on a single school year might treat age as immutable, whereas allowing age to change within the simulation might be central to models considering the lifecourse or overlapping generations (Axtell et al., 2002).
• Observable, partially observable, or unobservable to other agents. A simulation of farm behavior might treat size in acres as observable to other farms, but income as only partially observable (Brown et al., 2005a; Brown et al., 2005b; Happe et al., 2006; Magliocca et al., 2014; Sun et al., 2014). In a model of tobacco use, agents may be able to observe numerous cues that suggest whether another agent is a current smoker or not, but previous smoking history is likely harder to observe.
• Stored in a variety of data structures, from simple booleans to complex lists or arrays. A dichotomous variable can be easily stored in a simple data structure. Storing the mapping between food types and associated reward values (Hammond et al., 2012) or agents’ preferences for other types of choices (Kollman et al., 1992, 1997; Maroulis et al., 2014) might require a more complex data structure, such as a hash table or vector.

Not all types of agents represented in a model need to have all properties; for example, the property “market capitalization” is relevant for agents that represent firms but not for agents that represent employees of the firms. All properties must have well-defined conditions for initialization and for change through time. Initialization can involve draws from predefined distributions or from data and may be conditional on values assigned to other properties. By representing each individual actor as a separate software object, ABM allows enormous flexibility to capture heterogeneity across agents in their properties (see section 1.2).

Actions define the repertoire of specific behaviors that agents can perform within the simulation, such as moving around the environment, eating food, smoking tobacco, communicating information to a neighbor, forming a friendship tie, or buying a product. Agent actions can:

• Change an agent’s own Properties. For example, taking the action “eat” may affect the property “body mass index” over time (Hammond and Ornstein, 2014); taking the action “buy cigarettes” will immediately affect the property “inventory of cigarettes” (Luke et al., 2014).
• Change the Properties of other agents. For example, models of cooperation and reciprocity often contain an action “donate (to agent x)” which increases a wealth or wellbeing property of agent x while decreasing the same property for the actor (Axelrod et al., 2004; Hammond and Axelrod, 2006a; Hammond and Axelrod, 2006b; Nowak, 2006).
• Change the Environment. In models of land use and commons, for example, exploitation of a resource by one agent may reduce the amount of resource available at that location in the environment (temporarily or permanently) to other agents.
• Change an agent’s own Rules, for example through learning.

For every action included for any agent in an ABM, the modeler must define conditions under which the action is triggered or may be performed. Each action must also have defined consequences (which may include one or more of the changes above); actions that have no consequences do not affect simulation dynamics and do not belong in the model.

Rules are the central drivers of model dynamics, defining how agents choose an action, update properties, and interact with each other and their environment. Rules in an ABM can:

• Take as an input the current or past value of Properties (an agent’s own, those of others, or those of the environment); for example, “if age > 18, purchase tobacco.”
• Be dependent in some way on Time, and may involve learning or adaptation—for example, image scoring in reputation models, or the process of preference formation (Hammond et al., 2012; Nowak, 2006).
• Vary enormously in complexity from simple heuristics (for example, “when reaching any four-way intersection, always turn left”) to detailed internal models or calculations (for example, agents who collect data about the simulated world and optimize over some objective function).
• Cause not only modification of agents but creation or removal of agents, for example, creation of offspring in a demographic model or killing of other agents in a model of genocide (Bhavnani and Miodownik, 2009; Epstein, 2002).
• Involve stochastic probability, for example, “when reaching any four-way intersection, turn left with probability 50 percent.”

Time is central to a dynamic simulation model. Agent-based models (and other related simulation models) generally have a single, lowest-level fundamental unit of time that represents one pass by the computer through the set of instructions that embody the simulation. This is sometimes referred to as an “iteration,” a “tick,” or a “round.” Time in an ABM:

• Can remain abstract or can be calibrated to real-world time with additional design work. Such calibration may be easier for some models than for others, depending in part on whether behaviors in the model occur on known time scales. A model in which an agent’s age in years changes every 12 iterations or in which agents travel back and forth between home and office every other iteration is relatively easy to calibrate to calendar time (Luke et al., 2014). A model of chronic disease incidence driven by smoking or a model of opinion-change dynamics may require more work to calibrate (Garcia and Jager, 2011).
• Can involve multiple distinct “speeds” at which change occurs, for example, the speed with which a virus spreads through social contact versus the speed with which the virus itself evolves.
• Is the unit in which rules, action, and changes in agent properties or environment are defined.
• May also shape the simulation results through decisions about the order in which instructions are followed by the computer. For example, some types of diffusion models start with a single agent that deviates from the population and calculate the likelihood that the deviation will spread through the population. The choice between two implementations of diffusion—one in which the early adopter influences others before being influenced by its own neighbors and one in which the two directions of influence are evaluated in the reverse order—can result in very different outcomes, such as whether the deviation persists or dies out. A deterministic agent activation order could mistakenly lock the model into one of those outcomes (see section 4.3).

Environment provides the context for agents and their interactions in the model. The flexibility to represent many different types of environment effectively is a major strength of ABM (see section 1.2). The Environment in an ABM can:

• Range from simple and relatively abstract geometries (e.g., a lattice, ring, or torus) to highly complex ones (often empirically informed) such as a GIS shape file or a network structure.
• Contain “agent types” itself, with their own properties, actions, and rules. For example, a model of subsistence agriculture may contain rules for crop regeneration at any particular environmental location that are dependent on farming and harvest practices as well as intrinsic soil and water conditions (see Axtell et al., 2002).
• Can change over time, endogenously (as a result of such agent actions as crop rotation) or exogenously (for example, as a result of a policy change or external shock).

The PARTE framework describes fundamental building blocks that are present in every ABM, but as the illustrative examples above show, enormous variation is possible in the form that each element (P, A, R, T, and E) takes from model to model. This flexibility is part of the power and potential of ABM, but also underlines the importance of following best practices in making the numerous design choices required (see section 4 below).

4.2. Key Steps in Agent-Based Modeling

Just as ABMs share key building blocks in common (while exhibiting extensive heterogeneity in instantiation), the process of constructing and using an ABM generally follows six key steps. These steps are not unique to ABM—they are shared with many other forms of computational modeling—but several steps raise particular considerations for ABM. Table A-1 briefly describes the key steps.

TABLE A-1

Key Steps in Model Development.

With ABM in particular, progress from step to step may not be linear, but instead may involve iterative cycles or back-and-forth—especially as models are built up from simple to complex in stages (see section 4.3 BP3 below).

4.3. Considerations and Best Practices for Model Developers

This section outlines best practices that have emerged to guide design and use of ABMs. Some are specific to policy-aimed modeling; others are general best practices for good ABM (or even for modeling in general) but may have special relevance or importance when the aim is to inform policy. The best practices are organized according to the six steps of modeling (see section 4.2), and discussed in the subsections below.

4.4. A Few Considerations for Model Consumers

The sections above have described the many elements (section 4.1), steps (section 4.2), and best practices (section 4.3) involved in constructing and using ABM. With these in mind, a few guidelines arise for decision-makers who wish to use modeling as an input into the decision process.

Early engagement with the modeling effort can be helpful in communicating the goal or question of interest to model designers and in ensuring that the fit between the desired use of the model and the method and design of the model is appropriate. Model consumers are not always involved in the design and implementation phases of modeling, but they can be. Engagement in design itself can take many forms but often involves helping to ensure “face validity” and relevance of key design choices. Model consumers should ask the right questions throughout the process to ensure that they understand the choices being made.

Later in the process, model consumers should also ask the right questions to ensure that they fully understand the results and their boundaries and possible interpretations, the sensitivity of results to assumptions, and the role of uncertainty. It can be especially helpful for decision makers to understand the intuition and pathways behind results that seem counterintuitive.

4.5. Common Misperceptions About Agent-Based Modeling

A few misperceptions about the use of ABM commonly arise, especially in its use for policy purposes, and are worth brief discussion and clarification. One common misperception is that ABMs are necessarily “ad hoc” or reliant on poorly grounded inputs and assumptions. ABMs certainly can suffer from this problem (as can models of all types!) but they needn’t—there is nothing inherent in the ABM method that prohibits well-grounded assumptions. As described above (see section 4.3), care is needed in model design to motivate and ground assumptions and to avoid growing models too rapidly in complexity and stretching the ability of the modeling team to defend assumptions and explore sensitivity. The flexibility and individual-level focus of ABM confer great power, but they also require careful attention to and responsibility for assumptions on the part of the modeler.

A second common misperception concerns reuse of models. As described above, good models are usually designed for quite specific purposes with clear questions and boundaries in mind, and modelers make many specific implementation choices that flow from these goals. One implication of this is that “repurposing” models to answer questions or address topics and contexts for which they were not designed must be done with great care. Models can be used effectively in this way, but it requires carefully revisiting assumptions and design choices to ensure that they remain appropriate for the new application.

A third common misconception is in regard to the skill set required for ABM. Although ABMs are computational models, their rigorous design and use require much more than the skill of computer programming or computer science. Navigating the many elements and challenges of design, implementation, and interpretation of ABM requires another skill, the skill of “modeling,” and benefits from extensive experience and familiarity with the best practices outlined above.