Nonhuman Minds: Animal, Artificial or Other Minds
Cognitio 2011
Young researchers conference in cognitive science
Montréal, July 3rd, 4th and 5th 2011
Decision making in frequency-dependent situation: A computational model for Producer-Scrounger based on reinforcement learning
Mohammad Afshar and Luc-Alain Giraldeau
Abstract:
As the main approach in biology, and behavioral ecology in particular, the behavior of animals, individually or in groups, is more often determined by evolution than other mechanisms. However, there have been debates about the relation between learning and evolution and their impacts on animals’ behavior and decision making, especially in higher level and more advanced animals. In some social condition the rationality of an action depends on the actions of other individuals in the society. For example in social foraging, an animal may decides to search for a new source of food (produce) or join what the others already found (scrounge).
Empirical observations show the portion of individuals interested in each strategy is affected by different personal and environmental parameters. Although the cognitive mechanism for decision making in producer-scrounger (PS) game is not clearly known, a number of models have been proposed based on evolution to predict the equilibrium combination of producer and scrounger strategists on an evolutionary timescale. However, each model can handle only few environmental parameters at a time and none are specifically designed to address decision on an ecological timescale.
In this study we propose the first general producer scrounger model based on reinforcement learning (RL model) that can predict effects of a wide range of parameters on the expected equilibrium of producers and scroungers attained over ecological time. In this model each Individual tries to maximize its food income by choosing the better strategy based on its previous experience. The model, which also includes a mechanism for sampling and exploitation, shows how strategy use reaches equilibrium in a static environment. We explore the effects of a number of parameters including group size, finder's advantage, living requirements, and patch encounter probability. The RL model, which replicates empirical observations better than existing evolution-based models, shows that in some animals maybe the decision making in social situation is affected more by learning rather than evolution. So, the role of evolution is limited to the architecture of learning mechanism and setting its parameter. The RL model also provides a set of new predictions that still await testing.
Empirical observations show the portion of individuals interested in each strategy is affected by different personal and environmental parameters. Although the cognitive mechanism for decision making in producer-scrounger (PS) game is not clearly known, a number of models have been proposed based on evolution to predict the equilibrium combination of producer and scrounger strategists on an evolutionary timescale. However, each model can handle only few environmental parameters at a time and none are specifically designed to address decision on an ecological timescale.
In this study we propose the first general producer scrounger model based on reinforcement learning (RL model) that can predict effects of a wide range of parameters on the expected equilibrium of producers and scroungers attained over ecological time. In this model each Individual tries to maximize its food income by choosing the better strategy based on its previous experience. The model, which also includes a mechanism for sampling and exploitation, shows how strategy use reaches equilibrium in a static environment. We explore the effects of a number of parameters including group size, finder's advantage, living requirements, and patch encounter probability. The RL model, which replicates empirical observations better than existing evolution-based models, shows that in some animals maybe the decision making in social situation is affected more by learning rather than evolution. So, the role of evolution is limited to the architecture of learning mechanism and setting its parameter. The RL model also provides a set of new predictions that still await testing.