Dynamic decision policy reconfiguration under outcome uncertainty

Humans and other mammals flexibly select actions under noisy and unstable conditions. To shed light on this flexibility, we evaluated how the underlying decision policy evolves when humans change their minds about the most rewarding action. Participants performed a dynamic two-armed bandit task that manipulated the certainty in relative reward (conflict) and the reliability of action-outcomes (volatility). We found that conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height). Following a switch in the optimal choice, the drift rate reduces and the boundary height spikes, leading to a slow exploratory state. Surprisingly, we found no evidence that norepinephrine fluctuations, as measured by pupillometry, associated with this response. We conclude that the processes underlying a decision can rapidly reconfigure to adapt action selection policy under multiple forms of environmental uncertainty.