Dopamine Activity in Monkeys

JANUARY 5, 2005

Interesting discussion with a friend of mine that led on to subject of addiction. He brought up recent research on dopamine activity in monkeys by Schulz. This was continuing a long line of work which had originally demonstrated a learning effect in the dopamine system as follows. Take a monkey and give it a reward. You see dopamine get released as a form of reward (dopamine is a neurotransmitter that appears to both act as reward and increase plasticity of synapses and neurons). Next you start pairing the reward with a previous signal stimulus such as noise. Gradually over time you see the dopamine release move backwards in time from the point at which the reward is actually received to the point in time when the signal is given. (Aside: interesting parallel with adaptive expectations in economics).

Next you modify the experiment by only giving the monkey a reward with probability p (you have to obviously to quite a bit of training for this). Then what you find is that the surge in dopamine still occurs when the signal is received but the magnitude of this response is an increasing function of the entropy of the signal, given by - Sum (p log p). In this case this = - ( p log(p) + (1-p) log(1-p)) and is maximized at 1/2. This is very interesting. Why would it happen. One obvious reasons presents itself. When entropy is maximal crudely there is the most degree of uncertainty. This would suggest there is most to gain by doing something to try and influence the outcome, it would also suggest there is the most to gain in terms of new information by paying attention and learning from the outcome (dopamine increases plasticity, ie. learning).

This also demonstrates how the learning system gets hijacked by a truly random process. In general experiences are not truly random but will follow some pattern and gradually over time you can learn this pattern and alter your behaviour to take account of it. But in this case the probabilities will never change. This then provides the supposed connection with gambling addiction, with the reinforcement coming from the surge of dopamine in response to uncertainty. However the analogy is not very exact given that in fact gambling generates a very low p and hence low entropy. It is actually pretty certain that you will lose. Gambling seems to exploit something else in us namely that we respond to the structure of the payoff, in particular having a very large payoff but with very small probability. Many fewer people would gamble if there were no jackpot and the same odds were offered but on a payoff of, say, £10.