Lecture 13.1: decision making

— subjective utility

— virtual machine

— neuroeconomics

indecision

Some of the pros and cons of marriage: "Children -(if it Please God) -Constant companion, (& friend in old age) who will feel interested in one, -object to be beloved and played with. -better than a dog anyhow. -Home, & someone to take care of house -charms of music and female chit-chat. -These things good for one's health. -but terrible loss of time. -."

slide 2

indecision

slide 3

subjective utility

In decision making, the subjective dimensions — such as the value placed on the possible consequences — are more pronounced than in problem solving or in reasoning.

computing subjective utility in the brain

With the general-purpose computational device — the brain's virtual machine — dormant except in rare cases of explicit reasoning to which people are normally averse, the decision making system must fall back on mechanisms available to it by default: those of perception (just as the problem solving system does; cf. chunking in chess).

---

Unlike in a perceptual decision task, in general decision making reward is not perceived by a dedicated sensory system, and must therefore be estimated by the agent by tracking the outcomes of past decisions.

---

VTA: ventral tegmental area, the midbrain locus of dopaminergic neurons.

slide 5

digression: the brain's virtual machine

A virtual machine is a process that offers, usually by means of considerable extra computation, a functionality that is not directly available in the environment that supports it.^*

Because it abstracts computation away from neurobiology, a VM implemented by the brain transcends the limitations of its parallel, associative probabilistic circuit architecture, such as the impossibility of flexible, random access to memory.

The flexibility of the VM comes at the cost of much slower, serial processing that is confined to low-capacity working memory.

Working memory and virtual machine computation are key functional components of general fluid intelligence (see Lecture 14.1).

---

^*Compare this with the concept of virtual computation, introduced in the context of social memory and cognition.

slide 6

digression: Banburismus

Alan Turing

decision-making at Bletchley Park

causality in WWII

---

A screen shot of a fully functional simulation of the Enigma, available as a program for a modern general-purpose computer (which acts as a virtual machine).

slide 7

modeling decision-making behavior

The diffusion model: the two zigzagging sample paths represent moment-by-moment fluctuations in the evidence favoring one or the other response.

The process starts at z and accumulates evidence until it reaches one of two criteria, 0 or a. If the upper criterion is reached first, a `right' response is made; if the lower is reached first, a `left' response is made.

The fluctuations in the sample paths reflect noise in the decision process.

slide 8

Evidence in favor of h₁ over h₀: the cumulative difference between responses of a neuron tuned to h₁ and an "antineuron" tuned to h₀.

The meandering line represents how the weight of evidence might grow in a single trial as a function of time; the dashed line depicts the expectation (mean value) of this trajectory at each time point.

If the weight of evidence reaches the barrier at B, the process is stopped and a decision is made for h₁.

slide 9

neuroeconomics

Responses from neurons in the lateral intraparietal area (LIP) of the monkey during a motion direction discrimination task (recall Lecture 4.1).

Left: responses averaged from a population of LIP neurons, aligned to the onset of the motion stimulus. After an initial dip, the responses recover and begin to increase roughly linearly, with a slope that is approximately proportional to the motion coherence. These neural responses are thought to represent the accumulated weight of evidence in favor of one direction of motion.

Right: shows the LIP responses aligned to the beginning of the monkey's eye movement. When the response reaches a value of B=65 spikes/second, the monkey is committed to a decision, which is communicated by the eye movement 80 ms later.

slide 10

extra: neuroeconomics of eye movements

Area LIP maps the relative expected utilities for all possible saccades.

The utility values are sent to the frontal eye fields, which enforces a winner-take-all outcome.

The resulting best-candidate saccade target is queued for execution if it passes an activation threshold in the superior colliculus.

slide 11

neuroeconomics of eye movements (cont.)

The behavior of monkeys in a two-alternative lottery game, in which the subject must choose, in each trial, between two alternatives that have different, cumulative, independent probabilities of yielding a reward — 0.25 ml of juice.

The subject's probability of making either response is a linear function of the ratio of reward rates, which a game-theoretic analysis shows to be an efficient strategy under these conditions.

slide 12

neuroeconomics of eye movements (cont.)

A neuroeconomic model of probability matching, which computes an exponentially weighted average of the gains brought about by prior responses.

The gains are computed iteratively from the reward prediction error signal carried by dopaminergic neurons.

The average gain generates the physiological expected utility (PEU) for each movement, which is then used to construct the expected utility map in area LIP.

slide 13

neuroeconomics of eye movements (cont.)

wwtmd (what would the monkey do?)

Dynamic one-step-ahead prediction of the model:

the black line is a 20-trial moving average of a subject's choice behavior during three consecutive blocks of plays;

the red line is the dynamic behavior of the model.