Ainslie: Annotated Bibliography

Hyperbolic Discounting and Temporary Preference.  The first experiments on hyperbolic discounting using the necessary discrete trial design showed that some pigeons would peck a key, the only effect of which was to commit them not to peck a later key to get an immediate but inferior grain reward. Although only a minority of birds learned to do this, their learning was robust, showing that these subjects were motivated to forestall their own future behavior. This research was first described in Howard Rachlin's book, Introduction to Modern Behaviorism (San Francisco, Freeman, 1970). It was reported in detail in Ainslie's Impulse Control in Pigeons (Journal of the Experimental Analysis of Behavior 21:485-489, 1974).

These findings implied temporary preference for a reward that was unpreferred at a distance, but this preference pattern was not demonstrated directly until pigeons were tested in what became the standard self-control paradigm: They chose between a shorter duration of access to grain at delay D on one key, and a longer duration of access to grain at delay D plus lag L on another. As D was varied from 0 to 12 seconds all subjects switched their preference from the shorter to the longer access, with lag L held constant at four seconds (Ainslie and Herrnstein, Preference Reversal and Delayed Reinforcement. Animal Learning and Behavior 9:476-482, 1981).

In research with Varda Haendel (The Motives of the Will. In: E. Gottheil, K. A. Druley, T. E. Skoloda and H. Waxman, Eds., Etiologic Aspects of Alcohol and Drug Abuse. Springfield, Illinois: Charles C. Thomas, 1983) even human subjects were found to report temporary preferences for smaller, earlier amounts of money over larger, more delayed amounts, whether or not the subjects had a substance abuse disorder and whether or not real money was at stake.

Temporary preference has now been found in many studies using the self-control paradigm in human and animal subjects. These are reviewed in Picoeconomics: The Strategic Interaction of Successive Motivational States Within the Person (New York: Cambridge University, 1992) and Breakdown of Will (New York: Cambridge U., 2001 (release scheduled for April). A review with John Monterosso (Beyond Discounting: Possible Experimental Models of Impulse Control. Psychopharmacology 146, 339-347, 1999) compares temporary preference with other models of impulsiveness.
 
 

First Discussion of Implications. The implications of temporary preference based on delay were explored in Specious Reward: A Behavioral Theory of Impulsiveness and Impulse Control (Psychological Bulletin 82:473-496, 1975). This article reviewed theories of impulsiveness and impulse control in several behavioral sciences, noted that some described solutions that amounted to precommitment of choice at a distance, and pointed out that hyperbolic discount curves predict a relationship to your own future selves that somewhat resembled relationships with another person. This situation would motivate not only concrete forms of precommitment like taking Antabuse or depending on social approval, but also the making of "private side bets"-- an analog of economists' familiar self-enforcing contracts, in which you stake your expectation of future self-control against each successive impulse to seize a poor but imminent alternative. Under the term "personal rules" these bets have subsequently been discussed at length as the basic mechanism of willpower.

"Specious Reward" opened discussions on several other topics that have remained important in picoeconomics, including the compulsiveness that results from private side betting, the value of external facts for pacing freely available rewards to avoid premature satiation, and the possibility of constructing aversive experiences from sequences of reward and nonreward.

Addiction. The value of hyperbolic discounting in studying addictions was explored with Elmer Schaefer in The Application of Economic Concepts to the Motivational Conflict in Alcoholism. In: E. Gottheil, A.T. McLellan and K. A. Druley, Eds., Patient Needs and Treatment Methods. Springfield,Illinois: Charles C. Thomas, 1981.

A Research-Based Theory of Addictive Motivation (Law and Philosophy 19, 77-115, 2000) suggests mechanisms for addiction that redraw the conventional dichotomy between disease- and willful models
 
 

Defense Mechanisms. A Behavioral Economic Approach to the Defense Mechanisms: Freud's Energy Theory Revisited (Social Science Information (London) 21(6):735-779, 1982) distinguished the four possible categories of precommitment that have been used in subsequent analyses. Extrapsychic controls involve physical devices or social commitments; attention control keeps you from getting or processing information about the availability of short range rewards; emotion control nips seductive emotions in the bud, or fosters useful ones; private rules (later personal rules) give current choices additional value as precedents predicting future choices.

This article also proposed a re-founding of psychoanalytic metapsychology using hyperbolic discount curves rather than repression as the basic mechanism of impulses. The role of temporary preferences seems especially clear in the Freudian defense mechanisms, a point that was developed also in Freud and Picoeconomics (Behaviorism 17:11-19, 1989. A taxonomy of psychotherapy techniques based upon their strategies for dealing with both temporary preference and misguided countermeasures was proposed in Manipulation: Theoretical and Historical Perspectives. In: M. Nichols and T. Paolino, Eds., The Basic Techniques of Psychodynamic Psychotherapy (New York: Gardner, 1986. Pp. 127-166).

A study of self-reports by college students and jail inmates revealed correlates of the four precommitment tactics described in the Social Science Information article (Self-Reported Tactics of Impulse Control. International Journal of the Addictions, 22:167-179,1987). Endorsement of personal rules as useful tactics was correlated with compulsive traits, and negatively correlated with endorsement of extrapsychic devices.
 
 

Aversion and Classical Conditioning. Perhaps the most counterintuitive prediction of picoeconomics has been its distinction between the concepts of pleasure and reward. Temporary preferences shorter than those in addictions are experienced as itches, which reward attention to them but are only marginally pleasurable. It was hypothesized that an even more rapid cycle of reward and subsequent nonreward would be experienced as pain. This mechanism renders unnecessary a second kind of reinforcement for attending to aversive events, the supposed role of unconditioned stimuli in models where unwanted experiences are imposed by classical conditioning. Classical conditioning itself can be understood as a variety of reward-dependent learning. From the original suggestion in "Specious reward," this possibility was developed in a critique of behaviorist pain theory (Behavior Is What Can Be Reinforced. Behavioral and Brain Sciences 8:53-54, 1985) and expanded in Aversion with Only One Factor (In: M. Commons, J. Mazur, J. Nevin and H. Rachlin, Eds., Quantitative Analyses of Behavior, Volume 5: Reinforcement Value: Delay and Intervening Events. Hillsdale, N.J.: Erlbaum, 1987, P. 127-139. (The most recent critique of "two-factor" (conditioning + reward-based learning) theories is The Intuitive Explanation of Passionate Mistakes, and Why it is Not Adequate. In Jon Elster, Ed. Addiction: Entries and Exits. New York: Sage, 1999, pp. 209-238.)
 
 

Another argument against the necessity of classical conditioning was based on the experimental finding that a supposedly conditioned response in monkeys, increased heart rate during impending shock, could be reversed by operant learning (with Bernard Engel: Alteration of Classically Conditioned Heart Rate by Operant Reinforcement in Monkeys. Journal of Comparative and Physiological Psychology 87:373-383, 1974).
 

Game Theory. The usefulness of game theory in understanding the conflict among successive selves was first described in Beyond Microeconomics: Conflict among Interests in a Multiple Self as a Determinant of Value (In: J. Elster, Ed., The Multiple Self. Cambridge, U.K.: Cambridge U., 1986. P;. 133-175). The agents in intra-personal (intertemporal) bargaining were defined as interests, the sets of behaviors that had grown to reach particular goals. Long range interests could survive the lure of short range interests by defining personal rules, which became self-enforcing through perception of each relevant choice as a move in a game analogous to the repeated prisoner's dilemma. This work first suggested the term, picoeconomics.

In many situations people seem to behave as consistent, rational choice-makers. Factors that could permit this pattern despite a fundamental tendency to discount future events hyperbolically were described in Derivation of 'Rational' Economic Behavior from Hyperbolic Discount Curves. (American Economic Review 81:334-340, 1991).

The hypothetical process that is most important to picoeconomic analysis is the summation of discounted values from series of alternative rewards, which permits an intertemporal bargaining solution to the problem of temporary preferences. This solution fits many descriptions of the will, but since it would be a recursive process, it is hard to study by controlled experiment. Its relationship to other behavioral economic theories of will-like processes are discussed in Matching is the Integrating Framework (Behavioral and Brain Sciences 11:679-680, 1988), How do People Choose between Local and Global Bookkeeping? (Behavioral and Brain Sciences 19, 574-575, 1996), and Intention isn't Indivisible (with Barbara Gault-- Behavioral and Brain Sciences 20, 365-366, 1997). It is contrasted with the cognitivist approach in Studying Self-regulation the Hard Way (Psychological Inquiry 7, 16-20, 1996).

Several kinds of evidence now point to intertemporal bargaining as the mechanism of both the power and the freedom of the will. These include controlled experiments on animals and humans showing that linking series of choices leads to more delay of gratification, goodness of fit of expectable bargaining patterns with descriptions of will over the ages, interpersonal bargaining experiments with humans that model intertemporal bargaining, and thought experiments from the philosophy of mind. These are brought together for the first time in Breakdown of Will. This work also represents the latest update of several explorations of the implications of hyperbolic discounting:

A process by which personal rules come to imprison a person and even get perceived as external, alien forces was proposed in Behavioral Economics II: Motivated, Involuntary Behavior (Social Science Information (London) 23:47-78, 1984), which also further analyzed premature satiation. The negative side effects of intertemporal bargaining are more fully explored in The Dangers of Willpower (In Jon Elster and Ole-Jorgen Skog, Eds. Getting Hooked: Rationality and Addiction. Cambridge U.,1999, pp. 65-92.

A theory of emotions as freely available, sometimes seductive rewards that habituate unless paced by adequately rare and unpredictable external occasions was developed in Rationality and the Emotions: A Picoeconomic Approach (Social Science Information (London) 24:355-374, 1985). Short range reward that motivates attention but may be experienced as itch or pain is again contrasted with the long range reward that is experienced as pleasure.

The need for good occasions rather than turnkey releasers for emotions suggests a rationale for the social construction of beliefs-- that those beliefs not shaped by practical consequences are selected for their potential roles in reward-pacing strategies. This hypothesis was first argued in A Picoeconomic Rationale for Social Constructionism (Behavior and Philosophy 21, 63-75, 1993), and further described in the commentary, If Belief is a Behavior, What Controls it? (Behavioral and Brain Sciences 20, 103-104, 1997). The need for renewably unpredictable occasions for emotions suggests also a primary incentive for empathic human relationships, as opposed to the mutual cooperation hypotheses that are now standard. From an initial mention in "The Multiple Self," this possibility was developed in A utility-maximizing mechanism for vicarious reward (Rationality and Society 7, 393-403, 1995).
 

Pulling it Together.  The most complete overview of picoeconomics as of 1992 is Picoeconomics: The Strategic Interaction of Successive Motivational States Within the Person (New York: Cambridge University, 1992). It was summarized in two book chapters written with Nick Haslam, Hyperbolic Discounting and Self-control, in: G. Loewenstein & J. Elster, Eds., Choice Over Time. (New York: Russell Sage Foundation, 1992, pp. 57-92 and 177-209), and further condensed in Picoeconomics: A Bargaining Model of the Will and its Lapses. In Lennart Sjoberg, Richard P. Bagozzi, and David Ingvar, Eds. Will and EconomicBehavior. Stockholm: Economic Research Institute of the Stockholm School of Economics. 1998, pp. 163-174.

A more succinct and updated overview is Breakdown of Will (New York: Cambridge U., 2001). This is summarized in the precis at this website. An extremely condensed account of hyperbolic discounting is The Psychological Discount Rate (In Peter E. Earl and Simon Kemp, Eds., The Elgar Companion to Consumer Research and Economic Psychology. 1999, pp. 472-479).