Evaluating gambles using dynamics

Gambles are random variables that model possible changes in monetary wealth. Classic decision theory transforms money into utility through a utility function and defines the value of a gamble as the expectation value of utility changes. Utility functions aim to capture individual psychological characteristics, but their generality limits predictive power. Expectation value maximizers are defined as rational in economics, but expectation values are only meaningful in the presence of ensembles or in systems with ergodic properties, whereas decision-makers have no access to ensembles and the variables representing wealth in the usual growth models do not have the relevant ergodic properties. Simultaneously addressing the shortcomings of utility and those of expectations, we propose to evaluate gambles by averaging wealth growth over time. No utility function is needed, but a dynamic must be specified to compute time averages. Linear and logarithmic"utility functions"appear as transformations that generate ergodic observables for purely additive and purely multiplicative dynamics, respectively. We highlight inconsistencies throughout the development of decision theory, whose correction clarifies that our perspective is legitimate. These invalidate a commonly cited argument for bounded utility functions.

compared, the decision being which to subject one's wealth to, and more generally to what extent.
Gambles are versatile models, useful to describe a number of real-world prospects. An insurance contract may be modeled as a gamble, as may an investment. Lotteries are important in the historical development of decision theory. Here, possible payouts D(n) are purchased for a ticket price, P , leading to changes in monetary wealth, ∆W (n) = D(n) − P , that are negative up to some value of n and then positive. This creates a decision problem, when comparing to the option of doing nothing: the certain unpleasant prospect of losing the ticket price has to be weighed against the uncertain prospect of winning one of the n max possible payouts.

Outline
Section 3 is a modern treatment of the problem, using dynamics. The two most commonly considered quantities are the expected rate of change in wealth, 1 ∆t ∆W , for additive dynamics, and the expected exponential growth rate of wealth, 1 ∆t ∆ ln(W ) , for multiplicative dynamics.
Both quantities were suggested as criteria to evaluate a gamble, 1 ∆t ∆W by Huygens (1657), and 1 ∆t ∆ ln(W ) by Laplace (1814), although time scales ∆t were usually omitted and implicitly set to 1.
Section 4 discusses the complicated historical development of these two criteria, which we now briefly summarize. It is necessary to re-tell the history of the problem because of an important misconception that forbids the modern perspective. Bernoulli (1738) suggested a quantity similar to the exponential growth rate and called it a "moral expectation," interpreting the logarithm in the exponential growth rate as a psychological re-weighting that humans apply to monetary amounts. This presented a very simple criterion -maximizing the expected exponential growth rate -in a very complicated way. Laplace (1814) corrected Bernoulli formally, though not conceptually, writing down exactly the expected exponential growth rate, though not pointing out its dynamical significance. Menger (1934) did decision theory a crucial disservice by undoing Laplace's correction, adding further errors, and writing a persuasive but invalid paper on the subject that concluded incorrectly -in the language of utility theory -that only bounded utility functions are permissible. This forbade the use of either of the dynamically sensible quantities because -forced into the framework of utility theory -the expected rate of change in wealth corresponds to a linear (unbounded) utility function, and the expected exponential growth rate corresponds to a logarithmic (unbounded) utility function.
That bounded utility functions are not allowed became an established result. We ask here why we cannot use unbounded utility functions, and find no good reason. The arguments for the boundedness of utility functions that we found are not scientifically compelling. A visual representation of the convoluted history of the problem is shown in Fig. 1.
We conclude in Section 5 that the modern dynamic perspective is legitimate and powerful. The requirement of boundedness for utility functions is both unnecessary and detrimental to the formalism of decision theory. We aim to remove this unnecessary obstacle in the way of using physically sensible criteria in decision theory.

Bernoulli
Attempted to mitigate Huygens' problems, but introduced unnecessary complications.
PROBLEMS: Introduced utility as non-linear mapping of money. Failed to compute expected rate of change of utility.
Only arguments based on Bernoulli require utility.

Laplace
Corrected Bernoulli and computed expected exponential growth rate.
Ergodic observable for multiplicative growth process.
PROBLEM: Conceptually remained within utility framework.
No utility required. Typically, wealth processes are better modeled as multiplicative than as additive, meaning that Laplace's Criterion is usually more relevant, especially when changes in wealth ∆W are of similar scale as wealth W itself. Problematic aspects are color-coded in red.

The dynamic perspective
In order to evaluate a gamble, we ask how the dynamics that the gamble is part of are to be modeled. With this information we can construct an ergodic observable whose expectation value reflects the behavior over time.
Treating ∆W (n) as a stationary random variable, repetition of a gamble may mean different things. Firstly, a gamble may be repeated additively, so that the wealth after T rounds of the gamble is where n τ is the value of the random variable n in the τ th round of the gamble. Equation (1) implies that absolute changes in wealth, W (t 0 + T ∆t) − W (t 0 ), are stationary, i.e. their distribution does not depend on t 0 . Relative changes are not stationary. In this case the long-time average of the rate of change in wealth converges to the expectation value with probability one, The expectation value, by definition, is identical to the large-ensemble average, where W ν (t 0 + ∆t) are different parallel realizations of wealth after one round of the gamble.
This explains Huygens' Criterion: under additive dynamics, the rate of change in wealth is an ergodic observable, and he who chooses wisely with respect to its expectation value also chooses wisely with respect to the long-time average.
Secondly, a gamble may be repeated multiplicatively. To simplify notation, we define per-round relative returns r(n) = W * +∆W (n) W * , where W * is a reference-wealth. These inherit their stationarity from the stationarity of ∆W (n). In this case, which may be re-written as Under the dynamic given by (Eq. 3) relative changes in wealth, W (t 0 +T ∆t) , are stationary, i.e. their distribution does not depend on t 0 . Absolute changes are not Figure 2: Assume initial wealth W (t 0 ) = $1 and toss a fair coin. If tails shows (n = 1), W decreases to W (t 0 + ∆t) = $0.60. If heads shows (n = 2), W increases to W (t 0 + ∆t) = $1.50. The gamble is repeated (a) additively, linear plot, and (b) multiplicatively, log-linear plot (zoom-ins below the main panels). For clarity, the same sequence of heads and tails is used in both plots, and the colorcodings are identical. A typical trajectory is shown (magenta lines). Under (a) the expectation value of W (dashed line) grows in time with the expected rate of change (ergodic observable for this dynamic, blue line), and a trajectory growing exponentially at the expected exponential growth rate (green line) does not describe the long-time behavior. Under (b) the expectation value of W grows exponentially but has nothing to do with the long-time behavior -W typically decays exponentially in this case, following the expectation value of the exponential growth rate (ergodic observable for this dynamic). Linear growth in time at the expected rate of change in W does not describe the long-time behavior.
stationary. In this case the long-time average of the rate of change in the logarithm of wealth, i.e. the exponential growth rate, converges to the expectation value with probability one, The expectation value, by definition, is identical to the large-ensemble average, . This explains Laplace's Criterion: under multiplicative dynamics, the rate of change in the logarithm of wealth is an ergodic observable, and he who chooses wisely with respect to its expectation value also chooses wisely with respect to the long-time average. Multiplicative repetition is exemplified by geometric Brownian motion, the most influential model in mathematical finance.
From this modern perspective, the concept of utility is not needed to resolve problems such as the St Petersburg paradox (Peters, 2011b).

Common error
Prominent texts in decision theory make incorrect statements about ergodicity, i.e. the equality of expectation values and time averages, as for instance in the following passage: "If a game is 'favorable' from the point of view of the expectation value and you have the choice of repeating it many times, then it is wise to do so. For eventually, your amount of money and, consequently, your utility are bound to increase (assuming that utility increases if money increases)," (Chernoff and Moses, 1959, p. 98).
Chernoff and Moses' statement is not true if "favorability" is judged by an observable that is non-ergodic for a given dynamic. The general falsity of their statement is evident in panel (b) of Fig. 2, an example of the multiplicative binomial process, studied in detail by Redner (1990), see also http://youtu.be/LGqOH3sYmQA. Here, W is not ergodic, and the game is "favorable from the point of view of the expectation value" of W , but it is certainly not wise to repeat it many times. We will use red text in square boxes to highlight errors and weaknesses in arguments that are commonly believed to be valid.

Historical development of decision theory
In this section we relate the modern treatment of the gamble problem to classic treatments and highlight common misconceptions.

Pre-1713 decision theory -expected wealth
Following the first formal treatment by Fermat and Pascal (1654) of random events, it was widely believed that gambles are to be evaluated according to the expected rate of change in monetary wealth. To give it a label, this criterion may be attributed to Huygens (1657), who wrote "if any one should put 3 shillings in one hand without telling me which, and 7 in the other, and give me choice of either of them; I say, it is the same thing as if he should give me 5 shillings..." Huygens' Criterion: Maximize the rate of change in the expectation value of wealth, In modern terms, Huygens suggested to maximize the ergodic growth rate assuming additive dynamics. Nicolas Bernoulli, in a letter to Montmort (1713) challenged this notion by introducing a lottery whose expected payout, D(n) , diverges positively. Since the expected rate of change in wealth 1 ∆t ∆W (n) = 1 ∆t ( D(n) − P ) is linear in D(n) , it too diverges for any finite ticket price P . According to Huygens' Criterion any finite ticket price should be paid for the lottery. However, N. Bernoulli chose the lottery such that large gains only occur with small probability, and found that typical individuals when (hypothetically) offered this lottery were not willing to pay much to enter. This seeming incongruence became known as the St Petersburg paradox.

It exposes
Huygens' weakness Expectation values are averages over (imagined or real) ensembles of random realizations. The conceptual weakness of Huygens's Criterion is its limited relevance to an individual making a decision. Either the individual has to be part of a large resource-sharing group mimicking a statistical ensemble, or the wealth process W (t) has to be additive for the rate of change to be ergodic so that the expectation value reflects how the individual will fare over time. Wealth is often better modeled with multiplicative dynamics.
Specifically, N. Bernoulli proposed the following lottery: a fair coin is tossed until the first heads event occurs. The number of coin tosses necessary to arrive at this event is n ∈ N, probability mass function p n (n) = 2 −n , and the payout as a function of n is D(n) = $2 n−1 . It follows that D(n) is power-law distributed with diverging first moment, probability mass function p D (D) = (2D/$) −1 with D ∈ {$2 n−1 }. The time ∆t to generate an instance of the random variable, i.e. to play the lottery, is considered independent of n in this study. The lottery is usually presented without restriction on n; for a careful treatment of the problem one must limit n ≤ n max . For more than n max coin tosses the lottery is declared invalid and no change in wealth occurs. The divergence of D(n) is observed in the limit n max → ∞.

1738-1814 decision theory -utility
By 1738 N. Bernoulli's cousin Daniel Bernoulli and Cramer (Bernoulli, 1738, p. 33) had conceptualized the problem as follows. They argued that people attach a value to money that is non-linear in the dollar amount. Cramer had written to N. Bernoulli in 1728: "in their theory [i.e. Huygens' Criterion] mathematicians evaluate money in proportion to its quantity while, in practice, people with common sense evaluate money in proportion to the utility they can obtain from it." Bernoulli suggested a logarithm to map a dollar amount into utility U B (W ) = ln(W ). The quantity, Bernoulli suggested, that people consider when deciding whether to take part in the lottery is a combination of the expected gain in their utility if no ticket price were paid, and the loss in utility they suffer when they pay the ticket price. This leads to Bernoulli's Criterion: a lottery ticket is worth buying if the following quantity is positive (Bernoulli, 1738, pp. 26-27): The first terms on either side of the equation represent the expected gain in logarithmic utility, resulting from the payouts of the lottery. This would represent the net change in utility if tickets were given away for free, P = 0. The second terms represent the loss in logarithmic utility suffered at the time of purchase, i.e. after the ticket is bought but before any payout from the lottery is received. This is inconsistent with expected-utility theory, as was pointed out in (Peters, 2011c).
Bernoulli's inconsistency Bernoulli's Criterion is mathematically inconsistent with later work in expectedutility theory because Bernoulli did not calculate the expected net change in logarithmic utility. He did not only replace money with utility of money but also computed an observable other than the expected change in this new object.

1814-1934 decision theory -expected utility
The consensus in the literature on utility theory is that Bernoulli meant to compute the expected net change in utility and made a slight error. Laplace (1814) re-told Bernoulli's resolution of the St Petersburg paradox and the invention of utility. Perceiving Bernoulli's Criterion as an error, he implicitly "corrected" Bernoulli's formal inconsistency without mention.
Laplace's Criterion: Maximize the expected rate of change in logarithmic utility (Laplace, 1814, pp. 439-442), Later researchers adopted Laplace's corrected criterion. Todhunter (1865) followed Laplace, as do modern textbooks in stating that utility is an object encoding human preferences in its expectation value, e.g. (von Neumann and Morgenstern, 1944;Chernoff and Moses, 1959;Samuelson, 1983). Laplace stayed within Bernoulli's conceptual framework and was almost certainly not aware of the physical interpretation of his criterion as the ergodic growth rate under multiplicative dynamics (Eq. 5). Bernoulli motivated the logarithm by suggesting that the perceived utility change induced by an extra dollar is inversely proportional to totaly wealth, dU (W ) = 1/W , whose solution is the logarithm. But Bernoulli also considered Cramer's suggestion of U C = √ W a good representation of diminishing marginal utility. The modern perspective takes Bernoulli's logarithm more seriously than he himself did. The route to the modern treatment is to ask: "what if the logarithm was not merely convenient and a good fit to the data, what would be its physical meaning if it truly was a logarithm?" Using the logarithm in exactly the same place as the utility function is equivalent to assuming multiplicative dynamics and constructing an ergodic observable.

Post-1934 decision theory -bounded utility
Karl Menger (1934) re-visited Bernoulli's 1738 study, and came to the incorrect conclusion that only bounded utility functions are permissible. Of course, whether a utility function, or anything else, is bounded or not in the limit of diverging wealth is practically irrelevant because financial wealth will always be represented by a finite number. However, based on formal arguments Menger drew conclusions for the structure of the permissible formalism, namely he ruled out linear and logarithmic functions as models of behavior, and, equivalently, additive and multiplicative processes as models of wealth. Because of the central role of these dynamical models the development of decision theory suffered from this restriction, and it is satisfying to see that formal arguments against these important models are invalid, as intuition would suggest. Menger must have been unaware of the correction to Bernoulli's work by Laplace. His error may be phrased as using Bernoulli's criterion instead of Laplace's, and only considering the first term in Bernoulli's criterion, implicitly setting the ticket price to zero, P = 0. The invalidity of Menger's claim was pointed out in (Peters, 2011c), for a detailed discussion, see (Peters, 2011a). Menger's argument survives as received wisdom. For completeness, we state it here and specify the invalid inferences involved.
Menger's flawed argument 1. Logarithmic utility resolves the original St Petersburg paradox because it turns exponentially increasing wealth-payouts, D n ∝ exp(n), into linearly increasing utility-payouts ∆U (n) ∝ n for large n. 2. If payouts increase even faster, e.g. as the exponential of an exponential, exp(exp(n)), then expected utility changes will diverge positively as n max diverges, just as expected wealth changes diverge for exponentially increasing payouts. 3. In such games logarithmic utility predicts that the player will be willing to pay any ticket price, just as linear utility does for exponentially increasing payouts. In this sense logarithmic utility is not qualitatively different from linear utility. For utility theory to achieve the desired generality, utility functions must be bounded.
The argument sounds plausible. If the logarithm specifies the value attached to money, there is no intuitive reason why it should be qualitatively different from a linear function. But the logarithm encoding multiplicative dynamics provides us with additional intuition: multiplicative dynamics imply an absorbing boundary. Unlike under additive dynamics it is impossible to recover from bankruptcy, and this is a qualitative difference. Closer inspection of Menger's argument reveals that the issue is indeed more nuanced than he thought.
We separate out the first term, for the smallest payout, and write the expected utility change as This form motivates the following evaluation of the three steps in Menger's argument 1. Apart from turning exponential wealth changes into linear utility changes, logarithmic utility also imposes a no-bankruptcy condition. Bankruptcy becomes possible at P = W + D(1). Reflecting this, the limit lim P →W +D(1) ∆U B (W ) is negatively divergent for any n max . 2. If payouts increase as the exponential of an exponential then the expected utility change is positively divergent in the limit n max → ∞ only for ticket prices satisfying P < W + D(1). The double-limit lim P →W +D(1) lim nmax→∞ 1 ∆t ∆U B (W ) results in the indeterminate form "−∞ + ∞." Note that the positive divergence only happens in the unrealistic limit n max → ∞, whereas the negative divergence happens at finite P . The negative divergence is physically meaningful in that it reflects the impossibility to recover from bankruptcy under multiplicative dynamics. 3. In such games logarithmic utility does not predict that the player will want to pay any finite ticket price. Instead, it predicts that the player will not pay more than W + D(1), irrespective of how D(n) may diverge for large n. This is qualitatively different from behavior predicted by Huygens' criterion (linear utility), where under diverging expected payouts no ticket price exists that the player would not be willing to pay. Logarithmic utility, carefully interpreted, resolves the class of problems for which Menger thought it would fail.
Despite a persisting intuitive discomfort, renowned economists accepted Menger's conclusions and considered them an important milestone in the development of utility theory. Menger implicitly ruled out the all-important logarithmic function that connects utility theory to information theory (Kelly Jr., 1956;Cover and Thomas, 1991) and provides the most natural connection to ergodic theory (Peters, 2011b,c;Peters and Klein, 2013). Menger also ruled out the linear function that corresponds to Huygens' Criterion, which utility theory was supposed to generalize.
Requiring boundedness for utility functions is methodologically inapt. It is often stated that a diverging expected utility is "impossible" (Chernoff and Moses, 1959, p. 106), or that it "seems natural" to require all expected utilities to be finite (Arrow, 1974, p. 28-29). Presumably, these statements reflect the intuitive notion that no real thing can be infinitely useful. To implement this notion in the formalism of decision theory, it was decided to make utility functions bounded. A far more natural way to implement the same notion would be to recognize that money amounts (and quantities of anything physical, anything money could represent) are themselves bounded, and that this makes any usefulness one may assign to them finite, even if utility functions are unbounded. There is no need to place bounds on U (W ) if W itself is bounded.

Summary and conclusion
In presenting our results we have made a judgement call between clarity and generality. We have focused on the prototypical gamble problem of decision theory, discrete in time and wealth changes, and we have contrasted purely additive dynamics with purely multiplicative dynamics. Gambles that are continuous in time and wealth changes can be treated along the lines of (Peters, 2011b). The specific St Petersburg problem was treated in detail in (Peters, 2011c). A generalization beyond purely additive or multiplicative dynamics is possible, just as it is possible to define utility functions other than the linear or logarithmic function. This will be the subject of a future publication (Peters and Adamou, 2014).
Our method starts by recognizing the inevitable non-ergodicity of stochastic growth processes, e.g. noisy multiplicative growth. The specific process implies a set of meaningful ergodic observables, e.g. the exponential growth rate. These observables make use of a mapping that is traditionally viewed as a utility function, e.g. the logarithm.
The dynamic interpretation of the gamble problem makes sense of risk aversion as optimal behavior for a given dynamic and wealth. Laplace's Criterion interpreted as an ergodic growth rate under multiplicative dynamics avoids the fundamental circularity of the behavioral interpretation. In the latter, preferences, i.e. choices an individual would make, have to be encoded in a utility function, the utility function is passed through the formalism, and the output is the same as the input: the choices an individual would make.
We have repeated here that Bernoulli (1738) did not actually compute the expected net change in logarithmic utility, as was pointed out in (Peters, 2011c). Perceiving this as an error, Laplace (1814) corrected him implicitly without mention. Later researchers used Laplace's corrected criterion until Menger (1934) unwittingly re-introduced Bernoulli's inconsistency and introduced a new error by neglecting the second diverging term, ∆U − . Throughout the twentieth century, Menger's incorrect conclusions were accepted by prominent economists although they noticed, and struggled with, detrimental consequences of the (undetected) error for the developing formalism.
We have presented Menger's argument against unbounded utility functions as it is commonly stated nowadays. This argument is neither formally correct (it ignores the negative divergence of the logarithm), nor compatible with physical intuition (it ignores the absorbing boundary). Laplace's Criterion -contrary to common beliefelegantly resolves Menger-type games.
Logarithmic utility must not be banned formally because it is mathematically equivalent to the modern method of defining an ergodic observable for multiplicative dynamics. This point of view provides a firm basis on which to erect a scientific formalism. The concepts we have presented are not restricted to monetary wealth but apply to anything that is well modeled by a multiplicative stochastic growth process. Applications to ecology and biology seem natural. Some consequences of this different approach have been reported (Peters, 2011b,c;Peters and Klein, 2013;Peters and Adamou, 2013), and more will be the subject of future publications.

Symbol
Name and interpretation t time t 0 time before the gamble ∆t duration of one round of a gamble T total number of sequential rounds of a gamble τ integer specifying one sequential round of a gamble N total number of parallel realizations of a gamble ν index specifying one parallel realization of one round of a gamble n integer specifying an event n max number of possible events n τ random event that occurs in round τ W wealth ∆W (n) change in wealth from t to t + ∆t if event n occurs W ν (t + ∆t) wealth after one round of a gamble in realization ν p n (n) probability of event n p D (D) probability of monetary payout D D(n) payout resulting from a lottery if event n occurs ∆U (n) change in utility resulting from event n n max maximum number of coin tosses P price for a ticket in a lottery U utility function U C Cramer's square-root utility function U B Bernoulli's logarithmic utility function ∆U + B expectation value of gains in logarithmic utility at zero ticket price ∆U − B loss in logarithmic utility when reducing W by P · expectation value of · N set of positive integers W * reference-wealth to define multiplicative repetition