# Objective Bayesian probability.

Introduction

IN SEVERAL ARTICLES Crovelli (2009, 2010, 2011) has criticized the objective theory of probability of Richard von Mises (1981), and has presented a case for defining probability subjectively. In this paper an alternative objective Bayesian approach to the treatment of probability is described, based on the work of E.T. Jaynes (2005).1 In order to do this, the different meanings of the terms 'objective' and 'subjective' are first clarified, the Bayesian approach to probability is outlined, and then a general definition of probability is given.

Objective and Subjective Probability

The words 'objective' and 'subjective' applied to probability have been used by many writers, usually without defining what is meant by these terms. The result can be confusion in the mind of a reader, since the words can have more than one meaning. An important distinction for our purposes has been expressed by Ayn Rand (1990, p. 18):
```   Objectivity is both a metaphysical and an epistemological concept.
It pertains to the relationship of consciousness to existence.
Metaphysically, it is the recognition of the fact that reality
exists independent of any perceiver's consciousness.
Epistemologically, it is the recognition of the fact that a
perceiver's (man's) consciousness must acquire knowledge of reality
by certain means (reason) in accordance with certain rules (logic).
```

The theory of Richard von Mises (1981) can therefore be regarded as objective in the metaphysical sense. According to this theory, probability is an inherent property of a physical entity (such as a coin or a die) or of an experimental setup. This attitude is nicely summarized by Gillies (1973, p. ix) in his support of the Mises theory, which regards
```   ... probability theory as a mathematical science similar to
mechanics or electrodynamics, and probability itself as an
objective measurable concept similar to mass or charge.
```

This objective probability would generally be obtained by repeated measurements of some kind (tossing a coin is a simple example), and the relative frequency of occurrence of a favorable result (heads or tails) would be an estimate of a probability. Supporters of this approach are therefore often called frequentists.

Any theory of probability that rejects this objective view is therefore subjective in the metaphysical sense. For such a theory, probability is regarded as being 'in the mind' as it were, i.e. as a concept derived from our experience of uncertainty in the world. As Jaynes has written (2005, p. 44):
```   In the theory we are developing, any probability assignment is
necessarily 'subjective' in the sense that it describes only a
state of knowledge, and not anything that could be measured in a
physical experiment.
```

This (metaphysically) subjective position can in turn give rise to two different approaches: objective and subjective, these terms now referring to their epistemological status, i.e. how the knowledge needed to assign a probability is acquired. The objective approach is represented by Jaynes (2005) and earlier probability theorists such as Harold Jeffreys (1961), and the subjective approach is associated primarily with Bruno de Finetti (1974). Followers of both approaches rely on Bayesian analysis, i.e. the use of Bayes's theorem, to calculate probabilities. In order to illustrate their basic difference, the essentials of the Bayesian approach need to be described.

Conditional Probability and Bayes's Theorem

In the discussion that follows, the word 'probability' will refer to propositions rather than to events. Events occur or do not occur, whereas propositions are true or false. Any event can be described in the form of a proposition, but propositions have a wider application, since they can be used to assert hypotheses. Thus one can speak of the probability that an hypothesis is true, e.g. the hypothesis that smoking causes lung cancer.

The notation for conditional probability is needed to illustrate Bayes's theorem. If A and I are propositions, we write a conditional probability as P(A 11), which is the probability that A is true, given that I is true. If B is also a proposition, then P(A | BI) is the probability that A is true, given that both B and I are true. If D represents some data obtained to evaluate the truth of proposition A, then we can write Bayes's theorem in its simplest form as

P(A | DI) proportional to P(D |AI) x P(A|I)

Here P(A | I) is called the prior probability of A, and I is our prior information, i.e. any information we have regarding the truth of A before obtaining the data. P(A | DI) is the probability of A after obtaining the data, and is called the posterior probability. This posterior probability can then be used as the prior probability in a subsequent calculation using new data, leading to sequential updating of probabilities. This notation emphasizes the modern Bayesian approach of regarding every probability as conditional on some information or knowledge.

Bayes's theorem follows directly from the laws of probability and is not controversial. Critics, however, point to the need for a prior probability P(A 11) in order to use the theorem, and claim that its value is simply a 'subjective' choice. It is subjective--in the metaphysical sense, as explained above. It is also subjective in the epistemological sense, if one is a follower of the de Finetti theory of probability. In that theory, probability is regarded as a personal 'degree of belief', and probability values are generally obtained from preferences in betting on the outcome of an event. But objectivists such as Jaynes (2005, p. 655) assert that
```   ... our concern is not with the personal probabilities that
different people might happen to have, but with the probabilities
that they 'ought to' have, in view of their information.
```

Hence the objective approach involves assigning a prior probability by logically analyzing the prior information available (see the Ayn Rand quote given earlier in this paper). From such a prior one can then calculate further probabilities using the laws of probability (and Bayes's theorem). Different people may, of course, have different prior information, but

Our goal is that inferences are to be completely 'objective' in the sense that two persons with the same prior information must assign the same probabilities. (Jaynes, 2005, p. 373)

Definition of Probability

As Crovelli (2011) pointed out, there have been two general approaches to defining probability: an 'objective' approach, which refers to repeated measurements of some kind, and a 'subjective' approach, which refers to a personal subjective belief. In this paper an alternative approach to understanding probability has been described, and this calls for a different definition. Jaynes did not give a formal definition of probability, but the above analysis suggests the following (using standard genus and differentia format):

Probability P(A |I) is a number between 0 and 1 that indicates how plausible it is that proposition A is true, based on information I.

In addition, one could add that a probability of 1 indicates certainty that the proposition is true, while a probability of 0 indicates certainty that the proposition is false.

As Jaynes (2005) discusses, obtaining an actual value for a prior probability at the start of a calculation can be a difficult problem, but is one that needs to be addressed. He has described several valuable methods for doing this, for example the concept of maximum entropy, and the use of symmetry and group theory arguments. For further details see chapters 11 and 12 of his book.

References

Crovelli, Mark R. (2009). 'On the possibility of assigning probabilities to singular cases, or: probability is subjective too!', Libertarian Papers 1, 26.

--. (2010). 'A challenge to Ludwig von Mises's theory of probability', Libertarian Papers 2, 23.

--. (2011). 'Why the Definition of Probability Matters,' Mises Daily (Jan. 26), mises.org/daily/4979/Why-the-Definition-of-Probability-Matters

de Finetti, Bruno (1974). Theory of Probability, John Wiley & Sons.

Gillies, D. A. (1973). An Objective Theory of Probability, Methuen & Co. Ltd.

Jeffreys, Harold (1961). Theory of Probability, 3rd edition, Oxford University Press.

Jaynes, E.T. (2005). Probability Theory: The Logic of Science, reprint with corrections, Cambridge University Press.

Mises, Richard von (1981). Probability, Statistics and Truth, 2nd revised English edition, Dover Publications.

Rand, Ayn (1990). The Voice of Reason: Essays in Objectivist Thought, Meridian.

ARNOLD BAISE *

Arnold Baise can be reached at abaise@usa.net.

(1) My website at http://mysite.verizon.net/abaise contains name and subject indexes as well as an errata file for Jaynes's book.
COPYRIGHT 2011 Ludwig von Mises Institute
No portion of this article can be reproduced without the express written permission from the copyright holder.