:: Collaborations ::

The purpose of this article is to highlight some of the concepts of probability theory that frequently crop up in its applications to mathematical finance. Although merely abstract ideas they are key to attaining a full understanding of randomness and stochastic processes. It is expected that most readers will have encountered the idea of applied probability is some form of another. But this branch of probability (called applied probability because it refers to real life problems) most often omits the rigorous background from which it emerges, simply because this rigor is not needed in order to apply the results it generates. However, when dealing with stochastic processes, consideration of the underlying theory becomes almost a pre-requisite.

As good place to start probability theory is with a revision of basic set theory. The material in this article assumes full knowledge of the material from a previous article on Set theory.

Consider the toss of a fair coin. Since there are 2 possible outcomes, and they are assumed to have an equal chance of occurring, the probability of observing a head (or tail) is 1/2. The randomness in this experiment arises from the uncertainty towards this outcome of the toss of the coin. Now, if two coins are considered, immediately the experiment increases in complexity as there are more permutations and combinations that can be used to define the outcomes. In particular the possible (equally likely) outcomes are now HH, HT, TT and TT.

The outcomes from the two-coin experiment can be represented by a set:

corresponding to the outcomes 2 heads, a head followed by a tail, a tail followed by a head and 2 tails. In the study of probability theory it is conventional to use to denote the set of all possible outcomes from an experiment. This leads to the first definition:

Definition 1:
Sample space - The set of all possible outcomes from an experiment is called the sample space (denoted by ). Each time the experiment is performed, it can be stated with total confidence that one and only one of the elements of the sample space will be observed.

In the context of the example, the last statement makes intuitive sense. When 2 coins are tosses only one of the members of  will be observed. It is impossible to observe TT and HH, although the possibility of observing TT or HH does hold some meaning. The sample space strictly concerns the set outcomes that are possible and it is thus very specific.

Next consider the field generated by the sample space:

Although it may appear that the field is merely a re-combination of the elements of the sample space, it is in fact a set of events. Recall how  was a set of outcomes; F is now a set of events. These are two different but very closely associated concepts. All outcomes are events, however, not all events are outcomes. This has already been touched upon above, when the definition of a sample space was emphasized. An outcome is a definite result from an experiment. Hence, each of the elements of the sample space are outcomes, as any ONE of them will be observed at the end of the experiment. The set {HH, HT} on the other hand is not an outcome. It corresponds to the union of 2 outcomes. It is the set used to describe the scenario where an experiment yields either a double heads or a head followed by a tail. This is classified as an event. Likewise, the set {HH, HT, TH} can be used to define the event of not observing a double tails. It is still not an outcome, because it is not specific enough.

Similarly, the event labeled ‘’ is used to denote the event ‘nothing happens’. Of course, in this context this does not make any sense (you can never expect to toss a coin and not get a result). However, the empty set is included for completeness since this case is still an event no matter how unlikely it is; it is in any case made redundant via the probability measure (see below). The event  corresponds to the other extreme, the event that you ‘observe something’. In this example, this event is a certainty (there is a sure chance of observing one of the outcomes if a coin is tossed!). Having defined the concept of sample space and field, the next step is to assign probabilities to the various outcomes (this in turn assigns probabilities to the events also since they are formed by the combination of outcomes). This is called assigning a ‘probability measure’.

Definition 2:
Probability Measure: The probability measure is a function (map) that attaches each outcome a value to denote the probability of that outcome being observed.

Hence, the mapping is restricted to the range [0, 1] since probabilities are by definition enclosed within this range. However, it is not enough for each outcome to be in this range, but the sum over all outcomes (or events) must be bounded by one. This ensures that the probability measure of any event is also in this range. This results in the probability measure (denoted by P( )) having to satisfy the following properties:

The probability of the empty set is zero (i.e. this event can not happen): P() = 0
The probability of one of the outcomes (i.e. any one of them) being observed is 1: P() = 1.
P(A B) = P(A) + P(B), provided the events A and B are mutually exclusive (see standard text on probability); and A and B are two events.

Definition 3:
Probability Space: A sample space (), which generates a field (F) and on which a probability measure (P) is induced are collectively called a probability space. The representation (, F, P) is used frequently in literature.

The probability space is thus a neat and concise means of summarizing the information associated with a random outcome.

Definition 4:
Random Variable: A mapping from the set  to the set of real numbers is a random variable. Hence, even though the underlying randomness is in the coin tosses, a random variable attaches numerical values to the results of the coin tosses.

Example 1:
Consider the probability space induced by a single toss of a fair coin, where = {H, T}, F= {, H, T, } and P = 1/2 for a H, P = 1/2 for a T. A function which attaches a value 1 to a H and -1 to a T is a random variable.

Example 2:
Now consider the probability space induced by tossing two fair coins. The function X which assigns a score of 1 for each H and 0 for each T observed is again a random variable. It can take values 0, 1 or 4 depending on the outcome of the underlying randomness in (, F, P).

Note that a random variable is a function such that each outcome is given a numerical value. Furthermore, each outcome is only assigned one numerical value. It does not make sense to introduce a function that gives a particular outcome two numerical values. On the other hand, it is possible for several outcomes to be assigned the same numerical value. These points were encountered in example 2:

Each outcome was assigned only one numerical value via the random variable:

{HH}: 4
{HT}: 1
{TH}: 1
{TT}: 0

However, the value ‘1’ is associated with 2 outcomes. Thus, given the random variable that was observed from an experiment, it is not always possible to deduce exactly which outcome was observed. In the above example, if the information available was that the random variable outcome was 1, then this is not enough information to deduce what the actual outcome was. It was either HT or TH. This fact is not of much importance in this article but leads to deeper results and a full understanding of stochastic processes will require one to start considering these issues.  

The importance of random variables is obvious. The H/T outcomes do not have much use alone. There must be something to convert them into a numerical value to, which will then represent the numerical stochastic variable (e.g. stock price) that is of interest. For e.g. if one were interested in modeling a stochastic stock price, then there must be something to link the randomness (i.e. the random head/tail outcomes) with the numerical values (i.e. prices) that will be observed in the market. Also note that a coin is not the only model that can be used for this (there is nothing to stop the use of a different model for the randomness. This is discussed below. For now consider the coin tossing model to represent the underlying randomness).

With this, the first section of this article is completed. Arriving at the notion of a stochastic process is only a stone throw away and is merely a combination of the ideas that have been presented thus far.

Consider the coin tossing experiment again. However, instead of tossing two (or more) coins, the experiment is changed, so that a single coin is tossed 10 times successively. After each toss, the outcome observed is noted. This example will introduce the concept of a stochastic process. Like the single toss case, each successive toss has an associated probability space, and it will be assumed here that these spaces are identical.

For each toss, = ( , , )

= {H, T}, = {, H, T, }

This is actually a VERY incomplete picture of what is happening, but is enough to introduce the concept of a stochastic process. The concepts of ‘filtration’, ‘filtered space’, ‘adaptability’ and ‘fields generated by random variables’ should be introduced in order to gain a full understanding of stochastic processes, but these will be omitted to keep things simple.

Now if each probability space is attached with a random variable, the result is a Stochastic Process. For instance, if the random variable in example 1 is applied to each probability space then the variable

=

is a stochastic process. To make things more intuitive imagine that the coin is tossed every second, and hence Y changes every second. It changes randomly to reflect the outcome of the random observation from the coin. This is now a stochastic process through time. From this point of view, a stochastic process is nothing more than successive random variables on some underlying randomness. Here the underlying source of randomness was an equal probability, up versus down move (it does not have to involve a coin; this was only used to enhance understanding). This underlying source of randomness was converted to a numerical value (which we can use mathematically) via the random variable.

Hence, the situation is this:

Something random happens. The outcome from this randomness (modeled here by the coin tossing experiment) manifests itself numerically as a random variable . As soon as this occurs, the value of Y can be updated. Since Y cannot be updated in advance (i.e. it can only be updated when the results of the randomness are known) this is a stochastic process.

Of course, there is no need to use a coin tossing model for the underlying randomness. Why not assume that the there are 6 possible outcomes and hence 6 different values for the random variable (i.e. the underlying randomness is characterized by the results of a ‘dice’ throwing experiment). Take this further: what if there were an infinite number of outcomes? This now begins to tend to a continuous stochastic process as opposed to the discrete picture that has been built in this article.

Consider a financial example.

The underlying randomness can be represented by the outcomes from a fair dice; this will denote the probability space. The random variable is say, which is assigned as follows:

= 4 if dice shows a one
= 2 if dice shows a two
= 0 if dice shows a three or four
= -2 if dice shows a five
=-4 if dice shows a six

And define the stochastic process as

for some constant and n = 1, 2…

This can be used to model the movement of any stock price in the short term. The constant is the initial price (say today) and is the price n days later (i.e. n time units). Notice that for any n > 0 it is impossible to deduce at time zero due to the uncertainty associated with . All that is possible is to consider the different scenarios that may emerge for under this model. For instance, if were 100, and the stock price tomorrow was of interest, then under this model there are 5 possibilities: {104, 102, 100, 98, 96}. It is impossible to be sure which will be observed but each value in the set can be presented with a probability. That is the best that can be done. That is a stochastic process.

On a separate note, this is a reasonable, albeit extremely over-simplified model for a stock price movement. It models the fact that a stock price can move up/down or stay where it is over any 1 unit period. It also accounts for the fact that the stock price can stay stationary. It is perhaps not at all realistic (since the possible set for stock price movements are potentially huge is real life; but remember that this is just a model. Perhaps the user is only interested in approximations and does not require that kind of sophistication).

Now, a comment about the modeling of the underlying randomness. The model used here was that of a fair dice. However, this does not mean that there is someone sitting somewhere, rolling a die and using the outcomes to determine the prices in the stock market! It is merely a model that the user (i.e. the person interested in understanding the dynamics of a stock price) chooses arbitrarily depending on how he/she thinks stock prices move. A popular model is the binomial model, where the underlying randomness is taken to be an up/down movement (like example 1), but where the user specifies the up/down probabilities (i.e. an unfair coin!) to match his view of how stock prices change. But this is all it is - somebody’s view. By far the most popular model is Brownian motion/Wiener process. This is much too complicated to be given more than a mention here due to the fact that it is a continuous model. Everything discussed in this article was at the discrete stage. This is not to say that the continuous models are far superior and greatly over-mask their discrete counterparts. The continuous cases are much more rigorous and can lead to excellent results, but at a cost. They are by far much more mathematically demanding. The continuous cases are often approximated by discrete models (e.g. the Binomial model is very often used) simply to escape from the mathematical difficulties. The solutions they offer compare well with the continuous solutions.

In summary, this article has been a very gentle introduction to stochastic processes. It discussed the concept of an ‘underlying’ source of randomness, on which a random variable was induced. This random variable represents the stochastic process if there are several random variables that are combined together (through time or space; something can vary randomly through time (e.g. stocks) or space (e.g. pollen grains in water)). An important point about this article is the fact that everything was discussed at the discrete level. To move onto continuous stochastic processes one need to consider a few more concepts and definitions, but the general idea remains the same.

Written by Samy Mohammed.

Back for more