This post, marks the official beginning of the series, where we will try to move forward first slowly, trying to build basics into simple probability theory, to more advanced concepts.

Now, lets get started like this. Consider the term, variable. What does it mean? Something which can vary. Lets try to finetune it a bit more, can it vary within bounds? We can’t necessarily say that.

What about, can we impose some limits on its behaviour? Depends on what kind of limit we are talking about.

Okay, so if we consider the equation $ax=b$, where $a,b$ are constants, then, $x$ will be a simple deterministic variable. Given $a,b$ we do know how $x$ will vary. Nice and good. We in our daily lives used to seeing and working with deterministic variables.
Now this set of variables have got more snobby cousins, called random variables. What are random variables?

Practically just think of an experiment, where there are n outcomes possible. And we assign a value to those outcomes, say 1,2,3….n. It doesnt mean a weight or something like,it just is a marker of the outcome which has come out. You can even call it, by the names of your kids, or your girlfriends. Now, if I ask, what is the outcome of the experiment, we dont know about it, accurately. Because, each time we repeat the experiment any one of them can pop up randomly. Hence we assign the result of the outcome as X and say, X is a random variable and can take values from 1,2,3…n

So here the values which it can take, is called sample space.So the outcome of the experiment of tossing a coin, or throwing a dice will have a sample space as {H,T} or {1,2,3…6}.

Plain and simple eh?

Now lets try to crank the quotient a bit.

Lets consider we have a signal source, an information source. Randomly it switches itself on, transmits something for some time period 0 to T, the information is not known before hand*, and switches itself off.

So because what is being transmitted is unknown,  and each time it switches itself on, is considered a process. Now the signal which is being transmitted, its contents (or information content we dont know and can’t predict).

In these cases, to model the situation  at hand, we deduce something, we assume a few things, and then try to forward our journey. As a sidenote, most of my education, I have tried, (not necessarily succeeded) to connect the dots which are coming towards me with the path I have traversed till now. So for example you know linear algebra, you will try to work with say equations, then slowly move to quadratic equations, all the while trying to form relationships between what you have studied, and trying to solve the challenge at hand by what you already know. Nothing hi-fi, simple plain things.

So now, we want to come back to our signal source, and want to study its characteristics. We notice a few things, in the signal source, the signal source, can generate anything, and can take any of the probable ways. Not all are equiprobable, not necessarily. So such kind of a setup, a system, a non deterministic situation we call as a stochastic process.

What is stochastic process?

Lets add some rigour to it, we dont know how it will evolve into future, it will have multiple paths of evolution, not all equiprobable, yet given the initial state, we still wont be able to deduce where it will go. (it reminds me as I write, of chaotic systems)

Now, consider Anna is talking with her best friend, Bethany. They are discussing some very juicy details of gossips, which the other counterparty doesnt know. And as happens, Anna doesnt know what Bethany is going to talk about, or how will the conversation evolve. So if you just consider that Anna doesnt talk at all (or rather, Anna’s talks doesnt disturb Bethany’s rantings and ravings) then Bethany will be transmitting an effectively stochastic signal. Now if I assume the same thing, being repeated multiple number of times, ad infinitum we have a stochastic process. Mighty boring for Anna, and Beth as well, but its interesting to note, that information transmission is considered a blue blood stochastic process. Had Anne and Beth both would have talked, and responded to each other they would have been a process with its information content dependent on each other.

So mathematically, given a probability space(“experiment”) $(\Omega,F,P)$ , a stochastic process if sampled at any time period $t$ which lies between 0 and T,then we will have a randomvariable $X_{t}$ whose sample space is  each individual value in $F_{t}$.

So that means,you sample a stochastic process over its numerous trials at time ‘t’, what we get is a random variable with its sample space as those value of sampled stochastic process.

Sampling a stochastic process yields a random variable

Now, I have in this figure, tried showing visually “driftless”(non trending) signals.Consider for example I have some finite drift involved, in the process. Just for kicks.

That just means, we have at some point of time, a series which keeps on growing large. Now, though we don’t have anything against trends, but consider if I then sample a time unit in neighbourhood of those instances where it goes arbitrarily large, what I will get is a random variable with some possible outcomes as very large. i.e I have some considerable chances of getting “fat tails” whhoops!!!

Take care,

Soham

* Information is defined as the randomness or entropy in a signal. The less we  know, about a particular signal, the more is the information content.

October 17, 2009 6:51 am

happy diwali soham…the quant approach is a rare breed in india ..havent seen many spaces out here which document it in a practical way ..i hope the space keeps rocking for time to come..

• October 17, 2009 11:16 pm

Thanks, Saif. Just my \$0.02 contribution. Between us: Personally, I am not too satisfied with these two posts, hence, expect them to go significant change in content as well rigour and length in the coming posts.

• October 17, 2009 11:17 pm

By the way, Aapko bhi diwali ki badhai ho…

On 18/10/2009, Soham Das wrote: > > Thanks, Saif. > Just my \$0.02 contribution. Between us: Personally, I am not too satisfied > with these two posts, hence, expect them to go significant change in content > as well rigour and length in the coming posts. > > >