Adelsteen Normann (1848-1918) was a Norwegian painter.
We have sections
\begin{equation}{\label{a}}\tag{A}\mbox{}\end{equation}
Sigma-Field.
The concept of \(\sigma\)-fields is important for probability measure and stochastic analysis.
Definition. Let \(S\) be a sample space. A class \({\cal F}\) of subsets of \(S\) is said to be a \(\sigma\)-field when the following conditions are satisfied:
- \({\cal F}\) is a nonempty class.
- \(A\in {\cal F}\) implies \(A^{c}\in {\cal F}\) (i.e., \({\cal F}\) is closed under complementation).
- Given \(A_{i}\in {\cal F}\) for \(i=1,2,\cdots\), we have \(\bigcup_{i=1}^{\infty}A_{i}\in {\cal F}\) (i.e., \({\cal F}\) is closed under countable unions). \(\sharp\)
Given \(A_{i}\in {\cal F}\) for \(i=1,2,\cdots\), we can show \(\bigcap_{i=1}^{\infty}A_{i}\in {\cal F}\) (i.e., \({\cal F}\) is closed under countable intersections).
Example. The following three \({\cal F}_{i}\), \(i=1,2,3\), are \(\sigma\)-fileds:
- \({\cal F}_{1}=\{\emptyset ,S\}\);
- \({\cal F}_{2}\) consists of all subsets of \(S\);
- \({\cal F}_{3}=\{\emptyset ,A,A^{c},S\}\) for some \(A\subset S\). \(\sharp\)
Let \({\cal C}\) be an arbitrary class of subsets of \(S\). We define
\[\sigma ({\cal C})=\bigcap\left\{\mbox{all \(\sigma\)-fields containing \({\cal C}\)}\right\}.\]
Then \(\sigma ({\cal C})\) is a \(\sigma\)-field. We also have the following result.
Theorem. Let \({\cal C}\) be an arbitrary class of subsets of \(S\). Then \(\sigma ({\cal C})\) is a unique minimal \(\sigma\)-filed containing \({\cal C}\). In this case, we also say that \(\sigma ({\cal C})\) is a \(\sigma\)-field generated by \({\cal C}\). \(\sharp\)
Now we introduce the Borel $\sigma$-field. Let \({\cal C}_{0}\) be defined as follows:
\[{\cal C}_{0}=\left\{\mbox{all intervals in \(\mathbb{R}\)}\right\}.\]
Then \(\sigma ({\cal C}_{0})\) is called the Borel \(\sigma\)-field, and it is also denoted by \({\cal B}\equiv\sigma ({\cal C}_{0})\). In other words, the Borel \(\sigma\)-field is a unique minimal \(\sigma\)-filed containing \({\cal C}_{0}\).
Theorem. Each of the following classes generates the Borel \(\sigma\)-field.
\begin{align*} {\cal C}_{1} & =\{(x,y]:x,y\in\mathbb{R}, x<y\}\\ {\cal C}_{2} & =\{[x,y):x,y\in \mathbb{R}, x<y\}\\
{\cal C}_{3} & =\{[x,y]:x,y\in \mathbb{R}, x<y\}\\ {\cal C}_{4} & =\{(x,y):x,y\in \mathbb{R}, x<y\}\\
{\cal C}_{5} & =\{(x,\infty ):x\in \mathbb{R\}}\\ {\cal C}_{6} & =\{[x,\infty ):x\in \mathbb{R}\}\\
{\cal C}_{7} & =\{(\infty ,x):x\in \mathbb{R}\}\\ {\cal C}_{8} & =\{(\infty ,x]:x\in \mathbb{R}\}.\end{align*}
Also. the classes \({\cal C}’_{i}\) for \(i=1,\cdots ,8\) generate the Borel \(\sigma\)-field, where \({\cal C}’_{i}\) is defined the same way as \({\cal C}_{i}\) except that \(x\) and \(y\) are restricted to the rational numbers. \(\sharp\)
Let \(S=\mathbb{R}^{2}\) and define \({\cal C}_{0}\) as follows
\[{\cal C}_{0}=\left\{\mbox{all rectangles in \(\mathbb{R}^{2}\)}\right\}.\]
The \(\sigma\)-field generated by \({\cal C}_{0}\) is denoted by \({\cal B}^{2}\). Similarly, let \(S=\mathbb{R}^{n}\) and define \({\cal C}_{0}\) as follows
\[{\cal C}_{0}=\left\{\mbox{all \(n\)-dimensional rectangles in \(\mathbb{R}^{n}\)}\right\}.\]
The \(\sigma\)-field generated by \({\cal C}_{0}\) is denoted by \({\cal B}^{n}\).
\begin{equation}{\label{b}}\tag{B}\mbox{}\end{equation}
Probability Measures.
The probability of event \(A\), denoted by \(\mathbb{P}(A)\), is often called the chance of \(A\) occurring. A function such as \(\mathbb{P}(A)\), which is evaluated for a set \(A\), is called
a set function. Probability is a set function \(\mathbb{P}\) that assigns to each event \(A\) in the sample space \(S\) a number \(\mathbb{P}(A)\), called the probability of the event \(A\), and satisfy some axioms.
Definition. Let \(S\) be a sample space with \(\sigma\)-field \({\cal F}\). A probability measure \(\mathbb{P}\) is a set function \(\mathbb{P}:{\cal F}\rightarrow [0,1]\) such that the following conditions are satisfied.
- \(\mathbb{P}\) is nonnegative, i.e., \(\mathbb{P}(A)\geq 0\) for \(A\in {\cal F}\).
- We have \(\mathbb{P}(S)=1\).
- \(\mathbb{P}\) is \(\sigma\)-additive. In other words, given any collection of pairwise disjoint events \(A_{i}\) for \(i=1,2,\cdots\) (i.e., \(A_{i}\cap A_{j}=\emptyset\) for \(i\neq j\)), we have \[\mathbb{P}\left (\bigcup_{i=1}^{\infty}A_{i}\right )=\sum_{i=1}^{\infty}\mathbb{P}(A_{i}).\]
This is the axiomatic (Kolmogorov) definition of probability. The triple \((S,{\cal F},\mathbb{P})\) is known as a probability space.
Theorem. (Additive Theorem). For any finite number of events, we have
\begin{align*} \mathbb{P}\left (\bigcup_{j=1}^{n} A_{j}\right ) & =\sum_{j=1}^{n} \mathbb{P}(A_{j})-\sum_{1\leq j_{1}<j_{2}\leq n}\mathbb{P}(A_{j_{1}}\cap A_{j_{2}})\\ & +
\sum_{1\leq j_{1}<j_{2}<j_{3}\leq n} \mathbb{P}(A_{j_{1}}\cap A_{j_{2}}\cap A_{j_{3}})-\cdots\\ & +(-1)^{n+1}\mathbb{P}(A_{1}\cap A_{2}\cap\cdots\cap A_{n}).\end{align*}
The sequence of sets \(\{A_{n}\}_{n=1}^{\infty}\) is said to be an increasing sequence if \(A_{1}\subseteq A_{2}\subseteq\cdots\), which is also denoted by \(A_{n}\uparrow\), and is said to be a decreasing sequence if \(A_{1}\supseteq A_{2}\supseteq\cdots\), which is also denoted by \(A_{n}\downarrow\). The limit of \(\{A_{n}\}_{n=1}^{\infty}\) is defined as follows.
- For \(A_{n}\uparrow\), we define \[\lim_{n\rightarrow\infty}A_{n}=\bigcup_{n=1}^{\infty}A_{n}.\]
- For \(A_{n}\downarrow\), we define \[\lim_{n\rightarrow\infty}A_{n}=\bigcap_{n=1}^{\infty}A_{n}.\]
Then, we have the following interesting result.
Theorem. Let \(\{A_{n}\}_{n=1}^{\infty}\) be a sequence sets with \(A_{n}\uparrow\) or \(A_{n}\downarrow\). Then, we have
\[\mathbb{P}\left (\lim_{n\rightarrow\infty}A_{n}\right )=\lim_{n\rightarrow\infty}\mathbb{P}(A_{n}).\]
The conditional probability of an event \(A\) given that event \(B\) has occurred is defined by
\[\mathbb{P}(A|B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)},\]
provided that \(\mathbb{P}(B)>0\). We can think of the “given event B” as specifying the new sample space for which we now want to calculate the probability of that part of \(A\) that is contained in \(B\) to determine \(\mathbb{P}(A|B)\).
Theorem. The set function \(\mathbb{P}(\cdot |B)\) is a probability measure.
Proof. We are going to show that \(\mathbb{P}(\cdot |B)\) satisfies the axioms of probability measure. It is cleraly that \(\mathbb{P}(A|B)\geq 0\) for every \(A\in {\cal F}\). We also have
\begin{align*} \mathbb{P}(S|B) & =\frac{\mathbb{P}(S\cap B)}{\mathbb{P}(B)}\\ & =\frac{\mathbb{P}(B)}{\mathbb{P}(B)}=1.\end{align*}
Given \(A_{i}\in {\cal F}\) for \(i=1,2,\cdots\) and \(A_{i}\cap A_{j}=\emptyset\) for \(i\neq j\), we have
\begin{align*} P\left (\left .\bigcup_{i=1}^{\infty}A_{i}\right |B\right ) & =\frac{P\left ((\bigcup_{i=1}^{\infty}A_{i})\cap B\right )}{\mathbb{P}(B)}\\ & =\frac{P\left (\bigcup_{i=1}^{\infty}(A_{i}\cap B)\right )}{\mathbb{P}(B)}\\ & =\frac{1}{\mathbb{P}(B)}\sum_{i=1}^{\infty}\mathbb{P}(A_{i}\cap B)\\ & =\sum_{i=1}^{\infty}\frac{\mathbb{P}(A_{i}\cap B)}{\mathbb{P}(B)}\\ & = \sum_{i=1}^{\infty}\mathbb{P}(A_{i}|B).\end{align*}
This completes the proof. \(\blacksquare\)
Let \(B_{1},B_{2},\cdots ,B_{m}\) constitute a partition of the sample space \(S\). That is, \(S=B_{1}\cup B_{2}\cup\cdots\cup B_{m}\) and \(B_{i}\cap B_{j}=\emptyset\) for \(i\neq j\). Of course, the events \(B_{1},B_{2},\cdots ,B_{m}\) are mutually exclusive and exhaustive. Furthermore, suppose that the prior probability of the event \(B_{i}\) is positive. Let \(A\) be an event. Then \(A\) is the union of \(m\) mutually exclusive events, namely,
\[A=(B_{1}\cap A)\cup (B_{2}\cap A)\cup\cdots\cup (B_{m}\cap A).\]
Therefore, we have
\begin{align*} \mathbb{P}(A) & =\sum_{i=1}^{m} \mathbb{P}(B_{i}\cap A)\\ & =\sum_{i=1}^{m} \mathbb{P}(B_{i})\mathbb{P}(A|B_{i}).\end{align*}
Theorem. (Total Probability Theorem). Let \(B_{i}\) for \(i=1,2,\cdots\) be a partition of \(S\) with \(\mathbb{P}(B_{i})>0\) for all \(i\). Then, for \(A\in {\cal F}\), we have
\[\mathbb{P}(A)=\sum_{i=1}^{\infty} \mathbb{P}(B_{i})\mathbb{P}(A|B_{i}).\]
Theorem. (Bayes’ Formula). Let \(B_{i}\), \(i=1,2,\cdots\), be a partition of \(S\) with \(\mathbb{P}(B_{i})>0\) for all \(i\). Then, we have
\[\mathbb{P}(B_{k}|A)=\frac{\mathbb{P}(B_{k})\mathbb{P}(A|B_{k})}{\sum_{i=1}^{\infty}\mathbb{P}(B_{i})\mathbb{P}(A|B_{i})}\]
for \(k=1,2\cdots\).
Example. A multiple choice test question lists five alternative answer, of which only one is correct. If a student has done his homework, then he is certain to identify the correct answer; otherwise he chooses an answer at random. Let \(p\) denote the probability of the event \(A\) that the student does his homework and let \(B\) be the event that he answer the question
correctly. Find the expression of the conditional probability \(\mathbb{P}(A|B)\) in terms of \(p\). By noting that \(A\) and \(A^{c}\) form a partition of the appropriate sample space, we have
\begin{align*} \mathbb{P}(A|B) & =\frac{\mathbb{P}(B|A)\mathbb{P}(A)}{\mathbb{P}(B|A)\mathbb{P}(A)+\mathbb{P}(B|A^{c})\mathbb{P}(A^{c})}\\ & =\frac{1\cdot p}{1\cdot p+\frac{1}{5}(1-p)}\\ & =\frac{5p}{4p+1}.\end{align*}
Furthermore, it is easily seen that \(\mathbb{P}(A|B)=\mathbb{P}(A)\) if and only if \(p=0\) or \(1\). \(\sharp\)
