106 lines
2.9 KiB
Markdown
106 lines
2.9 KiB
Markdown
# BSP - Random Variables
|
|
|
|
## Discrete Random Variables
|
|
|
|
Imagine two sets, A and B.
|
|
|
|
There is A+B, which the set of elements in A or B,
|
|
and $AB = A \union B$, which is the elements in A and B.
|
|
|
|
Suppose our set S is {1, 2, 3, 4, 5, 6} where A is the evens,
|
|
and B is less than 5. AB is {2, 4}, while A+B is {1, 2, 3,
|
|
4, 6}.
|
|
|
|
Sometimes -A is known as NOT A, or $\bar A$.
|
|
$\bar S = {}$.
|
|
|
|
## Probability and Random Variables
|
|
Set $\Omega$ is the set of all possible outcomes of an
|
|
experiment.
|
|
|
|
Events of some subset of outcomes are called
|
|
$\omega = {odd, even}$
|
|
|
|
A trial is a single performance of an experiment (single
|
|
sample).
|
|
|
|
An experiment is to observe an single outcome $\zeta_i$.
|
|
|
|
Experiment E means a set S of outcomes $\zeta$, certain
|
|
subsets of S can be considered events.
|
|
|
|
The space S is the certain event. Basically if it's in S, it
|
|
happened. The null space is events that are impossible. E.g.
|
|
can't get a 7 on a six-sided die.
|
|
|
|
If event $\zeta_i$ consists of a single outcome then it is
|
|
an elementary event.
|
|
|
|
Events A and B are mutually exclusive if they have no common
|
|
elements (AB = null space)
|
|
|
|
**Defining exactly what the events and outcomes are is super
|
|
important**. It's very easy to get paradoxical results if
|
|
careful choice of outcome and event is not chosen.
|
|
|
|
### Example
|
|
|
|
Assign each event a number $A \rightarrow P(A)$
|
|
|
|
Axioms:
|
|
1. $P(A) \geq 0$
|
|
2. $P(S) = 1$
|
|
3. if $AB = 0$ then $P(A+B) = P(A) + P(B)$
|
|
|
|
Corollaries:
|
|
1. $P(0) = 0$
|
|
2. $P(A) = 1 - P(\bar A) \leq 1$
|
|
*fundamentally the probability of an event is some number
|
|
between 1 and 0*
|
|
3. If $A$ and $B$ are not mutually exclusive, then
|
|
$P(A+B) = P(A) + P(B) - P(AB) \leq P(A) + P(B)$
|
|
4. If $B \subset A \rightarrow P(A) = P(B) + P(\bar A B)
|
|
\geq P(B)$
|
|
|
|
## Conditional Probability
|
|
Given an event $B: P(B)\geq 0$, we define conditional
|
|
probability as $P(A|B) = \frac{P(AB)}{P(B)}$.
|
|
|
|
Sometimes people write *joint probability* as $P(A,B)$ which
|
|
is exactly the same as $P(AB)$.
|
|
|
|
Given $n$ mutually exclusive events $A_i,...,A_n$ where $A_1
|
|
+ A_2 + ... + A_n = S$. For an arbitrary event $B$,
|
|
|
|
$P(B) = \sum_i P(A_i, B)$
|
|
$P(B) = \sum_i P(B | A_i) P(A_i)$
|
|
|
|
We know $B = BS = B(A_1+...+A_n)$. With some rearranging
|
|
knowing each of the A's are independent, $BA_i$ and $BA_j$
|
|
are independent for all $i\neq j$.
|
|
|
|
Thus, $P(B) = P(BA_1) + ... + P(BA_n)$ but we showed earlier
|
|
there's a different way to find $P(B)$.
|
|
|
|
if $B = S$, $P(S|A_i) = 1$, then $P(S) = 1 = \sum_i P(A_i)$
|
|
if $B = A_i$, $P(A_j|A_i) = 1$ when i = j, 0 otherwise
|
|
|
|
|
|
> [!important] Development of Bayes' Theorem
|
|
>
|
|
> Bayesian statistics are a way of thinking about a **degree
|
|
> of belief** in a probability, not an estimation of the
|
|
> probability from a number of experiments.
|
|
>
|
|
> We know:
|
|
>
|
|
> $P(AB) = P(A|B)P(B) = P(B|A)P(A)$
|
|
>
|
|
> Then Bayes Theorem becomes
|
|
> $$P(A|B) = \frac{P(B|A) P(A)}{P(B)}$$
|
|
>
|
|
> and then when events $A_i$ are mutually exclusive...
|
|
>
|
|
> $$P(A_i|B) = \frac{P(B|A_i) P(A_i)}{\sum_i P(B|A_i) P(A_i)}$$
|
|
>
|