# BSP - Random Variables

## Discrete Random Variables

Imagine two sets, A and B. 

There is A+B, which the set of elements in A or B,
and $AB = A \union B$, which is the elements in A and B.

Suppose our set S is {1, 2, 3, 4, 5, 6} where A is the evens,
and B is less than 5. AB is {2, 4}, while A+B is {1, 2, 3,
4, 6}.

Sometimes  -A is known as NOT A, or $\bar A$.
$\bar S = {}$.

## Probability and Random Variables
Set $\Omega$ is the set of all possible outcomes of an
experiment.

Events of some subset of outcomes are called 
$\omega = {odd, even}$

A trial is a single performance of an experiment (single
sample).

An experiment is to observe an single outcome $\zeta_i$.

Experiment E means a set S of outcomes $\zeta$, certain
subsets of S can be considered events.

The space S is the certain event. Basically if it's in S, it
happened. The null space is events that are impossible. E.g.
can't get a 7 on a six-sided die.

If event $\zeta_i$ consists of a single outcome then it is
an elementary event. 

Events A and B are mutually exclusive if they have no common
elements (AB = null space)

**Defining exactly what the events and outcomes are is super
important**. It's very easy to get paradoxical results if
careful choice of outcome and event is not chosen.

### Example

Assign each event a number $A \rightarrow P(A)$

Axioms:
1. $P(A) \geq 0$
2. $P(S) = 1$
3. if $AB = 0$ then $P(A+B) = P(A) + P(B)$

Corollaries:
1. $P(0) = 0$
2. $P(A) = 1 - P(\bar A) \leq 1$
*fundamentally the probability of an event is some number
between 1 and 0*
3. If $A$ and $B$ are not mutually exclusive, then 
$P(A+B) = P(A) + P(B) - P(AB) \leq P(A) + P(B)$
4. If $B \subset A \rightarrow P(A) = P(B) + P(\bar A B)
   \geq P(B)$

## Conditional Probability
Given an event $B: P(B)\geq 0$, we define conditional
probability as $P(A|B) = \frac{P(AB)}{P(B)}$.

Sometimes people write *joint probability* as $P(A,B)$ which
is exactly the same as $P(AB)$.

Given $n$ mutually exclusive events $A_i,...,A_n$ where $A_1
+ A_2 + ... + A_n = S$. For an arbitrary event $B$, 

$P(B) = \sum_i P(A_i, B)$
$P(B) = \sum_i P(B | A_i) P(A_i)$

We know $B = BS = B(A_1+...+A_n)$. With some rearranging
knowing each of the A's are independent, $BA_i$ and $BA_j$
are independent for all $i\neq j$. 

Thus, $P(B) = P(BA_1) + ... + P(BA_n)$ but we showed earlier
there's a different way to find $P(B)$.

if $B = S$, $P(S|A_i) = 1$, then $P(S) = 1 = \sum_i P(A_i)$
if $B = A_i$, $P(A_j|A_i) = 1$ when i = j, 0 otherwise


> [!important] Development of Bayes' Theorem
>
> Bayesian statistics are a way of thinking about a **degree
> of belief** in a probability, not an estimation of the
> probability from a number of experiments.
> 
> We know:
>
> $P(AB) = P(A|B)P(B) = P(B|A)P(A)$
>
> Then Bayes Theorem becomes
> $$P(A|B) = \frac{P(B|A) P(A)}{P(B)}$$
>
> and then when events $A_i$ are mutually exclusive...
>
> $$P(A_i|B) = \frac{P(B|A_i) P(A_i)}{\sum_i P(B|A_i) P(A_i)}$$
>