ECSE-2500, Engineering Probability, Spring 2010, Rensselaer Polytechnic Institute

Lecture 3

Starting on page 28.

iClicker exercises: P={2,3,5,7}, Q={2,4,6,8}.
1. What is P{$\cap Q$}?
  - A: {3,5,7}
  - B: {4,6,8}
  - C: {2}
  - D: {2,3,4,5,6,7,8}
  - E: {3,4,5,6,7,8}
2. What is P{$\cup Q$}?
3. What is A\B?
4. What is B\A?
Note that this proof of de Morgan uses the fact that A equals B iff A is a subset of B and B is a subset of A.
Equivalently, just look at the Venn diagram; there are only 4 cases.
Countable vs non countable infinite
1. A countably infinite set means you can put the its elements in 1-1 correspondance with the natural numbers; you can count them.
2. E.g., integers, rational numbers
3. Uncountably infinite means you cannot.
4. E.g., real numbers.
5. That was proved in the 19 century.
6. Don't worry about this for this course.
2.1.4 Event classes
1. This is a technical detail I'm not going to worry about in this course.
2. The problem occurs when the outcome is a real number, say a voltage.
3. There are an uncountably infinite number of real numbers, even in a finite interval [0,1].
4. We cannot assign them probabilities; it just doesn't work.
5. It does work to use intervals of real numbers, like [.5,.6] and [.7,.8] and unions and complements of them.
6. This matches the real world. You can't measure a voltage as 3.14159265...; you measure it in the range [3.14,3.15].
7. {$\cal F$} class of events of interest: those sets of intervals.
2.2 Axioms of probability
1. I: 0<=P[A}
2. II: P[S]=1
3. III: {$ A\cap B=\emptyset \rightarrow P[A\cup B] = P[A]+P[B] $}
4. III': For {$ A_1, A_2, .... $} if {$ \forall_{i\ne j} A_i \cap A_j = \emptyset$} then {$P[\bigcup_{i=1}^\infty A_i] = \sum_{i=1}^\infty P[A_i] $}
Example: cards. Q=event that card is queen, H=event that card is heart. These events are not disjoint. Probabilities do not sum.
1. {$Q\cap H \ne\emptyset $}
2. P[Q] = 1/13=4/52, P[H] = 1/4=13/52, P[Q {$\cup$} H] = 16/52!=17/52.
Example C=event that card is clubs. H and C are disjoint. Probabilities do sum.
1. {$C\cap H \ne\emptyset $}
2. P[C] = 13/52, P[H] = 1/4=13/52, P[Q {$\cup$} H] = 26/52.
Example. Flip a fair coin {$A_i$} is the event that the first time you see heads is the i-th time, for {$i\ge1$}.
1. We can assign probabilities to these countably infinite number of events.
2. {$ P[A_i] = 1/2^i $}
3. They are disjoint, so probabilities sum.
4. Probability that the first head occurs in the 10th or later toss = {$ \sum_{i=10}^\infty 1/2^i $}
Corollory 1
1. {$ P[A^c] = 1-P[A] $}
2. E.g., P[heart] = 1/4, so P[not heart] = 3/4
Corollory 2: P[A] <=1
Corollory 3: P[{$ \emptyset $}] = 0
Corollory 4:
1. For {$ A_1, A_2, .... A_n $} if {$ \forall_{i\ne j} A_i \cap A_j = \emptyset$} then {$P\left[\bigcup_{i=1}^n A_i\right] = \sum_{i=1}^n P[A_i] $}
2. Proof by induction from axiom III.
Corollory 5: {$ P[A\cup B] = P[A] + P[B] - P[A\cap B] $}
1. Proof from the fact that {$ P[A], P[B], P[A\cap B] $} are disjoint.
2. Example: Queens and hearts. P[Q]=4/52, P[H]=13/52, P[Q {$\cup$} H]=16/52, P[Q {$\cap$} H]=1/52.
3. {$ P[A\cup B] \le P[A] + P[B] $}
Corollory 6: {$P\left[\cup_{i=1}^n A_i\right] = \sum_{i=1}^n P[A_i] - \sum_{i<j} P[A_i\cap A_j] + \sum_{i<j<k} P[A_i\cap A_j\cap A_k] \cdots + (-1)^{n+1} P[\cap_{i=1}^n A_i] $}
1. Example Q=queen card, H=heart, F=face card
  1. P[Q]=4/52, P[H]=13/52, P[F]=12/52,
  2. P[Q {$\cap$} H]=1/52, P[Q {$\cap$} F] = you tell me
  3. P[H {$\cap$} F]= you tell me
  4. P[Q {$\cap$} H {$\cap$} F] = you tell me
  5. So P[Q {$\cup$} H {$\cup$} F] = ?
2. Example from [([http://en.wikipedia.org/wiki/)Roulette]]:
  1. R=red, B=black, E=even, A=1-12
  2. P[R] = P[B] = P[E] = 16/38. P[A]=12/38
  3. {$ P[R\cup E \cup A] $} = ?
  4. You could try at http://www.roulette4fun.com/online_game.php but it's slow.
Corollory 7: if {$A\subset B$} then P[A] <= P[B]
Example: Probability of a repeated coin toss having its first head in the 2nd-4th toss (1/2+1/4+1/8) <= Prob of it happening in the 3rd toss (1/4).
2.2.1 Discrete sample space
1. If sample space is finite, probabilities of all the outcomes tell you everything.
2. sometimes they're all equal.
3. Then P[event] = (# outcomes in event) / (total # outcomes)
4. For countably infinite sample space, probabilities of all the outcomes also tell you everything.
5. E.g. fair coin. P[even] = 1/2
6. E.g. example 2.9. Try numbers from http://random.org/.
7. What probabilities to assign to outcomes is a good question.
8. Example 2.10. Toss coin 3 times.
  1. Choice 1: outcomes are TTT ... HHH, each w.p. 1/8
  2. Choice 2: outcomes are # heads: 0...3, each w.p. 1/4.
  3. Incompatible. What are probabilities of # heads for choice 1?
  4. Which is correct? Why?
Example 2.11: countably infinite sample space.
1. Toss fair coin, outcome is # tosses until 1st head.
2. What are reasonable probabilities?
3. Do they sum to 1?
2.2.2 Continuous sample spaces
1. Can't assign probabilities to points on real line. (It just doesn't work.)
2. Work with set of intervals, and Boolean operations on them.
3. Set may be finite or countable.
4. This set of events is a Borel set.
5. Notation:
  1. [a,b] closed. includes both. a<=x<=b
  2. (a,b) open. includes neither. a<x<b
  3. [a,b) includes a but not b, a<=x<b
  4. (a,b] includes b but not a, a<x<=b
6. Assign probabilities to intervals (open or closed).
7. E.g., uniform distribution on [0,1] P[a<=x<=b] = 1/(b-a).
8. Nonuniform distributions are common.