Engineering Probability Class 2 Mon 2019-01-14
Table of contents::
1 Review from last class
- Outcomes and events for die toss
- Outcome: 1 or 2 or ...
- Event: even, or prime, ...
- Venn diagram. Probabilities.
- Discrete outcome vs continuous:
- Weight of random student
- Number of cars in student union parking garage on random day
2 Chapter 1 ctd
-
Rossman-Chance coin toss applet demonstrates how the observed frequencies converge (slowly) to the theoretical probability.
-
Example of unreliable channel (page 12)
- Want to transmit a bit: 0, 1
- It arrives wrong with probability e, say 0.001
- Idea: transmit each bit 3 times and vote.
- 000 -> 0
- 001 -> 0
- 011 -> 1
- 3 bits arrive correct with probability \((1-e)^3\) = 0.997002999
- 1 error with probability \(3(1-e)^2e\) = 0.002994
- 2 errors with probability \(3(1-e)e^2\) = 0.000002997
- 3 errors with probability \(e^3\) = 0.000000001
- corrected bit is correct if 0 or 1 errors, with probability \((1-e)^3+3(1-e)^2e\) = 0.999996999
- We reduced probability of error by factor of 1000.
- Cost: triple the transmission plus a little logic HW.
-
Example of text compression (page 13)
- Simple way: Use 5 bits for each letter: A=00000, B=00001
- In English, 'E' common, 'Q' rare
- Use fewer bits for E than Q.
-
Morse code did this 170 years ago.
- E = .
- Q = _ _ . _
- Aside: An expert Morse coder is faster than texting.
- English can be compressed to about 1 bit per letter (with difficulty); 2 bits is easy.
- Aside: there is so much structure in English text, that if you add the bit strings for 2 different texts bit-by-bit, they can usually mostly be reconstructed.
- That's how cryptoanalysis works.
-
Example of reliable system design (page 13)
- Nuclear power plant fails if
- water leaks
- and operator asleep (a surprising number of disasters happen in the graveyard shift).
- and backup pump fails
- or was turned off for maintenance
- What's the probability of failure? This depends on the probabilities of the various failure modes. Those might be impossible to determine accurately.
- Design a better system? Coal mining kills.
- The backup procedures themselves can cause problems (and are almost impossible to test). A failure with the recovery procedure was part of the reason for a Skype outage.
- Nuclear power plant fails if
3 Chapter 2
- A random experiment (page 21) has 2 parts:
- experimental procedure
- set of measurements
- Random experiment may have subexperiments and sequences of experiments.
- Outcome or sample point \(\zeta\): a non-decomposable observation.
- Sample space S: set of all outcomes
-
\(|S|\):
- finite, e.g. {H,T}, or
- discrete = countable, e.g., 1,2,3,4,... Sometimes discrete includes finite. or
- uncountable, e.g., \(\Re\), aka continuous.
- Types of infinity:
- Some sets have finite size, e.g., 2 or 6.
- Other sets have infinite size.
- Those are either countable or uncountable.
- A countably infinite set can be arranged in order so that its elements can be numbered 1,2,3,...
- The set of natural numbers is obviously countable.
- The set of positive rational numbers between 0 and 1 is also countable. You can order it thus: \(\frac{1}{1}, \frac{1}{2}, \frac{1}{3}, \frac{2}{3}, \frac{1}{4}, \ \frac{3}{4}, \frac{1}{5}, \frac{2}{5}, \frac{3}{5}, \ \cdots\)
- The set of real numbers is not countable (aka uncountable). Proving this is beyond this course. (It uses something called diagonalization.
- Uncountably infinite is a bigger infinity than countably infinite, but that's beyond this course.
- Georg Cantor, who formulated this, was hospitalized in a mental health facility several times.
- Why is this relevant to probability?
- We can assign probabilities to discrete outcomes, but not to individual continuous outcomes.
- We can assign probabilities to some events, or sets of continuous outcomes.
- E.g. Consider this experiment to watch an atom of sodium-26.
- Its half-life is 1 second (Applet: Nuclear Isotope Half-lifes)
- Define the outcomes to be the number of complete seconds before it decays: \(S=\{0, 1, 2, 3, \cdots \}\)
- \(|S|\) is countably infinite, i.e., discrete.
- \(p(0)=\frac{1}{2}, p(1)=\frac{1}{4}, \cdots\) \(p(k)=2^{-(k+1)}\)
- \(\sum_{k=0}^\infty p(k) = 1\)
- We can define events like these:
- The atom decays within the 1st second. p=.5.
- The atom decays within the first 3 seconds. p=.875.
- The atom's lifetime is an even number of seconds. \(p = \frac{1}{2} + \frac{1}{8} + \frac{1}{32} + \cdots = \frac{2}{3}\)
- Now consider another experiment: Watch another atom of Na-26
- But this time the outcome is defined to be the real number, x, that is the time until it decays.
- \(S = \{ x | x\ge0 \}\)
- \(|S|\) is uncountably infinite.
- We cannot talk about the probability that x=1.23 exactly. (It just doesn't work out.)
- However, we can define the event that \(1.23 < x < 1.24\), and talk about its probability.
- \(P[x>x_0] = 2^{-x_0}\)
- \(P[1.23 < x < 1.24]\) \(= 2^{-1.23} - 2^{-1.24} = 0.003\)
- Event
- collection of outcomes, subset of S
- what we're interested in.
- e.g., outcome is voltage, event is V>5.
- certain event: S
- null event: \(\emptyset\)
- elementary event: one discrete outcome
- Set theory
- Sets: S, A, B, ...
- Universal set: U
- elements or points: a, b, c
- \(a\in S, a\notin S\), \(A\subset B\)
- Venn diagram
- empty set: {} or \(\emptyset\)
- operations on sets: equality, union, intersection, complement, relative complement
- properties (axioms): commutative, associative, distributive
- theorems: de Morgan
- Prove deMorgan 2 different ways.
- Use the fact that A equals B iff A is a subset of B and B is a subset of A.
- Look at the Venn diagram; there are only 4 cases.
- 2.1.4 Event classes
- Remember: an event is a set of outcomes of an experiment, e.g., voltage.
- In a continuous sample space, we're interested only in some possible events.
- We're interested in events that we can measure.
- E.g., we're not interested in the event that the voltage is exactly an irrational number.
- Events that we're interested in are intervals, like [.5,.6] and [.7,.8].
- Also unions and complements of intervals.
- This matches the real world. You can't measure a voltage as 3.14159265...; you measure it in the range [3.14,3.15].
- Define \(\cal F\) to be the class of events of interest: those sets of intervals.
- We assign probabilities only to events in \(\cal F\).
- 2.2 Axioms of probability
- An axiom system is a general set of rules. The probability axioms apply to all probabilities.
- Axioms start with common sense rules, but get less obvious.
- I: 0<=P[A]
- II: P[S]=1
- III: \(A\cap B=\emptyset \rightarrow\) \(P[A\cup B] = P[A]+P[B]\)
- III': For \(A_1, A_2, ....\) if \(\forall_{i\ne j} A_i \cap A_j = \emptyset\) then \(P[\bigcup_{i=1}^\infty A_i]\) \(= \sum_{i=1}^\infty P[A_i]\)
- Example: cards. Q=event that card is queen, H=event that card is heart.
These events are not disjoint. Probabilities do not sum.
- \(Q\cap H \ne\emptyset\)
- P[Q] = 1/13=4/52, P[H] = 1/4=13/52, P[Q \(\cup\) H] = 16/52!=17/52.
- Example C=event that card is clubs. H and C are disjoint. Probabilities
do sum.
- \(C\cap H = \emptyset\) (corrected).
- P[C] = 13/52, P[H] = 1/4=13/52, P[Q \(\cup\) H] = 26/52.
- Example. Flip a fair coin \(A_i\) is the event that the first time you see
heads is the i-th time, for \(i\ge1\).
- We can assign probabilities to these countably infinite number of events.
- \(P[A_i] = 1/2^i\)
- They are disjoint, so probabilities sum.
- Probability that the first head occurs in the 10th or later toss = \(\sum_{i=10}^\infty 1/2^i\)
- Corollory 1
- \(P[A^c] = 1-P[A]\)
- E.g., P[heart] = 1/4, so P[not heart] = 3/4
- Corollory 2: P[A] <=1
- Corollory 3: P[\(\emptyset\)] = 0
- Corollory 4:
- For \(A_1, A_2, .... A_n\) if \(\forall_{i\ne j} A_i \cap A_j = \emptyset\) then \(P\left[\bigcup_{i=1}^n A_i\right] = \sum_{i=1}^n P[A_i]\)
- Proof by induction from axiom III.