1   Review of normal distribution

1. Review of the normal distribution. If $\mu=0, \sigma=1$ (to keep it simple), then: $$f_N(x) = \frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}} $$
2. Show that $\int_{-\infty}^{\infty} f(x) dx =1$. This is example 4.21 on page 168.
3. Iclicker: Consider a normal r.v. with $\mu=500, \sigma=100$. What is the probability of being in the interval [400,600]? Page 169 might be useful.
   1. .02
   2. .16
   3. .48
   4. .68
   5. .84
4. Iclicker. Repeat that question for the interval [500,700].
5. Iclicker. Repeat that question for the interval [0,300].

2   Chapter 5, Two Random Variables

1. See intro I did in last class.
2. Today's reading: Chapter 5, page 233-242.
3. Review: An outcome is a result of a random experiment. It need not be a number. They are selected from the sample space. A random variable is a function mapping an outcome to a real number. An event is an interesting set of outcomes.
4. Example 5.3 on page 235. There's no calculation here, but this topic is used for several future problems.
5. Example 5.5 on page 238. We saw this on Monday.
6. Example 5.6 on page 240. Easy, look at it yourself.
7. Example 5.7 on page 241. Easy, look at it yourself.
8. Example 5.8 on page 242. Easy, look at it yourself.
9. Example 5.9 on page 242.
10. Example 5.11 on page 245. What is f(x,y)?
11. Cdf of mixed continuous - discrete random variables: section 5.3.1 on page 247. The input signal X is 1 or -1. It is perturbed by noise N that is U[-2,2] to give the output Y.. What is P[X=1|Y<=0]?
12. Example 5.14 on page 247.
13. Example 5.16 on page 252.

1   Markov and Chebyshev inequalities (Section 4.6, page 181)

1. Your web server averages 10 hits/second.
2. It will crash if it gets 20 hits.
3. By the Markov inequality, that has a probability at most 0.5.
4. That is way way too conservative, but it makes no assumptions about the distribution of hits.
5. For the Chebyshev inequality, assume that the variance is 10.
6. It gives the probability of crashing at under 0.1. That is tighter.
7. Assuming the distribution is Poisson with a=10, use Matlab 1-cdf('Poisson',20,10). That gives 0.0016.
8. The more we assume, the better the answer we can compute.
9. However, our assumptions had better be correct.
10. (Editorial): In the real world, and especially economics, the assumptions are, in fact, often false. However, the models still usually work (at least, we can't prove they don't work). Until they stop working, e.g., https://en.wikipedia.org/wiki/Long-Term_Capital_Management . Jamie Dimon, head of JP Morgan, has observed that the market swings more widely than is statistically reasonable.

2   Reliability (section 4.8, page 189)

1. The reliability R(t) is the probability that the item is still functioning at t. R(t) = 1-F(t).

2. What is the reliability of an exponential r.v.? ( $F(t)=1-e^{\lambda t}$ ).

3. The Mean Time to Failure (MTTF) is obvious. The equation near the top of page 190 should be

   $E[T] = \int_0^\infty \textbf{t} f(t) dt$

4. ... for an exponential r.v.?

5. The failure rate is the probability of a widget that is still alive now dying in the next second.

6. The importance of getting the fundamentals (or foundations) right:

   In the past 40 years, two major bridges in the Capital district have collapsed because of inadequate foundations. The Green Island Bridge collapsed on 3/15/77, see http://en.wikipedia.org/wiki/Green_Island_Bridge , http://cbs6albany.com/news/local/recalling-the-schoharie-bridge-collapse-30-years-later . The Thruway (I-90) bridge over Schoharie Creek collapsed on 4/5/87, killing 10 people.

   Why RPI likes the Roeblings: none of their bridges collapsed. E.g., when designing the Brooklyn Bridge, Roebling Sr knew what he didn't know. He realized that something hung on cables might sway in the wind, in a complicated way that he couldn't analyze. So he added a lot of diagonal bracing. The designers of the original Tacoma Narrows Bridge were smart enough that they didn't need this expensive margin of safety.

7. Another way to look at reliability: think of people.

   1. Your reliability R(t) is the probability that you live to age t, given that you were born alive. In the US, that's 98.7% for age 20, 96.4% for 40, 87.8% for 60.
   2. MTTF is your life expectancy at birth. In the US, that's 77.5 years.
   3. Your failure rate, r(t), is your probability of dying in the next dt, divided by dt, at different ages. E.g. for a 20-year-old, it's 0.13%/year for a male and 0.046%/year for a female http://www.ssa.gov/oact/STATS/table4c6.html . For 40-year-olds, it's 0.24% and 0.14%. For 60-year-olds, it's 1.2% and 0.7%. At 80, it's 7% and 5%. At 100, it's 37% and 32%.

8. Example 4.47, page 190. If the failure rate is constant, the distribution is exponential.

9. If several subsystems are all necessary, e.g., are in serial, then their reliabilities multiply. The result is less reliable.

   If only one of them is necessary, e.g. are in parallel, then their complementary reliabilities multiply. The result is more reliable.

   An application would be different types of RAIDs. (Redundant Array of Inexpensivexxxxxxxxxxxxx Independent Disks). In one version you stripe a file over two hard drives to get increased speed, but decreased reliability. In another version you triplicate the file over three drives to get increased reliability. (You can also do a hybrid setup.)

   (David Patterson at Berkeley invented RAID (and also RISC). He intended I to mean Inexpensive. However he said that when this was commercialized, companies said that the I meant Independent.)

10. Example 4.49 page 193, reliability of series subsystems.

11. Example 4.50 page 193, increased reliability of parallel subsystems.

3   4.9 Generating r.v

Ignore. It's surprisingly hard to do right, and has been implemented in builtin routines. Use them.

4   4.10 Entropy

Ignore since it's starred.

5   Chapter 5, Two Random Variables

1. One experiment might produce two r.v. E.g.,
   1. Shoot an arrow; it lands at (x,y).
   2. Toss two dice.
   3. Measure the height and weight of people.
   4. Measure the voltage of a signal at several times.
2. The definitions for pmf, pdf and cdf are reasonable extensions of one r.v.
3. The math is messier.
4. The two r.v. may be *dependent* and *correlated*.
5. The *correlation coefficient*, $\rho$, is a dimensionless measure of linear dependence. $-1\le\rho\le1$.
6. $\rho$ may be 0 when the variables have a nonlinear dependent relation.
7. Integrating (or summing) out one variable gives a marginal distribution.
8. We'll do some simple examples:
   1. Toss two 4-sided dice.
   2. Toss two 4-sided ''loaded'' dice. The marginal pmfs are uniform.
   3. Pick a point uniformly in a square.
   4. Pick a point uniformly in a triangle. x and y are now dependent.
9. The big example is a 2 variable normal distribution.
   1. The pdf is messier.
   2. It looks elliptical unless $\rho$=0.

1   Tutorial on probability density

Since the meaning of probability density when you transform variables is still causing problems for some people, think of changing units from English to metric. First, with one variable, X.

1. Let X be in feet and be U[0,1].

   $$f_X(x) = \begin{cases} 1& \text{if } 0\le x\le1\\ 0&\text{otherwise} \end{cases}$$

2. $P[.5\le x\le .51] = 0.01$.

3. Now change to centimeters. The transformation is $Y=30X$.

4. $$f_Y(y) = \begin{cases} 1/30 & \text{if } 0\le y\le30\\ 0&\text{otherwise} \end{cases}$$

5. Why is 1/30 reasonable?

6. First, the pdf has to integrate to 1: $$\int_{-\infty}^\infty f_Y(y) =1$$

7. Second, $$\begin{align} & P[.5\le x\le .51] \\ &= \int_.5^.51 f_X(x) dx \\& =0.01 \\& = P[15\le y\le 15.3] \\& = \int_{15}^{15.3} f_Y(y) dy \end{align}$$

2   Mathematica demo

1. Int
2. Sum
3. Manipulate
4. Binomial etc

3   Memoryless Exponential Distn

p 166.

4   4.4.3 Normal (Gaussian) dist

p 167.

Show that the pdf integrates to 1.

Lots of different notations:

Generally, F(x) = P(X<=x).

For normal: that is called $\Psi(x)$ .

$Q(x) = 1-\Psi(x)$ .

Example 4.22 page 169.

5   4.4.4 Gamma r.v.

1. 2 parameters
2. Has several useful special cases, e.g., chi-squared and m-Erlang.
3. The sum of m exponential r.v. has the m-Erlang dist.
4. Example 4.24 page 172.

6   Functions of a r.v.

1. Example 4.29 page 175.
2. Linear function: Example 4.31 on page 176.

1   This year's exam 1 online

with and w/o answers. See here.

We gave full points even if you didn't finish the arithmetic to compute a number. In the real world, you have computers. However, in the real world, accurate analysis and computation matter. In 1954, physicists made an eensy teensy error designing Castle Bravo.

2   Homework 5

online, due after break.

3   Notation

How to parse $F_X(x)$

1. Uppercase F means that this is a cdf. Different letters may indicate different distributions.
2. The subscript X is the name of the random variable.
3. The x is an argument, i.e., an input.
4. $F_X(x)$ returns the probability that the random variable is less or equal to the value x, i.e. prob(X<=x).

4   Matlab

1. Matlab, Mathematica, and Maple all will help you do problems too big to do by hand. Sometime I'll demo Matlab since IMO more of the class knows it.

2. Matlab

   1. Major functions:

      cdf(dist,X,A,...)
      pdf(dist,X,A,...)
      

   2. Common cases of dist (there are many others):

      'Binomial'
      'Exponential'
      'Poisson'
      'Normal'
      'Geometric'
      'Uniform'
      'Discrete Uniform'
      

   3. Examples:

      pdf('Normal',-2:2,0,1)
      cdf('Normal',-2:2,0,1)
      
      p=0.2
      n=10
      k=0:10
      bp=pdf('Binomial',k,n,p)
      bar(k,bp)
      grid on
      
      bc=cdf('Binomial',k,n,p)
      bar(k,bc)
      grid on
      
      x=-3:.2:3
      np=pdf('Normal',x,0,1)
      plot(x,np)
      

   4. Interactive GUI to explore distributions: disttool

   5. Random numbers:

      rand(3)
      rand(1,5)
      randn(1,10)
      randn(1,10)*100+500
      randi(100,4)
      

   6. Interactive GUI to explore random numbers: randtool

   7. Plotting two things at once:

      x=-3:.2:3
      n1=pdf('Normal',x,0,1)
      n2=pdf('Normal',x,0,2)
      plot(x,n1,n2)
      plot(x,n1,x,n2)
      plot(x,n1,'--r',x,n2,'.g')
      

3. Use Matlab to compute a geometric pdf w/o using the builtin function.

4. Iclicker. Which of the following do you prefer to use?

   1. Matlab
   2. Maple
   3. Mathematica
   4. Paper. It was good enough for Bernoulli and Gauss; it's good enough for me.
   5. Something else (please email about it me after the class).

5   My opinion

This is my opinion of Matlab.

1. Advantages
   1. Excellent quality numerical routines.
   2. Free at RPI.
   3. Many toolkits available.
   4. Uses parallel computers and GPUs.
   5. Interactive - you type commands and immediately see results.
   6. No need to compile programs.
2. Disadvantages
   1. Very expensive outside RPI.
   2. Once you start using Matlab, you can't easily move away when their prices rise.
   3. You must force your data structures to look like arrays.
   4. Long programs must still be developed offline.
   5. Hard to write in Matlab's style.
   6. Programs are hard to read.
3. Alternatives
   1. Free clones like Octave are not very good
   2. The excellent math routines in Matlab are also available free in C++ librarues
   3. With C++ libraries using template metaprogramming, your code looks like Matlab.
   4. They compile slowly.
   5. Error messages are inscrutable.
   6. Executables run very quickly.

6   Chapter 4 ctd

1. Taxi example: Sometimes there are mixed discrete and continuous r.v.

   1. Let X be the time X to get a taxi at the airport.
   2. 80% of the time a taxi is already there, so p(X=0)=.8.
   3. Otherwise we wait a uniform time from 0 to 20 minutes, so p(a<x<b)=.01(b-a), for 0<a<b<20.

2. Iclicker. For the taxi example, what is F(0)?

   1. 0
   2. .2
   3. .8
   4. .81
   5. 1

3. iclicker. For the taxi example, what is F(1)?

   1. 0
   2. .8
   3. .81
   4. .9
   5. 1

4. Text 4.2 p 148 pdf

5. Simple continuous r.v. examples: uniform, exponential.

6. The exponential distribution complements the Poisson distribution. The Poisson describes the number of arrivals per unit time. The exponential describes the distribution of the times between consecutive arrivals.

   Ex 4.7 p 150: exponential r.v.

7. Properties

   1. Memoryless.
   2. $f(x) = \lambda e^{-\lambda x}$ if $x\ge0$, 0 otherwise.
   3. Example: time for a radioactive atom to decay.

8. Ski p 4.2.1 for now.

9. The most common continuous distribution is the normal distribution.

10. 4.2.2 p 152. Conditional probabilities work the same with continuous distributions as with discrete distributions.

11. p 154. Gaussian r.v.

    1. $$f(x) = \frac{1}{\sqrt{2\pi} \cdot \sigma} e^{\frac{-(x-\mu)^2}{2\sigma^2}}$$
    2. cdf often called $\Psi(x)$
    3. cdf complement:
       1. $$Q(x)=1-\Psi(x) = \int_x^\infty \frac{1}{\sqrt{2\pi} \cdot \sigma} e^{\frac{-(t-\mu)^2}{2\sigma^2}} dt$$
       2. E.g., if $\mu=500, \sigma=100$,
          1. P[x>400]=0.66
          2. P[x>500]=0.5
          3. P[x>600]=0.16
          4. P[x>700]=0.02
          5. P[x>800]=0.001

12. Text 4.3 p 156 Expected value

13. Skip the other distributions (for now?).

1   Last year's exam 1 online

with and w/o answers. See here.

2   The different counting formulae for selecting k items from n

1. With replacement; order matters: \(n^k\).
2. W/o replacement; order matters: \(n(n-1)\cdots(n-k+1) = \frac{n!}{(n-k)!}\).
3. With replacement; order does not matter: \({{n-1+k} \choose k}\)
4. W/o replacement; order does not matter: \({n\choose k}=\frac{n!}{k!(n-k)!}\).

3   Review questions

1. Sampling with replacement with ordering: Each day I eat lunch at either the Union, Mcdonalds, Brueggers, or Sage. How many ways can I eat lunch over 5 days next week?
2. sampling w/o replacement and w/o order: How many different possible teams of 3 people can you pick from a group of 5?
3. sampling w/o replacement and with order: 5 people run a race for gold, silver, bronze. How many ways can the medals be won, w/o any ties?
4. binomial: A coin falls heads with p=.6. You toss it 3 times. What's the probability of 2 heads and 1 tail?
5. multinomial: You play 38-slot roulette 3 times. Once you got red, once black and once 0 or 00. What was the probability?
6. conditional probability: You have 2 dice, one 6-sided and one 12-sided. You pick one of them at random and throw it w/o looking; the top is 2. What's the probability that you threw the 6-sided die?
7. What's the expected number of times you'll have to toss the unknown die to get your first 2?
8. Independence: Consider {1,2,3,...12}. Is the set of even numbers independent of the set of multiples of 3? What if we use {1,2,..10}?
9. Useful review questions from the text.
   1. 2.83 on page 90.
   2. 2.99.
   3. 3.5 on page 130.
   4. 3.9.
   5. 3.15.
   6. 3.26.
   7. 3.88 on page 139.

4   Iclicker

1. We often add a check bit to an 8-bit byte, and set it so there are an odd number of 1 bits. When we read the byte, which is now 9 bits, if there are an even number of 1 s, then we know that there was an error.

   Assume that the probability of any one bit going bad is 1e-10. (The real number is much smaller.)

   What is the probability of the byte going bad (within 1 significant digit)?

   1. 1e-10
   2. 8e-10
   3. 9e-10
   4. 3.6e-19
   5. 7.2e-19

2. What is the probability of the byte going bad, but we don't notice that (because there were 2 bad bits)?

   1. 1e-10
   2. 8e-10
   3. 9e-10
   4. 3.6e-19
   5. 7.2e-19

1   Homework 4

2   Poisson vs Binomial vs Normal distributions

The binomial distribution is the exact formula for the probability of k successes from n trials (with replacement).

When n and k are large but p=k/n is small, then the Poisson distribution is a good approximation to the binomial. Roughly, n>10, k<5.

When n is large and p is not too small or too large, then the normal distribution, which we haven't seen yet, is an excellent approximation. Roughly, n>10 and \(|n-k|>2\ \sqrt{n}\) .

For big n, you cannot use binomial, and for really big n, cannot use Poisson. Imagine that your experiment is to measure the number of atoms decaying in this uranium ore . How would you compute \(\left(10^{23}\right)!\) ?

OTOH, for small n, you can compute binomial by hand. Poisson and normal probably require a calculator.

3   Homework solutions

are online under the Files tab at the top of the page.

4   Daily comic

Broomhilda

5   Iclicker questions

What is the best discrete probability distribution in the following cases.

1. Your car has five tires (including the spare), which may each independently be flat. The event is that not more than one tire is flat.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
2. 1,000,000 widgets are made this year, of which 1,000 are bad. You buy 5 at random. The event is that not more than one widget is bad.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
3. You toss a weighted coin, which lands heads 3/4 of the time.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
4. You toss a fair 12-sided die.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
5. You're learning to drive a car, and trying to pass the test. The event of interest is the number of times you have to take the test to pass. Assume that the tests are independent of each other and have equal probability.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
6. It's Nov 17 and you're outside in a dark place looking for Leonid meteorites. The event of interest is the number of meteorites per hour that you see.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform
7. It's Nov 17.... The new event of interest is the number of seconds until you see the next meteorite.
   1. Bernoulli
   2. binomial
   3. geometric
   4. Poisson
   5. uniform

6   Exam 1

1. Closed book but a calculator and one 2-sided letter-paper-size note sheet is allowed.
2. Material is from chapters 1-3.
3. Questions will be based on book, class, and homework, examples and exercises.
4. The hard part for you may be deciding what formula to use.
5. Any calculations will (IMHO) be easy.
6. Speed should not be a problem; most people should finish in 1/2 the time.

7   Chapter 3 exercises

We'll try these exercises from the text in class.

1. 3.51a in page 135.
2. 3.88 on page 139.
3. 3.91.

8   Chapter 4

1. I will try to ignore most of the theory at the start of the chapter.
2. Now we will see continuous random variables.
   1. The probability of the r.v being any exact value is infinitesimal,
   2. so we talk about the probability that it's in a range.
3. Sometimes there are mixed discrete and continuous r.v.
   1. Let X be the time X to get a taxi at the airport.
   2. 80% of the time a taxi is already there, so p(X=0)=.8.
   3. Otherwise we wait a uniform time from 0 to 20 minutes, so p(a<x<b)=.01(b-a), for 0<a<b<20.
4. Remember that for discrete r.v. we have a probability mass function (pmf).
5. For continuous r.v. we now have a probability density function (pdf), \(f_X(x)\).
6. p(a<x<a+da) = f(a)da
7. For any r.v., we have a cumulative distribution function (cdf) \(F_X(x)\).
8. The subscript is interesting only when we are using more than one cdf and need to tell them apart.
9. Definition: F(x) = P(X<=x).
10. The <= is relevant only for discrete r.v.
11. As usual Wikipedia isn't bad, and is deeper than we need here, Cumulative_distribution_function.
12. We compute means and other moments by the obvious integrals.