1   Iclicker questions

1. X and Y are two uniform r.v. on the interval [0,1]. X and Y are independent. Z=X+Y. What is E[Z]?

   1. 0
   2. 1/2
   3. 2/3

2. Now let W=max(X,Y). What is E[W]?

   1. 0
   2. 1/2
   3. 2/3

3. Experiment: toss two fair coins, one after the other. Observe two random variables:

   1. X is the number of heads.
   2. Y is the toss when the first head occurred, with 0 meaning both coins were tails.

   What is P[X=1]?

   1. 0
   2. 1/4
   3. 1/2
   4. 3/4
   5. 1

4. What is P[Y=1]?

   1. 0
   2. 1/4
   3. 1/2
   4. 3/4
   5. 1

5. What is P[Y=1 & X=1]?

   1. 0
   2. 1/4
   3. 1/2
   4. 3/4
   5. 1

6. What is P[Y=1|X=1]?

   1. 0
   2. 1/4
   3. 1/2
   4. 3/4
   5. 1

7. What is P[X=1|Y=1]?

   1. 0
   2. 1/4
   3. 1/2
   4. 3/4
   5. 1

2   Mathematica demo

1. Exercise 6.47, page 353.

3   Material from text

1   Iclicker questions

1. What is $$\int_{-\infty}^\infty e^{\big(-\frac{x^2}{2}\big)} dx$$?
   1. 1/2
   2. 1
   3. $2\pi$
   4. $\sqrt{2\pi}$
   5. $1/\sqrt{2\pi}$
2. What is the largest possible value for a correlation coefficient?
   1. 1/2
   2. 1
   3. $2\pi$
   4. $\sqrt{2\pi}$
   5. $1/\sqrt{2\pi}$
3. The most reasonable probability distribution for the number of defects on an integrated circuit caused by dust particles, cosmic rays, etc, is
   1. Exponential
   2. Poisson
   3. Normal
   4. Uniform
   5. Binomial
4. The most reasonable probability distribution for the time until the next request hits your web server is:
   1. Exponential
   2. Poisson
   3. Normal
   4. Uniform
   5. Binomial
5. If you add two independent normal random variables, each with variance 10, what is the variance of the sum?
   1. 1
   2. $\sqrt2$
   3. 10
   4. $10\sqrt2$
   5. 20

2   Material from text

1   Exam 2 stats

1. Number of students registered: 107
2. Number of exam2 submissions: 104
3. Highest score: 80
4. Lowest score: 10
5. Average score: 59.9
6. Median score: 64

2   Normal distribution table

For your convenience. I computed it with Matlab.:

x          f(x)      F(x)      Q(x)
-3.0000    0.0044    0.0013    0.9987
-2.9000    0.0060    0.0019    0.9981
-2.8000    0.0079    0.0026    0.9974
-2.7000    0.0104    0.0035    0.9965
-2.6000    0.0136    0.0047    0.9953
-2.5000    0.0175    0.0062    0.9938
-2.4000    0.0224    0.0082    0.9918
-2.3000    0.0283    0.0107    0.9893
-2.2000    0.0355    0.0139    0.9861
-2.1000    0.0440    0.0179    0.9821
-2.0000    0.0540    0.0228    0.9772
-1.9000    0.0656    0.0287    0.9713
-1.8000    0.0790    0.0359    0.9641
-1.7000    0.0940    0.0446    0.9554
-1.6000    0.1109    0.0548    0.9452
-1.5000    0.1295    0.0668    0.9332
-1.4000    0.1497    0.0808    0.9192
-1.3000    0.1714    0.0968    0.9032
-1.2000    0.1942    0.1151    0.8849
-1.1000    0.2179    0.1357    0.8643
-1.0000    0.2420    0.1587    0.8413
-0.9000    0.2661    0.1841    0.8159
-0.8000    0.2897    0.2119    0.7881
-0.7000    0.3123    0.2420    0.7580
-0.6000    0.3332    0.2743    0.7257
-0.5000    0.3521    0.3085    0.6915
-0.4000    0.3683    0.3446    0.6554
-0.3000    0.3814    0.3821    0.6179
-0.2000    0.3910    0.4207    0.5793
-0.1000    0.3970    0.4602    0.5398
      0    0.3989    0.5000    0.5000
 0.1000    0.3970    0.5398    0.4602
 0.2000    0.3910    0.5793    0.4207
 0.3000    0.3814    0.6179    0.3821
 0.4000    0.3683    0.6554    0.3446
 0.5000    0.3521    0.6915    0.3085
 0.6000    0.3332    0.7257    0.2743
 0.7000    0.3123    0.7580    0.2420
 0.8000    0.2897    0.7881    0.2119
 0.9000    0.2661    0.8159    0.1841
 1.0000    0.2420    0.8413    0.1587
 1.1000    0.2179    0.8643    0.1357
 1.2000    0.1942    0.8849    0.1151
 1.3000    0.1714    0.9032    0.0968
 1.4000    0.1497    0.9192    0.0808
 1.5000    0.1295    0.9332    0.0668
 1.6000    0.1109    0.9452    0.0548
 1.7000    0.0940    0.9554    0.0446
 1.8000    0.0790    0.9641    0.0359
 1.9000    0.0656    0.9713    0.0287
 2.0000    0.0540    0.9772    0.0228
 2.1000    0.0440    0.9821    0.0179
 2.2000    0.0355    0.9861    0.0139
 2.3000    0.0283    0.9893    0.0107
 2.4000    0.0224    0.9918    0.0082
 2.5000    0.0175    0.9938    0.0062
 2.6000    0.0136    0.9953    0.0047
 2.7000    0.0104    0.9965    0.0035
 2.8000    0.0079    0.9974    0.0026
 2.9000    0.0060    0.9981    0.0019
 3.0000    0.0044    0.9987    0.0013

x is often called z.

More info: https://en.wikipedia.org/wiki/Standard_normal_table

3   The large effect of a small bias

This is enrichment material. It is not in the text, and will not be on the exam. However, it might be in a future homework.

Consider tossing $n=10^6$ fair coins.

1. P[more heads than tails] = 0.5

2. Now assume that each coin has chance of being heads $p=0.5005$.

   What's P[more heads than tails]?

   1. Approx with a Gaussian. $\mu=500500, \sigma=500$.
   2. Let X be the r.v. for the number of heads.
   3. P[X>500000] = Q(-1) = .84
   4. I.e., increasing the probability of winning 1 toss by 1 part in 1000, increased the probability of winning 1,000,000 tosses from 50% to 84%.

3. Now assume that 999,000 of the coins are fair, but 1,000 will always be heads.

   What's P[more heads than tails]?

   1. Let X = number of heads in 999,000 tosses.
   2. We want P[X>499,000].
   1. Approx with a Gaussian. $\mu=499,500, \sigma=500$.
   2. P[X>499,000] = Q(-1) = .84 as before.
   3. I.e., fixing 0.1% of the coins increased the probability of winning 1,000,000 tosses from 50% to 84%.

The lesson for fixing elections: you decide.

4   Min, max of 2 r.v.

1. Example 5.43, page 274.

5   Chapter 6: Vector random variables, page 303-

1. Skip the starred sections.
2. Examples:
   1. arrivals in a multiport switch,
   2. audio signal at different times.
3. pmf, cdf, marginal pmf and cdf are obvious.
4. conditional pmf has a nice chaining rule.
5. For continuous random variables, the pdf, cdf, conditional pdf etc are all obvious.
6. Independence is obvious.
7. Work out example 6.5, page 306. The input ports are a distraction. This problem reduces to a multinomial probability where N is itself a random variable.

1   No new homework today

Enjoy GM week. The next homework will be posted Thurs, due next Thurs.

2   No iclicker today

Ditto.

3   Section 5.7 Conditional probability ctd

1. Example 5.35 Maximum A Posteriori Receiver on page 268.
2. Example 5.37, page 270.
3. Remember equations 5.49 a,b for total probability on page 269-70 for conditional expectation of Y given X.

4   Section 5.8 page 271: Functions of two random variables, ctd

1. Example 5.39 Sum of Two Random Variables, page 271.

2. Example 5.40 Sum of Nonindependent Gaussian Random Variables, page 272.

   I'll do an easier case of independent N(0,1) r.v. The sum will be N(0, $\sqrt{2}$ ).

3. Example 5.44, page 275. Tranform two independent Gaussian r.v from

   (X,Y) to (R, $\theta$).

5   Section 5.9, page 278: pairs of jointly Gaussian r.v.

1. I will simplify formula 5.61a by assuming that $\mu=0, \sigma=1$.

   $$f_{XY}(x,y)= \frac{1}{2\pi \sqrt{1-\rho^2}} e^{ \frac{-\left( x^2-2\rho x y + y^2\right)}{2(1-\rho^2)} } $$ .

2. The r.v. are probably dependent. $\rho$} says how much.

3. The formula degenerates if $|\rho|=1$ since the numerator and denominator are both zero. However the pdf is still valid. You could make the formula valid with l'Hopital's rule.

4. The lines of equal probability density are ellipses.

5. The marginal pdf is a 1 variable Gaussian.

6. Example 5.47, page 282: Estimation of signal in noise

   1. This is our perennial example of signal and noise. However, here the signal is not just $\pm1$ but is normal. Our job is to find the ''most likely'' input signal for a given output.

7. Important concept in the noisy channel example (with X and N both being Gaussian): The most likely value of X given Y is not Y but is somewhat smaller, depending on the relative sizes of \(\sigma_X\) and \(\sigma_N\). This is true in spite of \(\mu_N=0\). It would be really useful for you to understand this intuitively. Here's one way:

   If you don't know Y, then the most likely value of X is 0. Knowing Y gives you more information, which you combine with your initial info (that X is \(N(0,\sigma_X)\) to get a new estimate for the most likely X. The smaller the noise, the more valuable is Y. If the noise is very small, then the mostly likely X is close to Y. If the noise is very large (on average) then the most likely X is still close to 0.

6   Tutorial on probability density - 2 variables

In class 15, I tried to motivate the effect of changing one variable on probability density. Here's a try at motivating changing 2 variables.

1. We're throwing darts uniformly at a one foot square dartboard.
2. We observe 2 random variables, X, Y, where the dart hits (in Cartesian coordinates).
3. $$f_{X,Y}(x,y) = \begin{cases} 1& \text{if}\,\, 0\le x\le1 \cap 0\le y\le1\\ 0&\text{otherwise} \end{cases}$$
4. $$P[.5\le x\le .6 \cap .8\le y\le.9] = \int_{.5}^{.6}\int_{.8}^{.9} f_{XY}(x,y) dx \, dy = 0.01 $$
5. Transform to centimeters: $$\begin{bmatrix}V\\W\end{bmatrix} = \begin{pmatrix}30&0\\0&30\end{pmatrix} \begin{bmatrix}X\\Y\end{bmatrix}$$
6. $$f_{V,W}(v,w) = \begin{cases} 1/900& \text{if } 0\le v\le30 \cap 0\le w\le30\\ 0&\text{otherwise} \end{cases}$$
7. $$P[15\le v\le 18 \cap 24\le w\le27] = \\ \int_{15}^{18}\int_{24}^{27} f_{VW}(v,w)\, dv\, dw = \frac{ (18-15)(27-24) }{900} = 0.01$$
8. See Section 5.8.3 on page 286.
9. Next time: We've seen 1 r.v., we've seen 2 r.v. Now we'll see several r.v.

1   Test 2

You may bring 2 2-sided note sheets, each 8.5x11". They may be handwritten or printed or both. You may collaborate with others on the sheet. You may form a business to sell note sheets. You may study the textbook before the test. You may bring a calculator. You may not communicate during the exam except with the TAs or me. Since it is impossible for me to police all possible communication technologies, I have to trust you on this.

2   Sect 5.5 Independence, page 254

1. Example 5.19 on page 255.
2. Independence: Example 5.22 on page 256. Are 2 normal r.v. independent for different values of $\rho$ ?

3   Sect 5.6 Joint moments, p 257

1. Example 5.24, sum.
2. Example 5.25, product.
3. Example 5.26 page 259. Covariance of independent variables.
4. Correlation coefficient.
5. Example 5.27 page 260 uncorrelated but dependent

4   Sect 5.7 Conditional page 261

1. Example 5.29 loaded dice

2. Example 5.30.

3. Example 5.31 on page 264. This is a noisy comm channel, now with Gaussian (normal) noise. The problems are:

   1. what input signal to infer from each output, and
   2. how accurate is this?

4. 5.6.2 Joint moments etc

   1. Work out for 2 3-sided dice.
   2. Work out for tossing dart onto triangular board.

5. Example 5.27: correlation measures ''linear dependence''. If the dependence is more complicated, the variables may be dependent but not correlated.

6. Covariance, correlation coefficient.

7. Section 5.7, page 261. Conditional pdf. There is nothing majorly new here; it's an obvious extension of 1 variable.

   1. Discrete: Work out an example with a pair of 3-sided loaded dice.
   2. Continuous: a triangular dart board. There is one little trick because for P[X=x]=0 since X is continuous, so how can we compute P[Y=y|X=x] = P[Y=y &amp; X=x]/P[x]? The answer is that we take the limiting probability P[x<X<x+dx] etc as dx shrinks, which nets out to using f(x) etc.

8. Example 5.31 on page 264. This is a noisy comm channel, now with Gaussian (normal) noise. This is a more realistic version of the earlier example with uniform noise. The application problems are:

   1. what input signal to infer from each output,
   2. how accurate is this, and
   3. what cutoff minimizes this?

   In the real world there are several ways you could reduce that error:

   1. Increase the transmitted signal,
   2. Reduce the noise,
   3. Retransmit several times and vote.
   4. Handshake: Include a checksum and ask for retransmission if it fails.
   5. Instead of just deciding X=+1 or X=-1 depending on Y, have a 3rd decision, i.e., uncertain if $|Y|<0.5$, and ask for retransmission in that case.

9. Section 5.8 page 271: Functions of two random variables.

   1. We already saw how to compute the pdf of the sum and max of 2 r.v.

10. What's the point of transforming variables in engineering? E.g. in video, (R,G,B) might be transformed to (Y,I,Q) with a 3x3 matrix multiply. Y is brightness (mostly the green component). I and Q are approximately the red and blue. Since we see brightness more accurately than color hue, we want to transmit Y with greater precision. So, we want to do probabilities on all this.

11. Functions of 2 random variables

    1. This is an important topic.
    2. Example 5.44, page 275. Tranform two independent Gaussian r.v from (X,Y) to (R, $\theta$} ).
    3. Linear transformation of two Gaussian r.v.
    4. Sum and difference of 2 Gaussian r.v. are independent.

12. Section 5.9, page 278: pairs of jointly Gaussian r.v.

    1. I will simplify formula 5.61a by assuming that $\mu=0, \sigma=1$.

       $$f_{XY}(x,y)= \frac{1}{2\pi \sqrt{1-\rho^2}} e^{ \frac{-\left( x^2-2\rho x y + y^2\right)}{2(1-\rho^2)} } $$ .

    2. The r.v. are probably dependent. $\rho$} says how much.

    3. The formula degenerates if $|\rho|=1$ since the numerator and denominator are both zero. However the pdf is still valid. You could make the formula valid with l'Hopital's rule.

    4. The lines of equal probability density are ellipses.

    5. The marginal pdf is a 1 variable Gaussian.

13. Example 5.47, page 282: Estimation of signal in noise

    1. This is our perennial example of signal and noise. However, here the signal is not just $\pm1$ but is normal. Our job is to find the ''most likely'' input signal for a given output.

14. Next time: We've seen 1 r.v., we've seen 2 r.v. Now we'll see several r.v.