1 Day off

Next Thurs Apr 8 is ECSE Wellness Day. There will be no class, nor homework due.

2 2nd Exam

1. Mon Apr 5, same rules as before.

2. It will cover material up thru class 16 (3/22/21).

3. That's roughly up thru Radke video 46.

3 New material

1. Example 5.31 on page 264. This is a noisy comm channel, now with Gaussian (normal) noise. This is a more realistic version of the earlier example with uniform noise. The application problems are:

   1. what input signal to infer from each output,

   2. how accurate is this, and

   3. what cutoff minimizes this?

   In the real world there are several ways you could reduce that error:

   1. Increase the transmitted signal,

   2. Reduce the noise,

   3. Retransmit several times and vote.

   4. Handshake: Include a checksum and ask for retransmission if it fails.

   5. Instead of just deciding X=+1 or X=-1 depending on Y, have a 3rd decision, i.e., uncertain if $|Y|<0.5$, and ask for retransmission in that case.

2. Section 5.8 page 271: Functions of two random variables.

   1. We already saw how to compute the pdf of the sum and max of 2 r.v.

3. What's the point of transforming variables in engineering? E.g. in video, (R,G,B) might be transformed to (Y,I,Q) with a 3x3 matrix multiply. Y is brightness (mostly the green component). I and Q are approximately the red and blue. Since we see brightness more accurately than color hue, we want to transmit Y with greater precision. So, we want to do probabilities on all this.

4. Functions of 2 random variables

   1. This is an important topic.

   2. Example 5.44, page 275. Tranform two independent Gaussian r.v from (X,Y) to (R, $\theta$} ).

   3. Linear transformation of two Gaussian r.v.

   4. Sum and difference of 2 Gaussian r.v. are independent.

5. Section 5.9, page 278: pairs of jointly Gaussian r.v.

   1. I will simplify formula 5.61a by assuming that $\mu=0, \sigma=1$.

      $$f_{XY}(x,y)= \frac{1}{2\pi \sqrt{1-\rho^2}} e^{ \frac{-\left( x^2-2\rho x y + y^2\right)}{2(1-\rho^2)} } $$ .

   2. The r.v. are probably dependent. $\rho$} says how much.

   3. The formula degenerates if $|\rho|=1$ since the numerator and denominator are both zero. However the pdf is still valid. You could make the formula valid with l'Hopital's rule.

   4. The lines of equal probability density are ellipses.

   5. The marginal pdf is a 1 variable Gaussian.

6. Example 5.47, page 282: Estimation of signal in noise

   1. This is our perennial example of signal and noise. However, here the signal is not just $\pm1$ but is normal. Our job is to find the ''most likely'' input signal for a given output.

7. Important concept in the noisy channel example (with X and N both being Gaussian): The most likely value of X given Y is not Y but is somewhat smaller, depending on the relative sizes of \(\sigma_X\) and \(\sigma_N\). This is true in spite of \(\mu_N=0\). It would be really useful for you to understand this intuitively. Here's one way:

   If you don't know Y, then the most likely value of X is 0. Knowing Y gives you more information, which you combine with your initial info (that X is \(N(0,\sigma_X)\) to get a new estimate for the most likely X. The smaller the noise, the more valuable is Y. If the noise is very small, then the mostly likely X is close to Y. If the noise is very large (on average) then the most likely X is still close to 0.

4 Tutorial on probability density - 2 variables

In class 15, I tried to motivate the effect of changing one variable on probability density. Here's a try at motivating changing 2 variables.

1. We're throwing darts uniformly at a one foot square dartboard.

2. We observe 2 random variables, X, Y, where the dart hits (in Cartesian coordinates).

3. $$f_{X,Y}(x,y) = \begin{cases} 1& \text{if}\,\, 0\le x\le1 \cap 0\le y\le1\\ 0&\text{otherwise} \end{cases}$$

4. $$P[.5\le x\le .6 \cap .8\le y\le.9] = \int_{.5}^{.6}\int_{.8}^{.9} f_{XY}(x,y) dx \, dy = 0.01 $$

5. Transform to centimeters: $$\begin{bmatrix}V\\W\end{bmatrix} = \begin{pmatrix}30&0\\0&30\end{pmatrix} \begin{bmatrix}X\\Y\end{bmatrix}$$

6. $$f_{V,W}(v,w) = \begin{cases} 1/900& \text{if } 0\le v\le30 \cap 0\le w\le30\\ 0&\text{otherwise} \end{cases}$$

7. $$P[15\le v\le 18 \cap 24\le w\le27] = \\ \int_{15}^{18}\int_{24}^{27} f_{VW}(v,w)\, dv\, dw = \frac{ (18-15)(27-24) }{900} = 0.01$$

8. See Section 5.8.3 on page 286.

9. Next time: We've seen 1 r.v., we've seen 2 r.v. Now we'll see several r.v.