Engineering Probability Class 22 Thu 2019-04-04

Assume that we want to know X but can only see Y, which depends on X.

This is a generalization of our long-running noisy communication channel example. We'll do things a little more precisely now.

Another application would be to estimate tomorrow's price of GOOG (X) given the prices to date (Y).

Sometimes, but not always, we have a prior probability for X.

For the communication channel we do, for GOOG, we don't.

If we do, it's a ''maximum a posteriori estimator''.

If we don't, it's a ''maximum likelihood estimator''. We effectively assume that that prior probability of X is uniform, even though that may not completely make sense.

You toss a fair coin 3 times. X is the number of heads, from 0 to 3. Y is the position of the 1st head. from 0 to 3. If there are no heads, we'll say that the first head's position is 0.

E.g., 1 head can occur 3 ways (out of 8): HTT, THT, TTH. The 1st (and only) head occurs in position 1, one of those ways. p=1/8.

Conditional probabilities:

The total probability of error is 3/8.

We observe Y and want to guess X from Y. E.g., If we observe $$\small y= \begin{pmatrix}0\\1\\2\\3\end{pmatrix} \text{then } x= \begin{pmatrix}0\\ 2 \text{ most likely} \\ 1, 2 \text{ equally likely} \\ 1 \end{pmatrix}$$

There are different formulae. The above one was the MAP, maximum a posteriori probability.

$$g_{\text{MAP}} (y) = \max_x p_x(x|y) \text{ or } f_x(x|y)$$

That means, the value of $x$ that maximizes $p_x(x|y)$

What if we don't know p(x|y)? If we know p(y|x), we can use Bayes. We might measure p(y|x) experimentally, e.g., by sending many messages over the channel.

Bayes requires p(x). What if we don't know even that? E.g. we don't know the probability of the different possible transmitted messages.

Then use maximum likelihood estimator, ML. $$g_{\text{ML}} (y) = \max_x p_y(y|x) \text{ or } f_y(y|x)$$

There are other estimators for different applications. E.g., regression using least squares might attempt to predict a graduate's QPA from his/her entering SAT scores. At Saratoga in August we might attempt to predict a horse's chance of winning a race from its speed in previous races. Some years ago, an Engineering Assoc Dean would do that each summer.

Historically, IMO, some of the techniques, like least squares and logistic regression, have been used more because they're computationally easy than because they're logically justified.