tag:blogger.com,1999:blog-6141980.post3965708845530435396..comments2024-03-20T12:28:35.004-05:00Comments on Nuit Blanche: Tolerance to Ambiguity: Sparsity and the Bayesian PerspectiveIgorhttp://www.blogger.com/profile/17474880327699002140noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-6141980.post-27657196943984242282013-02-19T12:40:53.600-06:002013-02-19T12:40:53.600-06:00Hi,
This is an interesting paper. However, I think...Hi,<br />This is an interesting paper. However, I think that there is something important missing in this discussion: the loss function. Any (Bayesian) point estimate is the minimizer of the posterior expectation of some loss; in particular, the MAP is the minimizer of the posterior expectation of the 0/1 loss (in fact, it is the limit of a family of estimates, but that can be ignored in this discussion). Accordingly, there is no reason whatsoever why the distribution of MAP estimates has to be similar to the prior; why would it? Furthermore, for a MAP estimate to yield "correct results" (whatever "correct" means), there is no reason why typical samples from the prior should look like those "correct results". In fact, the compressed sensing (CS) example in Section 3.3 of the paper illustrates this quite clearly: (a) CS theory guarantees that solving (4) or (5) yields the "correct" solution; (b) as explained in section 3.1, (5) is the MAP estimate of x under a Laplacian prior (and the linear-Gaussian likelihood therein explained); (c) thus, the solution of (5) is the minimizer of the posterior expectation of the 0/1 loss under the likelihood and prior just mentioned; (e) in conclusion, if the underlying vectors are exactly sparse enough (obviously not typical samples of a Laplacian) they can be recovered by computing the MAP estimate under a Laplacian prior, that is, by computing the minimizer of the posterior expectation of the 0/1 loss. This is simply a fact. There is nothing surprising here: the message is that the prior is only half of the story and it doesn't make sense to look at a prior without looking also at the loss function. In (Bayesian) point estimation, a prior is "good", not if it describes well the underlying objects to be estimated, but if used (in combination with the likelihood function and observations) to obtain a minimizer of the posterior expectation of some loss it leads to "good" estimates.<br />Regards,<br />Mario Figueiredo.<br /> <br />Unknownhttps://www.blogger.com/profile/05035893681282763567noreply@blogger.com