Assume that the data d is obtained by the model
Ax + = d
where A is an n × m matrix with n < m and is a noise vector assumed Gaussian iid with variance 1. Since the number of the data is smaller than the number of parameters we seek there are infinite solutions that fit the data, that is, we can easily find ˆx such that Aˆx = d. Nevertheless, such x will overfit the data. To obtain a meaningful x we introduce
the following optimization problem
dk2 ≤ n
The function ρ(x) is often called a penalty function. review a few next.
arious choices are possible and we
Example 3 MRI image processing When generating MRI images of the brain it is possible to obtain a sequence of images I 1 , I 2 , . . . , I n . T h e i n t e n s i t y o f e a c h p i x e l i n t h e i m a g e d e c a y s a t a d i e r e n t r a t e .
A model for the intensity decay of a pixel located at xj is
I(t, xj) =
ai(xj) exp( λi(xj)t)
ˆ where ai(x) and λi(x) are space dependant coefficients. Given s time measurements I(tn, xj), n =
1, . . . s we would like to evaluate the coefficients ai(xj) and λi(xj) as they indicate possible pathology. This can be done by solving the following optimization problem
a i e x p ( λ i t n )
ai ≥ 0 λi ≥ 0
i = 1, . . . , k i = 1, . . . , k
Note the nonegativity assumption on the coefficients.
Example 4 Portfolio optimization Assume that there are n assets and that you have T dollars. The question is how to invest your T dollars within the given assets.