# Assume that the data d is obtained by the model

Ax + = d

where A is an n × m matrix with n < m and is a noise vector assumed Gaussian iid with variance 1. Since the number of the data is smaller than the number of parameters we seek there are infinite solutions that fit the data, that is, we can easily find ˆx such that Aˆx = d. Nevertheless, such x will overfit the data. To obtain a meaningful x we introduce

the following optimization problem

min

ρ(x)

x

s.t

kAx

dk^{2 }≤ n

(1.2) (1.3)

The function ρ(x) is often called a penalty function. review a few next.

arious choices are possible and we

Example 3 MRI image processing When generating MRI images of the brain it is possible to obtain a sequence of images I 1 , I 2 , . . . , I n . T h e i n t e n s i t y o f e a c h p i x e l i n t h e i m a g e d e c a y s a t a d i e r e n t r a t e .

A model for the intensity decay of a pixel located at x_{j }is

k

I(t, x_{j}) =

X

a_{i}(x_{j}) exp( λ_{i}(x_{j})t)

(1.4)

i=1

ˆ where a_{i}(x) and λ_{i}(x) are space dependant coefficients. Given s time measurements I(t_{n}, x_{j}), n =

1, . . . s we would like to evaluate the coefficients a_{i}(x_{j}) and λ_{i}(x_{j}) as they indicate possible pathology. This can be done by solving the following optimization problem

min

a,λ

s.t

s

k

X

X

n=1

i=1

1 2

a i e x p ( λ i t n )

a_{i }≥ 0 λ_{i }≥ 0

i = 1, . . . , k i = 1, . . . , k

ˆ I(t_{n})

!

2

(1.5)

Note the nonegativity assumption on the coefficients.

Example 4 Portfolio optimization Assume that there are n assets and that you have T dollars. The question is how to invest your T dollars within the given assets.

2