1.2.2

# Regularizing

nondi

erentiability

For many other problems it is difficult to obtain an exact equivalent differentiable formula- tion. On the other hand, it is easy to obtain a regularized formulation, that is, a differentiable formulation that yields a similar result to the nondifferentiable one.

min

^{X }ρ(x_{i}; θ)

i

s.t

Ax = b

Example 7 L_{1 }minimization again We approximate the L_{1 }minimization

(1.13)

where

ρ(t; θ) =

1 2

t^{2 }+ |t|

2

if |t| ≤ θ otherwise

(1.14)

The function ρ is known as the Huber function. It is easy to see that the function ρ(t; θ) is continuously di erentiable with respect to t and as θ → 0 one obtains the original L_{1 }minimization

1.2.3

# Sequential

minimization

In some problems it is possble to divide the unknowns into two groups of variables, p and q. If we assume that the second group, q is known then it is easy to minimize with respect to p. Therefore, it is possible to solve the problem in two stages

min f(p, q) = min

p,q

q

_{p } min f(p, q)

Example 8 Combination of linear and nonlinear unknowns

## Let

f(p, q) = (p exp(q)

1)^{2 }+ p^{2 }+ q^{2 }

Then, it is easy to verify that the minimum with respect to p is

p=

exp(q) exp(2q) + 1

And therefore, the problem is eqivalent to

min f(p(q), q) =

exp(2q) exp(2q) + 1

1

2

+

exp(q) exp(2q) + 1

2

+

q

^{2 }

5