# Exercise 3: Gauss-Newton SQP

TEMPO Spring School: Theory and Numerics for Nonlinear Model Predictive Control Exercise 3: Gauss-Newton SQP J. Andersson M. Diehl J. Rawlings M. Zano...

TEMPO Spring School: Theory and Numerics for Nonlinear Model Predictive Control

Exercise 3: Gauss-Newton SQP J. Andersson

M. Diehl

J. Rawlings

M. Zanon

University of Freiburg, March 27, 2015

Gauss-Newton sequential quadratic programming (SQP) In the exercises so far, we solved the NLPs with IPOPT. IPOPT is a popular open-source primaldual interior point code employing so-called filter line-search to ensure global convergence. Other NLP solvers that can be used from CasADi include SNOPT, WORHP and KNITRO. In the following, we will write our own simple NLP solver implementing sequential quadratic programming (SQP). (0) Starting from a given initial guess for the primal and dual variables (x(0) , λg ), SQP solves the NLP by iteratively computing local convex quadratic approximations of the NLP at the (k) current iterate (x(k) , λg ) and solving them by using a quadratic programming (QP) solver. For an NLP of the form: minimize x

f (x)

subject to x ≤ x ≤ x,

g ≤ g(x) ≤ g,

(1)

these quadratic approximations take the form: minimize ∆x

1 2

(k)

(k)

∆x| ∇2x L(x(k) , λg , λx ) ∆x + ∇x f (x(k) )| ∆x

subject to x − x(k) ≤ ∆x ≤ x − x(k) , (2) ∂g (k) (k) (k) g − g(x ) ≤ (x ) ∆x ≤ g − g(x ), ∂x where L(x, λg , λx ) = f (x) + λ|g g(x) + λ|x x is the so-called Lagrangian function. By solving this (k+1) QP, we get the (primal) step ∆x(k) := x(k+1) − x(k) as well as the Lagrange multipliers λg (k+1) and λx . For NLPs with a least-squares objective function f (x) = 21 kR(x)k22 , it is often a good idea to use the so-called Gauss-Newton method, which uses the following approximation of the Hessian of the Lagrangian:   ∂R  | ∂R   (k) (k) (k) ∇2x L x(k) , λ(k) , λ ≈ x x . g x ∂x ∂x Note that for least-squares objective functions, the gradient of the objective function is given |  ∂R (k) (k) by ∇x f (x) = ∂x x R x , independently of the Hessian approximation. The Gauss-Newton method has two major advantages: • The quadratic approximation will always be a convex optimization problem, for which efficient algorithms and software exist. • Since the approximation of ∇2x L does not depend on the Lagrange multipliers explicitly, we do not need to consider it at all during the iterations and we can work only with the primal iterates x(k) . 1

Tasks: 3.1 Note that the objective function of the Rosenbrock problem from Exercise 1 can be put in a least-squares form. Identify R(x) such that f (x) = 21 kR(x)k22 (on paper, no coding needed). 3.2 Implement a CasADi SXFunction g : R3 → R that calculates the equality constraint function g(x). Evaluate it numerically and confirm that you get the expected result. 3.3 Use algorithmic differentiation in CasADi to calculate the Jacobian merically and confirm that you get the expected result.

∂g ∂x .

Evaluate it nu-

3.4 Derive the Gauss-Newton QP subproblem (2) (on paper, no coding needed). 3.5 Implement a full-step Gauss-Newton method with 10 iterations to solve the Rosenbrock problem, using x(0) := [2.5, 3.0, 0.75]| . QP solvers that are available in CasADi include qpOASES (mainly for dense QPs), CPLEX and OOQP. Out of these, only qpOASES is distributed with CasADi – others must be installed separately. You can also use an NLP solver such as IPOPT to solve the QP. CasADi uses the following formulation of QPs: minimize x

1 | 2x H

x + g| x

subject to x ≤ x ≤ x,

a ≤ A x ≤ a,

(3)

QP solvers in CasADi are allocated for a fixed sparsity pattern of H and A: qp = qpStruct ( h = H . sparsity () , a = A . sparsity ()) solver = QpSolver ( " nlp . ipopt " , qp ) # Allocate solver ( IPOPT ) # solver = QpSolver (" qpoases " , qp ) # Allocate solver ( qpOASES ) solver . init () # Initialize the solver

and can then be “evaluated” to solve QPs as follows: solver . setInput (H , " h " ) solver . setInput (g , " g " ) solver . setInput (A , " a " ) solver . setInput ( lbx , " lbx " ) solver . setInput ( ubx , " ubx " ) solver . setInput ( lba , " lba " ) solver . setInput ( uba , " uba " ) solver . evaluate () print solver . getOutput ( " x " )

3.6 Extra: At each SQP iteration, print the constraint violation (primal infeasibility) as well as gradient of the Lagrangian (dual infeasibility). These are given by nx  X

(k) max(0, xi

− xi ) + max(0, xi −

(k) xi )

ng   X  + max(0, gi (x(k) ) − g i ) + max(0, g i − gi (x(k) )) i=0

i=0

and

∂R  (k) |  (k)  ∂g  (k) | x R x x λg + λx + ∂x ∂x

respecively.

2