Principles of Optimal Design Modeling and Computation, third edition

Design optimization is a standard concept in engineering design, and in other disciplines which utilize mathematical decision-making methods. This textbook focuses on the close relationship between a design problem’s mathematical model and the solution-driven methods which optimize it. Along with extensive material on modeling problems, this book also features useful techniques for checking whether a model is suitable for computational treatment. Throughout, key concepts are discussed in the context of why and when a particular algorithm may be successful, and a large number of examples demonstrate the theory or method right after it is presented. This book also contains step-by-step instructions for executing a design optimization project – from building the problem statement to interpreting the computer results. All chapters contain exercises from which instructors can easily build quizzes, and a chapter on “principles and practice” offers the reader tips and guidance based on the authors’ vast research and instruction experience. Panos Y. Papalambros is the J. B. Angell Distinguished University Professor and the Donald C. Graham Professor of Engineering, and holds additional professorships in Mechanical Engineering, Art and Design, and Architecture, at the University of Michigan. His research and teaching interests are in design science and design optimization. He is the author of more than 350 research publications. He served as the Chief Editor of the ASME Journal of Mechanical Design (2008–2012) and is the founding Chief Editor of the Design Science journal. Douglass J. Wilde is Professor Emeritus of Mechanical Engineering at Stanford University. He conducted research in optimization, computational geometry, and control theory. On retirement he began exploring student team building in design contests. In addition to previous editions of Principles of Optimal Design, he has written two books, Teamology (2009) and Jung’s Personality Theory Quantiied (2011).

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Principles of Optimal Design Modeling and Computation THIRD EDITION

PA N O S Y. PA PA L A M B RO S University of Michigan

D O U G L A S S J . WI L D E Stanford University

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

University Printing House, Cambridge CB2 8BS, United Kingdom One Liberty Plaza, 20th Floor, New York, NY 10006, USA 477 Williamstown Road, Port Melbourne, VIC 3207, Australia 4843/24, 2nd Floor, Ansari Road, Daryaganj, Delhi - 110002, India 79 Anson Road, #06-04/06, Singapore 079906 Cambridge University Press is part of the University of Cambridge. It furthers the University’s mission by disseminating knowledge in the pursuit of education, learning and research at the highest international levels of excellence. www.cambridge.org Information on this title: www.cambridge.org/9781107132672 DOI: 10.1017/9781316451038 © Panos Y. Papalambros and Douglass J. Wilde 2017 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First edition 1988 Second edition 2000 Third edition 2017 Printed in the United States of America by Sheridan Books, Inc. A catalogue record for this publication is available from the British Library Library of Congress Cataloging-in-Publication data Names: Papalambros, Panos Y., author. | Wilde, Douglass J., author. Title: Principles of optimal design modeling and computation / Panos Y. Papalambros, University of Michigan, Douglass J. Wilde, Stanford University. Description: 3rd editon. | Cambridge, United Kingdom ; New York, NY 10006, USA : Cambridge University Press, [2017] | Includes bibliographical references and indexes. Identiiers: LCCN 2016040412 | ISBN 9781107132672 Subjects: LCSH: Mathematical optimization. | Mathematical models. Classiication: LCC QA402.5 .P374 2017 | DDC 519.6 – dc23 LC record available at https://lccn.loc.gov/2016040412 ISBN 978-1-107-13267-2 Hardback Additional resources for this publication at www.cambridge.org/optimaldesign Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

To our families And thus both here and in that journey of a thousand years, whereof I have told you, we shall fare well. Plato (The Republic, Book X)

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Contents

Preface to the Third Edition Notation

page xiii xvii

1 Optimization Models 1.1 Mathematical Modeling The System Concept • Hierarchical Levels • Mathematical Models • Elements of Models • Analysis and Design Models • Decision Making 1.2 Design Optimization The Optimal Design Concept • Formal Optimization Models • Multicriteria Models • Nature of Model Functions • The Question of Design Coniguration • Systems and Components • System Partitioning 1.3 Feasibility and Boundedness Feasible Domain • Boundedness • Activity 1.4 Topography of the Design Space Interior and Boundary Optima • Local and Global Optima • Constraint Interaction 1.5 Modeling and Computation 1.6 Design Projects 1.7 Summary Notes Exercises 2 Model Construction 2.1 Modeling Data Graphical and Tabular Data • Families of Curves • Numerically Generated Data 2.2 Best-Fit Curves and Least Squares

1 1

10

24 31

39 40 41

46 46

51

vii

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

viii

Contents

2.3 Neural Networks 2.4 Kriging 2.5 Modeling a Drive Screw Linear Actuator Assembling the Model Functions • Model Assumptions • Model Parameters • Negative Null Form 2.6 Modeling an Internal Combustion Engine Flat Head Chamber Design • Compound Valve Head Chamber Design 2.7 Design of a Geartrain Model Development • Model Summary • Model Reduction 2.8 Modeling Considerations Prior to Computation Natural and Practical Constraints • Asymptotic Substitution • Feasible Domain Reduction 2.9 Summary Notes Exercises 3 Model Boundedness 3.1 Bounds, Extrema, and Optima Well-Bounded Functions • Nonminimizing Lower Bound • Multivariable Extension • Air Tank Design 3.2 Constrained Optimum Partial Minimization • Constraint Activity • Cases 3.3 Underconstrained Models Monotonicity • First Monotonicity Principle • Criticality • Optimizing a Variable Out • Adding Constraints 3.4 Recognizing Monotonicity Simple and Composite Functions • Integrals 3.5 Inequalities Conditional Criticality • Multiple Criticality • Dominance • Relaxation • Uncriticality 3.6 Equality Constraints Equality and Activity • Replacing Monotonic Equalities by Inequalities • Directing an Equality • Regional Monotonicity of Nonmonotonic Constraints 3.7 Variables Not in the Objective Hydraulic Cylinder Design • Second Monotonicity Principle 3.8 Nonmonotonic Functions 3.9 Parametric Solution Particular Optimum and Parametric Procedures • Branching • Graphical Interpretation • Parametric Tests 3.10 Monotonicity Table and Model Reduction Model Reduction • Monotonicity Table • Setting Up

© in this web service Cambridge University Press

53 57 60

66

75 83

87

91 92

97 103

108 110

114

118 121 124

132

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Contents

3.11 3.12 3.13 3.14 3.15

• First New Table: Reduction • Second New Table: Two Directions and Reductions • Third New Table: Final Reduction Functional Monotonicity Analysis Explicit Algebraic Elimination • Implicit Numerical Solution Discrete Variables Discrete Design Activity and Optimality Constraint Activity Extended • Discrete Local Optima Model Preparation Procedure Summary Notes Exercises

ix

136 140 142 150 152

4 Interior Optima 4.1 Existence The Weierstrass Theorem • Suficiency 4.2 Local Approximation Taylor Series • Quadratic Functions • Vector Functions 4.3 Optimality First-Order Necessity • Second-Order Suficiency • Nature of Stationary Points 4.4 Convexity Convex Sets and Functions • Differentiable Functions 4.5 Local Exploration Gradient Descent • Newton’s Method 4.6 Searching Along a Line Gradient Method • Modiied Newton’s Method 4.7 Stabilization Modiied Cholesky Factorization 4.8 Trust Regions Moving with Trust • Trust Region Algorithms 4.9 Summary Notes Exercises

162

5 Boundary Optima 5.1 Feasible Directions 5.2 Describing the Constraint Surface Regularity • Tangent and Normal Hyperplanes 5.3 Equality Constraints Reduced (Constrained) Gradient • Lagrange Multipliers 5.4 Curvature at the Boundary Constrained Hessian • Second-Order Suficiency • Bordered Hessians 5.5 Feasible Iterations Generalized Reduced Gradient Method • Gradient Projection Method

202

© in this web service Cambridge University Press

163 165 171

177 183 188 191 194 197

202 205 209 214

220

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

x

Contents

5.6 Inequality Constraints Karush–Kuhn–Tucker Conditions • Lagrangian Standard Forms 5.7 Geometry of Boundary Optima Interpretation of KKT Conditions • Interpretation of Suficiency Conditions 5.8 Linear Programming Optimality Conditions • Basic LP Algorithm 5.9 Sensitivity Sensitivity Coeficients 5.10 Summary Notes Exercises 6 Local Computation 6.1 Numerical Algorithms Local and Global Convergence • Termination Criteria 6.2 Single-Variable Minimization Bracketing, Sectioning, and Interpolation • The Davies, Swann, and Campey Method • Inexact Line Search 6.3 Quasi-Newton Methods Hessian Matrix Updates • DFP and BFGS Formulas 6.4 Active Set Strategies Adding and Deleting Constraints • Lagrange Multiplier Estimates 6.5 Moving Along the Boundary 6.6 Penalties and Barriers Barrier Functions • Penalty Functions • Augmented Lagrangian (Multiplier) Methods 6.7 Sequential Quadratic Programming The Lagrange–Newton Equations • Enhancements of the Basic Algorithm • Solving the Quadratic Subproblem 6.8 Trust Regions with Constraints Relaxing Constraints • Using Exact Penalty Functions • Modifying the Trust Region and Accepting Steps • Yuan’s Trust Region Algorithm 6.9 Convex Approximation Algorithms Convex Linearization • Moving Asymptotes • Choosing Moving Asymptotes and Move Limits 6.10 Summary Notes Exercises

228 232

236 249 251

258 259 266

276 280 285 287

294

301

305

310

7 Nongradient Search 318 7.1 Direct Search 319 Coordinate Search • Extensions to Coordinate Search • Generalized Pattern Search • Nelder–Mead Algorithm

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Contents

7.2 Heuristic Methods Simulated Annealing • Genetic Algorithm • Multiobjective Genetic Algorithm (MOGA) • Particle Swarm Optimization • Extensions to Basic PSO 7.3 Black-Box Methods Dividing Rectangles (DIRECT) • Extensions to DIRECT • Eficient Global Optimization 7.4 Summary Notes Exercises 8 Systems Design 8.1 Decomposition-Based Design Optimization 8.2 Representation of System Interactions Functional Representation • Structure Matrix • Design Structure Matrix • Functional Dependence Table 8.3 Optimal System Design Properties Coupling, Shared, Linking, and Local Variables • System Consistency • System Optimality • Distributed Optimization 8.4 Partitioning Methods Hierarchical and Nonhierarchical Partitioning • Partitioning Synthesis with Linking Variables • Partitioning Synthesis with Linking Functions 8.5 Coordination Strategies Multidisciplinary Feasible • Individual Disciplinary Feasible 8.6 Airlow Sensor Design Problem Setup • MDF and IDF Solutions 8.7 Turbine Blade Design Problem Setup • Structural Analysis • Thermal Analysis • Surrogate Models • System Analysis • MDF and IDF Solutions 8.8 Analytical Target Cascading Target Cascading in Product Development • Mathematical Formulation • Numerical Solution • Augmented Lagrangian Penalty Function 8.9 Nonhierarchical ATC 8.10 Heavy-Duty Truck Suspension Design Problem Formulation • Solution Results 8.11 Optimal Design and Control Coupling • Measures of Coupling • Solution Strategies 8.12 Summary Notes Exercises 9 Principles and Practice 9.1 Preparing Models for Numerical Computation

© in this web service Cambridge University Press

xi

326

338

348

355 356 357

363

365

374 376 381

387

398 400 405 415

421 422

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Contents

xii

9.2 9.3 9.4 9.5 9.6 9.7

9.8

9.9

Modeling the Constraint Set • Modeling the Functions • Modeling the Objective Computing Derivatives Finite Differences • Automatic Differentiation Scaling Interpreting Numerical Results Code Output Data • Degeneracy Global Optimization Selecting Algorithms and Software Partial List of Software Packages • Partial List of Internet Sites Optimization Checklist Problem Identiication • Initial Problem Statement • Analysis Models • Optimal Design Model • Model Transformation • Local Iterative Techniques • Global Veriication • Final Review Concepts and Principles Model Building • Model Analysis • Local Searching • Global Searching • System Design Summary Notes References Index

© in this web service Cambridge University Press

426 433 437 438 439 444

449

454

456 477

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Preface to the Third Edition

It is almost three decades since this book was irst written. Much has changed since then. Perhaps the change most relevant to our readers is the central role that design has taken in society’s interests, and in education and research, but also in how it impacts our lives. Design of products and systems is recognized as an important element of a vibrant economy and an innovative society. More importantly, there is increased awareness that the many big problems we face today, such as environmental sustainability, can be addressed through thoughtful design and up-front assessment of the trade-offs involved, rather than as remedial efforts made after the fact. Understanding and quantifying such trade-offs to support our collective decision making means that design optimization is now more important than ever. Optimal design is the goal not only of engineering, but also of every other social effort to shape our world. Many of our problems usually grow from our inability to agree on what is “optimal.” The book was born out of our own desire to address explicitly what we mean by “optimal” and to put the concept of optimal design on a irm, rigorous foundation. There is an intimate relationship between the mathematical model that describes a design and the solution methods that optimize it. A basic premise from the start was that a good model can make optimization almost trivial, whereas a bad one can make correct optimization dificult or impossible. Software tools today provide capabilities for intricate analysis of many dificult performance aspects of a system. These analysis models, often referred to also as simulations, can be coupled with numerical optimization software to generate better designs iteratively. This virtual prototyping ability has grown dramatically and is an important contributor to reducing product development time and increasing robustness of systems. The success of such attempts depends strongly on how well the design problem has been formulated for an optimization study, and on how familiar the designer is with the workings and pitfalls of mathematical optimization techniques. As our computing capability increases, so does the complexity of our design problems. Hence, the basic premise of this book remains a modern one: there is need for a more than xiii

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

xiv

Preface to the Third Edition

casual understanding of the interactions between modeling and solution strategies in optimal design. The book grew out of graduate engineering design courses developed and taught at Michigan and Stanford for more than four decades. Deinitions of new concepts and rigorous proofs of principles are followed by immediate application to simple examples. In our courses a term design project has been an integral part of the experience, and so the book attempts to support that goal, namely to offer an integrated procedure of design optimization where global analysis and local iterative methods complement each other in a natural way. In this third edition, the chapters on model analysis, particularly with respect to boundedness and monotonicity, have been consolidated into a new Chapter 3. The discussion on metamodels using neural nets and kriging has been updated. A completely new chapter on nongradient methods has been added, recognizing that these methods are now mature and part of our toolkit. A new chapter on systems design optimization has also been added to address the reality that most design problems today must be viewed as system problems. The inal chapter on optimization practice has been expanded to include a short discussion on global optimization and when it may be worthwhile investing in this most elusive optimization goal. The book contains much more material than would be necessary for three lecture hours a week for one semester. Any course that requires an optimal design project should include Chapters 1, 2, and 9. Placing emphasis on problem formulation should include Chapter 3. A strong theme on gradient-based solution methods would include material from Chapters 4, 5, and 6. A selection from the nongradient approaches in Chapter 7 would round out the basic ideas for all algorithms in use today. Chapter 8 on systems optimization would be the basis for a course that emphasizes what has become known as multidisciplinary design optimization strategies. Linear programming for problems with purely linear functions is included in Chapter 5 on boundary optima, as a special case of boundary-tracking, active set strategy algorithms, thus avoiding the overhead of the specialized terminology traditionally associated with the subject. Problems with discrete variables irst encountered in Chapter 3 can be addressed with the methods described in Chapter 7. An effort has been expended to maintain consistency in terminology and symbols whilst bringing together concepts from a diversity of narrower disciplines and topics. We try to avoid using the same symbol with different meanings. The list of notation at the beginning of the book should help in this respect. For design examples and applications, we have maintained the symbols used locally for the particular problem and have not included them in the notation. Some instructors may wish to have their students code basic optimization algorithms. This is a very useful experience for students who are well versed in coding. We have occasionally required such coding as homework, but in a tight teaching term we have typically chosen to let students use existing optimization codes and concentrate on the mathematical model, while studying the theory behind the algorithms. Such decisions depend often on the availability and content of other optimization courses

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Preface to the Third Edition

xv

at a given institution, which may augment the course offered using this book as a text. Increased student familiarity with high-level, general purpose, computational tools and symbolic mathematics will continue to affect instructional strategies. Specialized design optimization topics, such as structural optimization and optimal control, are beyond the scope of this book. However, the ideas developed here are useful in understanding the specialized approaches needed for the solution of these problems. We have made notes throughout the text pointing to such directions. The book was also designed with self study in mind. A design engineer would require a brush up of introductory calculus and linear algebra before making good use of this book. Then, starting with the irst two chapters and the checklist in Chapter 9, one can model a problem and proceed toward numerical solution using commercial optimization software. After getting (or not getting) some initial results, one can go to Chapter 9 and start reading about what may go wrong. Understanding the material in Chapter 9 would require selective backtracking to the main chapters on modeling (Chapter 3), the foundations of gradient-based algorithms (Chapters 4, 5, and 6), and the concepts behind nongradient algorithms (Chapter 7). In a way, the book aims to give a stronger sense of control to the design engineers that use optimization tools. The book’s engineering lavor should not discourage its study by operations analysts, business analysts, economists, and other optimization practitioners. Monotonicity and boundedness analysis in particular may be applied to all optimization problems, not just to the design examples developed here for engineers. We offer our approach to design as a paradigm for studying and solving any decision problem. Many colleagues and students have reviewed or studied parts of the manuscript and offered valuable comments. We are particularly grateful to all of the students at Michigan and other institutions who found various errors in the irst two editions and also pointed to desired improvements in the manuscript. For this third edition, we especially acknowledge Alparslan Emrah Bayrak, Alex Burnap, Namwoo Kang, and Max Yi Ren, who provided extensive help in editing the new parts of the book and using them for teaching the design optimization course at Michigan. Comments and feedback by James Allison, Harrison Kim, Michael Kokkolaras, Jeremy Michalek, and Steven Hoffenson were most valuable in improving clarity and catching errors. The material on neural nets and kriging was based on guest lectures prepared for the Michigan course by Sigurd Nelson and updated by Max Yi Ren and Alex Burnap. The material on trust regions was also a contribution by Sigurd Nelson based on his dissertation. Chapter 3 was carefully edited by Alparslan Emrah Bayrak. The new Chapter 7 uses materials from the masters theses of Ryan Fellini and John Whitefoot, further expanded by Alex Burnap and Max Yi Ren during their teaching of the Michigan course. The new Chapter 8 uses materials from the dissertations of James Allison, Hossam Fathy, Namwoo Kang, Ramprasad Krishnamachari, Harrison Kim, Diane Peters, and Terry Wagner. Allison’s design examples in that chapter were originally developed for his dissertation. Special thanks go to Michael Kokkolaras for sustained advice on how to improve the textbook from his own experiences teaching the design optimization course both at Michigan and at McGill.

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

xvi

Preface to the Third Edition

The third edition is largely due to the insistence and patience of the editor, Peter Gordon (now retired), who never lost faith in the value of another edition. Starting with David Tranah, who recruited me (PYP) for the irst edition, Cambridge University Press has been a faithful and pleasant partner. Finishing this third edition has once again required the indulgence of my family, for which I am always grateful. I remain particularly grateful to my co-author and long-time mentor Douglass Wilde for encouraging me into this venture while he continues to study how design teams work most effectively. Doug taught me to think in optimization terms about almost everything, a practice I have followed ever since. P.Y.P. January 2017

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Notation

Integrating different approaches with different traditions brings typical notation dificulties. While one wishes for a uniform and consistent notation throughout, tradition and practice force us to use the same symbol with different meanings, or different symbols with the same meanings, depending on the subject treated. This is particularly important in an introductory book that encourages excursions to other specialized texts. In this book we have tried to use the notation that most commonly appears for the subject matter in each chapter – particularly for those chapters that lead to further study from other texts. Recognizing this additional burden on comprehension, we list symbols that are typically used in more than one section. The meanings given are those most commonly used in the text, but are not exclusive. The engineering examples throughout may employ many of these symbols in the specialized way of the particular discipline of the example. These symbols are not included in the list; they are given in the section containing the relevant examples. All symbols are deined the irst time they occur. A general notation practice used in this text for mathematical theory and examples is as follows. Lowercase bold letters indicate vectors; uppercase bold letters (usually Latin) indicate matrices; and uppercase script letters represent sets. Lowercase italic letters from the beginning of the alphabet (e.g., a, b, c) are often used for parameters, whereas those from the end of the alphabet (e.g., u, v, x, y, z) frequently indicate variables. Lowercase italic letters from the middle of the alphabet (e.g., i, j, k, l, m, n, p, q) are typically used as indices, subscripts, or superscripts. Lowercase Greek letters from the beginning of the alphabet (e.g., α, β, γ ) are often used as exponents. In engineering examples, when convenient, uppercase italic (but not bold) letters represent parameters, and lowercase letters stand for design variables. Symbols

A A

coeficient matrix of linear constraints working set (in active set strategies) xvii

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

xviii

Notation

ai b B B(x) c ck Cg Ch d D Di Ds det(A) e E f (x) fa (x) fc (x) f (x+ ) f (x− ) f n (x) ∂ f /∂xi ∂ 2 f /∂x2 , fxx , ∇ 2 f ∂ f /∂x, fx , ∇ f ∂f/∂x, ∇f F g j , g j (x) g(x)

ga (x) gc (x) g ∂g/∂x, ∇g ∂ 2 g/∂x2 h

ith analysis function right-hand-side coeficient vector of linear constraints (1) quasi-Newton approximation to the inverse of the Hessian; (2) “bordered” Hessian of the Lagrangian barrier function (in penalty transformations) vector of consistency constraints vector of consistency constraints at kth iteration number of linking inequality constraints number of linking equality constraints decision variables (1) diagonal matrix; (2) inverse of coeficient matrix A (in linear programming) feasible domain of all inequality constraints except the ith set of indices of analysis functions that depend on shared variable determinant of A (1) unit vector; (2) error vector expected value of objective function to be minimized with respect to (wrt) x artifact design objective function to be minimized controller design objective function to be minimized function increasing wrt x function decreasing wrt x nth derivative of f (x) irst partial derivative of f (x) wrt xi Hessian matrix of f (x); its element ∂ 2 f /∂xi ∂x j is the ith row and jth column (other symbol: H) gradient vector of f (x) – a row vector (other symbol: gT ) Jacobian matrix of f wrt x; it is m × m if f is an m-vector and x is an n-vector (other symbol: J) feasible set (other symbol: X ) jth inequality constraint function usually written in negative null form (1) vector of inequality constraint functions; (2) the transpose of the gradient of the objective function: g = ∇ f T , a column vector vector of inequality constraint functions for artifact design vector of inequality constraint functions for controller design greatest lower bound of f (x) Jacobian matrix of inequality constraints g(x) column vector of Hessians of g(x); see ∂ 2 y/∂x2 step size in inite differencing

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Notation

hj , hj (x) h(x) ha (x) hc (x) ∂h/∂x, ∇h ∂ 2 h/∂x2 , hxx H haux I J k Ki l l(x) L Lxx L LDLT Li M, Mk μk n N(0, σ 2 ) N (x) N o(x) P Pi Pi j P(x) P q(x) r, r ri r ji R Rj

xix

jth equality constraint function vector of equality constraint functions vector of equality constraint functions for artifact design vector of equality constraint functions for controller design Jacobian of equality constraints h(x) column vector of Hessians of h(x); see ∂ 2 y/∂x2 Hessian matrix of the objective function f auxiliary constraints identity matrix Jacobian matrix (subscript only) denotes values at kth iteration constraint set deined by ith constraint lower bound of f (x) lower bounding function Lagrangian function Hessian of the Lagrangian wrt x lower triangular matrix Cholesky factorization of a matrix index set of conditionally critical constraints bounding xi from below a “metric” matrix, i.e., a symmetric positive-deinite replacement of the Hessian in local iterations parameter in modiication of Hk in Mk number of design variables normal distribution with standard deviation σ normal subspace (hyperplane) of constraint surface deined by equalities and/or inequalities set of nonnegative real numbers including ininity order higher than x; it implies terms negligible compared to x projection matrix ith subproblem jth subproblem at the ith level penalty function (in penalty transformation) set of positive inite real numbers quadratic function of x (1) controlling parameters in penalty transformations; (2) lower bound on the condition number response quantity computed by ith analysis function responses from subproblem Pi to subproblem Pj (1) rank of Jacobian of tight constraints in a case; (2) condition number of a matrix set of neighbors for which subproblem j computes responses

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

xx

Notation

Rn s S ti j T (x) Tj T (x, r) T (x, λ, r) v w Ui x (xi ) xL xU x xa xc x0 , x1 , . . . xi( j) xi,k xli xsi ∂xi ∂x ∂xk x (x) xi Xi x X

n-dimensional Euclidean (real) space (1) state or solution variables; (2) search direction vectors (sk at kth iteration) Boolean matrix selecting components of analysis functions that correspond to coupling variables targets from subproblem Pj to subproblem Pi tangent subspace (hyperplane) of the constraint surface deined by equalities and/or inequalities set of neighbors for which subproblem j sets targets penalty transformation augmented Lagrangian function (a penalty transformation) penalty weights for linear terms of augmented Lagrangian penalty function penalty weights for quadratic terms of augmented Lagrangian penalty function index set of conditionally critical constraints bounding xi from above (ith) design variable lower bound on x upper bound on x vector of design variables, a point in Rn ; x = (x1 , x2 , . . . , xn )T vector of design variables for artifact design vector of design variables for controller design vectors corresponding to points 0, 1, . . .; not to be confused with the components x0 , x1 , . . . ith component of vector x j ; not used very often ith component of vector xk (k is iteration number) local variable of ith analysis function shared variable of ith analysis function ith element of ∂x, equals xi − xi(0) perturbation vector about point x0 , equals x − x0 ; subscript 0 is dropped for simplicity perturbation vector about xk , equals xk+1 − xk argument of the ininum (supremum) of the problem over P argument of the partial minumum (i.e., the minimizer) of the objective wrt xi an n − 1 vector made from x = (x1 , . . . , xn )T with all components ixed except xi ; we write x = (xi ; Xi ) minimizer to a relaxed problem a subset of Rn to which x belongs; the feasible domain; the set constraint

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Notation

X Xi X∗ yi j y p (x) ∂ 2 y/∂x2

zi z(d) ∂z/∂d ∂ 2 z/∂d2 (∂z/∂h)∗ α, αk Ŵv Ŵm δ ε λ λmin , λmax μ φ ϕ

ωi

xxi

set of x set of minimizers to a problem with the ith constraint relaxed set of all minimizers in a problem coupling variable computed by the jth analysis function and required as input to ith analysis function solution to the system analysis equations for a given design a vector of Hessians ∂ 2 yi /∂x2 , i = 1, . . . , m, of a vector function y = (y1 , . . . , ym )T ; it equals (∂ 2 y1 /∂x2 , ∂ 2 y2 /∂x2 , . . . , ∂ 2 ym /∂x2 ) set of linking variables of subproblem i including shared and coupling variables reduced objective function, equals f as a function of d only reduced gradient of f reduced Hessian of f sensitivity coeficient wrt equality constraints at the optimum step length in line search, kth iteration coupling vector coupling matrix step length used in coordinate search a small positive quantity a small positive quantity – often used in termination criteria Lagrange multiplier vector associated with equality constraints smallest and largest eigenvalues of the Hessian of f at x∗ Lagrange multiplier vector associated with inequality constraints (approximate) penalty function line search function, including merit function in sequential quadratic programming exact penalty function weights of entity i

Special Symbols

≤, ≥ = <, > < =, > = , ≡ ≡<, ≡> ·

inequality (active or inactive) equality (active or inactive) inactive inequality active or critical inequality uncritical inequality constraint active equality constraint active directed equality norm; a Euclidean norm is assumed unless otherwise stated

© in this web service Cambridge University Press

www.cambridge.org

Cambridge University Press 978-1-107-13267-2 — Principles of Optimal Design 3rd Edition Frontmatter More Information

Notation

xxii

∂x ∇f ∇2 f n xi i=1 n

xi

perturbation in the quantity x; a small (differential) change in x gradient of f (a row vector) Hessian of f (a symmetric matrix) sum over i; i = 1, 2, . . . , n (= x1 + x2 + · · · + xn ) product over i; i = 1, 2, . . . , n (= x1 x2 . . . xn )

i=1

arg min f (x) † ∗ T △

= ⊂, ⊆ ∈ ◦ (·)U (·)L

the value of x (argument) that minimizes f (subscript only) denotes values of quantities at stationary points (subscript only) denotes values of quantities at minimizing point(s) (superscript only) transpose of a vector or matrix deinition subset of belongs Hadamard product, element-by-element vector multiplication upper-level variables lower-level variables

© in this web service Cambridge University Press

www.cambridge.org