Numerical Optimization,9780387987934

Numerical Optimization

by ;
ISBN13: 9780387987934
ISBN10: 0387987932
Format: Hardcover
Pub. Date: 1999-09-01
Publisher(s): Springer Verlag
Upgraded Edition: Click here!
List Price: $95.14

Rent Textbook

Select for Price
Add to Cart
There was a problem. Please try again later.

Digital

Rent Digital Options
Online:30 Days access
Downloadable:30 Days
$32.04
Online:60 Days access
Downloadable:60 Days
$42.72
Online:90 Days access
Downloadable:90 Days
$53.40
Online:120 Days access
Downloadable:120 Days
$64.08
Online:180 Days access
Downloadable:180 Days
$69.42
Online:1825 Days access
Downloadable:Lifetime Access
$106.80
*To support the delivery of the digital material to you, a non-refundable digital delivery fee of $3.99 will be charged on each digital item.
$69.42*
Add to Cart

New Textbook

We're Sorry
Sold Out

Used Textbook

We're Sorry
Sold Out

Buy from our Marketplace starting at $88.00

How Marketplace Works:

  • This item is offered by an independent seller and not shipped from our warehouse
  • Item details like edition and cover design may differ from our description; see seller's comments before ordering.
  • Sellers much confirm and ship within two business days; otherwise, the order will be cancelled and refunded.
  • Marketplace purchases cannot be returned to eCampus.com. Contact the seller directly for inquiries; if no response within two days, contact customer service.
  • Additional shipping costs apply to Marketplace purchases. Review shipping costs at checkout.

Summary

Presents a comprehensive and current description of the most effective methods in continuous optimization. Responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. DLC: Mathematical optimization.

Table of Contents

Preface vii
Introduction
1(9)
Mathematical Formulation
2(2)
Example: A Transportation Problem
4(1)
Continuous versus Discrete Optimization
4(2)
Constrained and Unconstrained Optimization
6(1)
Global and Local Optimization
6(1)
Stochastic and Deterministic Optimization
7(1)
Optimization Algorithms
7(1)
Convexity
8(2)
Notes and References
9(1)
Fundamentals of Unconstrained Optimization
10(24)
What Is a Solution?
13(6)
Recognizing a Local Minimum
15(3)
Nonsmooth Problems
18(1)
Overview of Algorithms
19(15)
Two Strategies: Line Search and Trust Region
19(2)
Search Directions for Line Search Methods
21(5)
Models for Trust-Region Methods
26(1)
Scaling
27(1)
Rates of Convergence
28(1)
R-Rates of Convergence
29(1)
Notes and References
30(1)
Exercises
30(4)
Line Search Methods
34(30)
Step Length
36(7)
The Wolfe Conditions
37(4)
The Goldstein Conditions
41(1)
Sufficient Decrease and Backtracking
41(2)
Convergence of Line Search Methods
43(3)
Rate of Convergence
46(9)
Convergence Rate of Steepest Descent
47(2)
Quasi-Newton Methods
49(2)
Newton's Method
51(2)
Coordinate Descent Methods
53(2)
Step-Length Selection Algorithms
55(9)
Interpolation
56(2)
The Initial Step Length
58(1)
A Line Search Algorithm for the Wolfe Conditions
58(3)
Notes and References
61(1)
Exercises
62(2)
Trust-Region Methods
64(36)
Outline of the Algorithm
67(2)
The Cauchy Point and Related Algorithms
69(8)
The Cauchy Point
69(1)
Improving on the Cauchy Point
70(1)
The Dogleg Method
71(3)
Two-Dimensional Subspace Minimization
74(1)
Steihaug's Approach
75(2)
Using Nearly Exact Solutions to the Subproblem
77(10)
Characterizing Exact Solutions
77(1)
Calculating Nearly Exact Solutions
78(4)
The Hard Case
82(2)
Proof of Theorem 4.3
84(3)
Global Convergence
87(7)
Reduction Obtained by the Cauchy Point
87(2)
Convergence to Stationary Points
89(4)
Convergence of Algorithms Based on Nearly Exact Solutions
93(1)
Other Enhancements
94(6)
Scaling
94(2)
Non-Euclidean Trust Regions
96(1)
Notes and References
97(1)
Exercises
97(3)
Conjugate Gradient Methods
100(34)
The Linear Conjugate Gradient Method
102(18)
Conjugate Direction Methods
102(5)
Basic Properties of the Conjugate Gradient Method
107(4)
A Practical Form of the Conjugate Gradient Method
111(1)
Rate of Convergence
112(6)
Preconditioning
118(1)
Practical Preconditioners
119(1)
Nonlinear Conjugate Gradient Methods
120(14)
The Fletcher-Reeves Method
120(1)
The Polak-Ribiere Method
121(1)
Quadratic Termination and Restarts
122(2)
Numerical Performance
124(1)
Behavior of the Fletcher-Reeves Method
124(3)
Global Convergence
127(4)
Notes and References
131(1)
Exercises
132(2)
Practical Newton Methods
134(30)
Inexact Newton Steps
136(3)
Line Search Newton Methods
139(3)
Line Search Newton-CG Method
139(2)
Modified Newton's Method
141(1)
Hessian Modifications
142(12)
Eigenvalue Modification
143(1)
Adding a Multiple of the Identity
144(1)
Modified Cholesky Factorization
145(5)
Gershgorin Modification
150(1)
Modified Symmetric Indefinite Factorization
151(3)
Trust-Region Newton Methods
154(10)
Newton-Dogleg and Subspace-Minimization Methods
154(1)
Accurate Solution of the Trust-REgion Problem
155(1)
Trust-Region Newton-CG Method
156(1)
Preconditioning the Newton-CG Method
157(2)
Local Convergence of Trust-Region Newton Methods
159(3)
Notes and References
162(1)
Exercises
162(2)
Calculating Derivatives
164(28)
Finite-Difference Derivative Approximations
166(10)
Approximating the Gradient
166(3)
Approximating a Sparse Jacobian
169(4)
Approximating the Hessian
173(1)
Approximating a Sparse Hessian
174(2)
Automatic Differentiation
176(16)
An Example
177(1)
The Forward Mode
178(1)
The Reverse Mode
179(4)
Vector Functions and Partial Separability
183(1)
Calculating Jacobians of Vector Functions
184(1)
Calculating Hessians: Forward Mode
185(2)
Calculating Hessians: Reverse Mode
187(1)
Current Limitations
188(1)
Notes and References
189(1)
Exercises
189(3)
Quasi-Newton Methods
192(30)
The BFGS Method
194(8)
Properties of the BFGS Method
199(1)
Implementation
200(2)
The SR1 Method
202(5)
Properties of SR1 Updating
205(2)
The Broyden Class
207(4)
Properties of the Broyden Class
209(2)
Convergence Analysis
211(11)
Global Convergence of the BFGS Method
211(3)
Superlinear Convergence of BFGS
214(4)
Convergence Analysis of the SR1 Method
218(1)
Notes and References
219(1)
Exercises
220(2)
Large-Scale Quasi-Newton and Partially Separable Optimization
222(28)
Limited-Memory BFGS
224(5)
Relationship with Conjugate Gradient Methods
227(2)
General Limited-Memory Updating
229(4)
Compact Representation of BFGS Updating
230(2)
SR1 Matrices
232(1)
Unrolling the Update
232(1)
Sparse Quasi-Newton Updates
233(2)
Partially Separable Functions
235(5)
A Simple Example
236(1)
Internal Variables
237(3)
Invariant Subspaces and Partial Separability
240(4)
Sparsity vs. Partial Separability
242(1)
Group Partial Separability
243(1)
Algorithms for Partially Separable Functions
244(6)
Exploiting Partial Separability in Newton's Method
244(1)
Quasi-Newton Methods for Partially Separable Functions
245(2)
Notes and References
247(1)
Exercises
248(2)
Nonlinear Least-Squares Problems
250(26)
Background
253(6)
Modeling, Regression, Statistics
253(3)
Linear Least-Squares Problems
256(3)
Algorithms for Nonlinear Least-Squares Problems
259(12)
The Gauss-Newton Method
259(3)
The Levenberg-Marquardt Method
262(2)
Implementation of the Levenberg-Marquardt Method
264(2)
Large-Residual Problems
266(3)
Large-Scale Problems
269(2)
Orthogonal Distance Regression
271(5)
Notes and References
273(1)
Exercises
274(2)
Nonlinear Equations
276(38)
Local Algorithms
281(11)
Newton's Method for Nonlinear Equations
281(3)
Inexact Newton Methods
284(2)
Broyden's Method
286(4)
Tensor Methods
290(2)
Practical Methods
292(12)
Merit Functions
292(2)
Line Search Methods
294(4)
Trust-Region Methods
298(6)
Continuation/Homotopy Methods
304(10)
Motivation
304(2)
Practical Continuation Methods
306(4)
Notes and References
310(1)
Exercises
311(3)
Theory of Constrained Optimization
314(48)
Local and Global Solutions
316(1)
Smoothness
317(2)
Examples
319(8)
A Single Equality Constraint
319(2)
A Single Inequality Constraint
321(3)
Two Inequality Constraints
324(3)
First-Order Optimality Conditions
327(4)
Statement of First-Order Necessary Conditions
327(3)
Sensitivity
330(1)
Derivation of the First-Order Conditions
331(11)
Feasible Sequences
332(4)
Characterizing Limiting Directions: Constraint Qualifications
336(3)
Introducing Lagrange Multipliers
339(2)
Proof of Theorem 12.1
341(1)
Second-Order Conditions
342(9)
Second-Order Conditions and Projected Hessians
348(2)
Convex Programs
350(1)
Other Constraint Qualifications
351(3)
A Geometric Viewpoint
354(8)
Notes and References
357(1)
Exercises
358(4)
Linear Programming: The Simplex Method
362(32)
Linear Programming
364(2)
Optimality and Duality
366(4)
Optimality Conditions
366(1)
The Dual Problem
367(3)
Geometry of the Feasible Set
370(4)
Basic Feasible Points
370(2)
Vertices of the Feasible Polytope
372(2)
The Simplex Method
374(4)
Outline of the Method
374(3)
Finite Termination of the Simplex Method
377(1)
A Single Step of the Method
378(1)
Linear Algebra in the Simplex Method
378(5)
Other (Important) Details
383(8)
Pricing and Selection of the Entering Index
383(3)
Starting the Simplex Method
386(3)
Degenerate Steps and Cycling
389(2)
Where Does the Simplex Method Fit?
391(3)
Notes and References
392(1)
Exercises
392(2)
Linear Programming: Interior-Point Methods
394(26)
Primal-Dual Methods
396(8)
Outline
396(3)
The Central Path
399(2)
A Primal-Dual Framework
401(1)
Path-Following Methods
402(2)
A Practical Primal-Dual Algorithm
404(5)
Solving the Linear Systems
408(1)
Other Primal-Dual Algorithms and Extensions
409(2)
Other Path-Following Methods
409(1)
Potential-Reduction Methods
409(1)
Extensions
410(1)
Analysis of Algorithm 14.2
411(9)
Notes and References
416(1)
Exercises
417(3)
Fundamentals of Algorithms for Nonlinear Constrained Optimization
420(20)
Initial Study of a Problem
422(1)
Categorizing Optimization Algorithms
423(3)
Elimination of Variables
426(8)
Simple Elimination for Linear Constraints
427(3)
General Reduction Strategies for Linear Constraints
430(4)
The Effect of Inequality Constraints
434(1)
Measuring Progress: Merit Functions
434(6)
Notes and References
437(1)
Exercises
438(2)
Quadratic Programming
440(50)
An Example: Portfolio Optimization
442(1)
Equality--Constrained Quadratic Programs
443(4)
Properties of Equality-Constrained QPs
444(3)
Solving the KKT System
447(6)
Direct Solution of the KKT System
448(1)
Range-Space Method
449(1)
Null-Space Method
450(2)
A Method Based on Conjugacy
452(1)
Inequality-Constrained Problems
453(4)
Optimality Conditions for Inequality-Constrained Problems
454(1)
Degeneracy
455(2)
Active-Set Methods for Convex QP
457(13)
Specification of the Active-Set Method for Convex QP
461(2)
An Example
463(2)
Further Remarks on the Active-Set Method
465(1)
Finite Termination of the Convex QP Algorithm
466(1)
Updating Factorizations
467(3)
Active-Set Methods for Indefinite QP
470(6)
Illustration
472(2)
Choice of Starting Point
474(1)
Failure of the Active-Set Method
475(1)
Detecting Indefiniteness Using the LBLT Factorization
475(1)
The Gradient-Projection Method
476(5)
Cauchy Point Computation
477(3)
Subspace Minimization
480(1)
Interior-Point Methods
481(3)
Extensions and Comparison with Active-Set Methods
484(1)
Duality
484(6)
Notes and References
485(1)
Exercises
486(4)
Penalty, Barrier, and Augmented Lagrangian Methods
490(38)
The Quadratic Penalty Method
492(8)
Motivation
492(2)
Algorithmic Framework
494(1)
Convergence of the Quadratic Penalty Function
495(5)
The Logarithmic Barrier Method
500(12)
Properties of Logarithmic Barrier Functions
500(5)
Algorithms Based on the Log-Barrier Function
505(2)
Properties of the Log-Barrier Function and Framework 17.2
507(2)
Handling Equality Constraints
509(1)
Relationship to Primal-Dual Methods
510(2)
Exact Penalty Functions
512(1)
Augmented Lagrangian Method
513(10)
Motivation and Algorithm Framework
513(3)
Extension to Inequality Constraints
516(2)
Properties of the Augmented Lagrangian
518(3)
Practical Implementation
521(2)
Sequential Linearly Constrained Methods
523(5)
Notes and References
525(1)
Exercises
526(2)
Sequential Quadratic Programming
528(48)
Local SQP Method
530(4)
SQP Framework
531(2)
Inequality Constraints
533(1)
IQP vs. EQP
534(1)
Preview of Practical SQP Methods
534(2)
Step Computation
536(3)
Equality Constraints
536(2)
Inequality Constraints
538(1)
The Hessian of the Quadratic Model
539(5)
Full Quasi-Newton Approximations
540(1)
Hessian of Augmented Lagrangian
541(1)
Reduced-Hessian Approximations
542(2)
Merit Functions and Descent
544(3)
A Line Search SQP Method
547(1)
Reduced-Hessian SQP Methods
548(5)
Some Properties of Reduced-Hessian Methods
549(1)
Update Criteria for Reduced-Hessian Updating
550(1)
Changes of Bases
551(1)
A Practical Reduced-Hessian Method
552(1)
Trust-Region SQP Methods
553(7)
Approach I: Shifting the Constraints
555(1)
Approach II: Two Elliptical Constraints
556(1)
Approach III: Sl1 QP (Sequential l1 Quadratic Programming)
557(3)
A Practical Trust-Region SQP Algorithm
560(3)
Rate of Convergence
563(4)
Convergence Rate of Reduced-Hessian Methods
565(2)
The Maratos Effect
567(9)
Second-Order Correction
570(1)
Watchdog (Nonmonotone) Strategy
571(2)
Notes and References
573(1)
Exercises
574(2)
A Background Material 576(35)
A.1 Elements of Analysis, Geometry, Topology
577(16)
Topology of the Euclidean Space Rn
577(3)
Continuity and Limits
580(1)
Derivatives
581(2)
Directional Derivatives
583(1)
Mean Value Theorem
584(1)
Implicit Function Theorem
585(1)
Geometry of Feasible Sets
586(5)
Order Notation
591(1)
Root-Finding for Scalar Equations
592(1)
A.2 Elements of Linear Algebra
593(18)
Vectors and Matrices
593(1)
Norms
594(3)
Subspaces
597(1)
Eigenvalues, Eigenvectors, and the Singular-Value Decomposition
598(1)
Determinant and Trace
599(1)
Matrix Factorizations: Cholesky, LU, QR
600(5)
Sherman-Morrison-Woodbury Formula
605(1)
Interlacing Eigenvalue Theorem
605(1)
Error Analysis and Floating-Point Arithmetic
606(2)
Conditioning and Stability
608(3)
References 611(14)
Index 625

An electronic version of this book is available through VitalSource.

This book is viewable on PC, Mac, iPhone, iPad, iPod Touch, and most smartphones.

By purchasing, you will be able to view this book online, as well as download it, for the chosen number of days.

Digital License

You are licensing a digital product for a set duration. Durations are set forth in the product description, with "Lifetime" typically meaning five (5) years of online access and permanent download to a supported device. All licenses are non-transferable.

More details can be found here.

A downloadable version of this book is available through the eCampus Reader or compatible Adobe readers.

Applications are available on iOS, Android, PC, Mac, and Windows Mobile platforms.

Please view the compatibility matrix prior to purchase.