Professor:
Francisco N. C. Sobral
Room:
Room 211 Building F67
E-mail:
Teaching times:
Tuesdays (asynchronous studies) 7:45 - 9:45 and Thursdays (online class) 13:30 - 15:30. Implementation discussions on some Thursdays 15:30 - 16:30
Thursdays GoogleMeet link: https://meet.google.com/dyc-nhmy-fdx
Thursdays GoogleMeet link: https://meet.google.com/dyc-nhmy-fdx
Bibliography
GOLUB, G. H.; VAN LOAN, C. F. Matrix computations. 3rd ed. The Johns Hopkins University Press, 1996.
DENNIS JR., J. E.; SCHNABEL, R. B. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Society for Industrial and Applied Mathematics, 1996.
NOCEDAL, J.; WRIGHT, S. J. Numerical Optimization. 2nd ed. Springer, New York, 2006.
MARTÍNEZ, J. M.; SANTOS, S. A. Métodos de Otimização Computacional. Publicações do 20o Colóquio Brasileiro de Matemática, 1995. Download
Extras
Content
Week 1
- Introduction
- Slides: Introduction V3
- Notes
Week 2
- Systems of nonlinear equation
- Newton and quasi-Newton methods
- Slides: Systems of nonlinear equations V5
- Notes: 1 2
Week 3
- Newton and quasi-Newton methods
- Linear convergence
- Slides: Systems of nonlinear equations V5
- Notes
- Implementation details
newton.jl
The Moré, Garbow and Hillstrom test problems
WL. Cruz and M. Raydan - Large scale test problems
Week 4
- Quadratic and superlinear convergence
- Slides: Systems of nonlinear equations V5
- Notes
Week 5
- Introduction to optimization
- Unconstrained optimization
- Line-search methods
- Slides: Introduction to unconstrained optimization V1
- Slides: Line-search methods V3
- Notes: 1, 2
Week 6
- Newton and quasi-Newton with global convergence
- Slides: Line-search methods V3
- Notes: 1 and 2
Week 7
- Newton and quasi-Newton with global convergence
- Practical stopping criteria
- Slides: Line-search methods V3, Scaling and stopping V1 and Trust region methods V3
- Notes: 1, 2 and 3
Week 10
- Convergence of trust-region algorithms
- Non-Euclidean norms for trust region
- Linear Conjugate Gradients
- Auxiliar text: An Introduction to the Conjugate Gradient Method Without the Agonizing Pain
- Slides: Trust Region V4 and Conjugate Gradient Methods V2
- Notes: 1 and 2
Week 11
- Linear Conjugate Gradients
- Convergence rate of CG and preconditioners
- Slides: Conjugate Gradient Methods V2
- Notes
Week 12
- Examples of constrained optimization problems
- First-order conditions in constrained optimization
- Constraint qualifications
- Slides: Introduction to constrained optimization V1 and First order conditions V1
- Notes: 1 and 2
Week 13
- KKT conditions
- Slides: First order conditions V1
- Notes
Week 14
- Projected gradient methods
- Spectral Projected Gradient
- Book: Opúsculo - J. M. Martínez
- Slides: Projected gradient V1
- Notes
Week 15
- Gencan
- Introduction to Augmented Lagrangians
- Slides: Projected gradient V1 and Augmented Lagrangian V1
- Code: gencan.jl
- Notes: 1 and 2
Last Update: February, 4 2021.