Optimization is the problem of finding a set of inputs to an objective function that results in a maximum or minimum function evaluation. There are perhaps hundreds of popular optimization algorithms, and perhaps tens of algorithms to choose from in popular scientific code libraries. This can make it challenging to know which algorithms to consider for a given optimization problem.
In this tutorial, you will discover a guided tour of different optimization algorithms.
After completing this tutorial, you will know:
- Optimization algorithms may be grouped into those that use derivatives and those that do not.
- Classical algorithms use the first and sometimes second derivative of the objective function.
- Direct search and stochastic algorithms are designed for objective functions where function derivatives are unavailable.

Brownlee, J., 2022. How to Choose an Optimization Algorithm. [online] Machine Learning Mastery. Available at: [Accessed 3 May 2022].

Ana I. Pereira

Instituto Politécnico de Bragança