Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Education and Culture Executive Agency (EACEA). Neither the European Union nor EACEA can be held responsible for them.


Understanding Optimization Algorithms

Users: 1 - Average Rating: 4.00

Why do we optimize our machine learning models? We compare the results in every iteration by changing the hyperparameters in each step until we reach the optimum results. We create an accurate model with less error rate. There are different ways using which we can optimize a model. In this article, let’s discuss two important Optimization algorithms: Gradient Descent and Stochastic Gradient Descent Algorithms; how they are used in Machine Learning Models, and the mathematics behind them.


Machine Learning, Optimization

Target Group:

Cite as:
Secherla, S., 2022. Understanding Optimization Algorithms in Machine Learning. [online] Medium. Available at: [Accessed 3 May 2022].

Author of the review:
Ana I. Pereira
Instituto Politécnico de Bragança


You have to login to leave a comment. If you are not registered click here

Maria de Fátima Pacheco

Includes nice geometric interpretations of optimization concepts that support ML algorithms but I felt the provided information could be a little deeper.