The method developed long ago (1964) was already known in the numerical recipes 30 years ago to be far more effective than the more favorite Nelder and Mead method. It minimizes from a starting point a function that does not have to be differentiable. The main theoretical focus is that it converges in a finite number of iterations for a quadratic objective. Taken that smooth functions behave quadratically around their optimum, it is very efficient in the end.
Powell, M. J. D. (1964). "An efficient method for finding the minimum of a function of several variables without calculating derivatives". Computer Journal. 7 (2): 155–162. doi:10.1093/comjnl/7.2.155. hdl:10338.dmlcz/103029
Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007). "Section 10.7. Direction Set (Powell's) Methods in Multidimensions". Numerical Recipes: The Art of Scientific Computing (3rd ed.). New York: Cambridge University Press