I’m trying to teach a lesson on gradient descent from a more statistical and theoretical perspective, and need a good example to show its usefulness.
What is the simplest possible algebraic function that would be impossible or rather difficult to optimize for, by setting its 1st derivative to 0, but easily doable with gradient descent? I preferably want to demonstrate this in context linear regression or some extremely simple machine learning model.
Logistic Regression is a simple machine learning model with no closed form solution.
You can write a log likelihood function and take its derivative and set it equal to zero.
no, to convince yourself, just try to do it.
Why is this not the most upvoted answer?