What is Gradient Descent?
Gradient descent is an optimization algorithm used to minimize the cost function, which is the error term. It is an iterative method that converges to the optimum solution by moving in the direction of the steepest descent as defined by the negative of the gradient. The gradient descent technique has a hyperparameter called learning rate, α that specifies the jumps the algorithm takes to move towards the optimal solution.
