The three main hyperparameter optimization algorithms are :
Grid Search : It is a way to detect the family of models parameterized by a grid of parameters. It trains the model for all the possible combinations from the value of hyperparameters provided.
Random Search : In this, it randomly searches the sample space and evaluates the sets from a probability distribution. Here, the model is run only a fixed number of times.
Bayesian Optimization : It uses Bayes theorem to direct the search to find the minimum or maximum objective function. It is most useful for objective functions that are complex, noisy, and/or expensive to evaluate.
