Hyperparameters in Machine learning are those parameters that are explicitly defined by the user to control the learning process. These hyperparameters are used to improve the learning of the model, and their values are set before starting the learning process of the model.

In this topic, we are going to discuss one of the most important concepts of machine learning, i.e., Hyperparameters, their examples, hyperparameter tuning, categories of hyperparameters, how hyperparameter is different from parameter in Machine Learning? But before starting, let's first understand the Hyperparameter.

What are hyperparameters?

In Machine Learning/Deep Learning, a model is represented by its parameters. In contrast, a training process involves selecting the best/optimal hyperparameters that are used by learning algorithms to provide the best result. So, what are these hyperparameters? The answer is, "Hyperparameters are defined as the parameters that are explicitly defined by the user to control the learning process."

Here the prefix "hyper" suggests that the parameters are top-level parameters that are used in controlling the learning process. The value of the Hyperparameter is selected and set by the machine learning engineer before the learning algorithm begins training the model. Hence, these are external to the model, and their values cannot be changed during the training process.

Some examples of Hyperparameters in Machine Learning

• • The k in kNN or K-Nearest Neighbour algorithm
• • Learning rate for training a neural network
• • Train-test split ratio
• • Batch Size
• • Number of Epochs
• • Branches in Decision Tree
• • Number of clusters in Clustering Algorithm

Difference between Parameter and Hyperparameter?

There is always a big confusion between Parameters and hyperparameters or model hyperparameters. So, in order to clear this confusion, let's understand the difference between both of them and how they are related to each other.

Model Parameters:

Model parameters are configuration variables that are internal to the model, and a model learns them on its own. For example, W Weights or Coefficients of independent variables in the Linear regression model. or Weights or Coefficients of independent variables in SVM, weight, and biases of a neural network, cluster centroid in clustering. Some key points for model parameters are as follows:

• • They are used by the model for making predictions.
• • They are learned by the model from the data itself
• • These are usually not set manually.
• • These are the part of the model and key to a machine learning Algorithm.

Model Hyperparameters:

Hyperparameters are those parameters that are explicitly defined by the user to control the learning process. Some key points for model parameters are as follows:

• • These are usually defined manually by the machine learning engineer.
• • One cannot know the exact best value for hyperparameters for the given problem. The best value can be determined either by the rule of thumb or by trial and error.
• • Some examples of Hyperparameters are the learning rate for training a neural network, K in the KNN algorithm,