 |
VOOZH | about |
|
VOOZH | about |
Hyperparameter tuning is the process of finding the optimal values for the hyperparameters of a machine-learning model. Hyperparameters are parameters that control the behaviour of the model but are not learned during training. Hyperparameter tuning is an important step in developing machine learning models because it can significantly improve the model's performance on new data. However, hyperparameter tuning can be a time-consuming and challenging task. Scikit-learn provides several tools that can help you tune the hyperparameters of your machine-learning models. In this guide, we will provide a comprehensive overview of hyperparameter tuning in Scikit-learn.
Hyperparameters are parameters that control the behaviour of a machine-learning model but are not learned during training. Some common examples of hyperparameters include:
Tuning hyperparameters is important because it can improve the performance of a training model on new data. For example, a poorly calibrated model will have high bias, meaning it is unsuitable for new data. On the other hand, a well-calibrated model will have bias and high variance, meaning it will extend well to new data and be accurate.
Scikit-Learn provides a variety of tools to help you tune the hyperparameters of your machine-learning models. A popular method is to use grid search.
GridSearch CV : Grid search is a brute force method that iterates through all possible combinations of hyperparameter values. You can implement grid search in scikit-learn using the GridSearchCV class. The GridSearchCV class defines a machine learning model and hyperparameter search space. A hyperparameter search space is a dictionary that defines the range of values for each hyperparameter. The model is then evaluated on the delayed validation dataset. The combination of hyperparameters that best fit the data used was selected as the optimal model.
Another popular way to tune hyperparameters is to use random search.
Random Search : Compared to grid search, random search is a cheaper method because it tests only a random sample of hyperparameter values. You can implement random search in sci-kit-learn using the RandomizedSearchCV class. The RandomizedSearchCV class takes a machine-learning model and a hyperparameter distribution as input. A hyperparameter distribution is a dictionary that defines the distribution of values to be tested for each hyperparameter. In the RandomizedSearchCV lecture, we train a machine learning program to randomly check hyperparameter values in hyperparameter passes.
At this point, the demo is evaluated based on the delayed assertion data set. The combination of hyperparameters that achieves the best performance on the assertion dataset is selected as the key metric.
In addition to grid search and random search, there are several other advanced hyperparameter tuning techniques that you can use in Scikit-learn. These techniques include:
Output:
Best Hyperparameters: {'C': 0.1, 'gamma': 0.1, 'kernel': 'poly'}
Best Accuracy Score: 95.83%
Test Accuracy: 100.00%
Output:
Grid Search - Best Hyperparameters: {'C': 0.1, 'gamma': 0.1, 'kernel': 'poly'}
Grid Search - Best Accuracy Score: 95.83%
Random Search - Best Hyperparameters: {'C': 3.900736564361965, 'gamma': 0.4094567581571069, 'kernel': 'linear'}
Random Search - Best Accuracy Score: 96.67%
Test Accuracy (Grid Search): 100.00%
Test Accuracy (Random Search): 96.67%
The output will display the best hyperparameters found during grid search and random search, along with their corresponding cross-validation accuracy scores.
It will also show the accuracy of the best models found by both methods on the test set.
You can compare the performance of grid search and random search in finding the best hyperparameters for the SVM classifier.
Output:
Best Hyperparameters: {'colsample_bytree': 1.0, 'learning_rate': 0.01, 'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 200, 'subsample': 1.0}
Accuracy on test set: 1.00
In this output:
Output:
Best Hyperparameters: {'subsample': 0.8, 'n_estimators': 200, 'min_child_weight': 1, 'max_depth': 7, 'learning_rate': 0.01, 'lambda': 0.3, 'gamma': 0.3, 'colsample_bytree': 0.9}
Accuracy on test set: 1.00
In this output:
Output:
Best Hyperparameters: {'C': 1, 'penalty': 'l2', 'solver': 'lbfgs'}
Accuracy on test set: 1.00
In this code:
Output:
Best Hyperparameters: {'solver': 'lbfgs', 'penalty': 'l2', 'C': 0.6280291441834259}
Accuracy on test set: 1.00
In this code:
Hyperparameter tuning is an imperative step in machine learning show improvement. Tuning hyperparameters can essentially make strides demonstrate execution on modern information. Scikit-learn gives a few devices to assist you tune the hyperparameters of your machine learning demonstrate.
