Classification
- Import Necessary Libraries
Python3
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, precision_score, recall_score
|
Python3
iris = datasets.load_iris()
|
Split in Train and Test Set
Python3
X_train, X_test, y_train, y_test = train_test_split(
iris['data'], iris['target'], test_size=0.33, random_state=42)
|
Model training using Hinge Loss - We have imported SGD Classifier from scikit-learn and specified the loss function as ‘hinge’.
- Model uses the training data and corresponding labels to classify data based on hinge loss function.
Python3
from sklearn.linear_model import SGDClassifier
clf_hinge = SGDClassifier(loss="hinge", max_iter=1000)
clf_hinge.fit(X_train, y_train)
y_test_pred_hinge = clf_hinge.predict(X_test)
|
Model Evaluation when, loss = ‘hinge’
Python3
print('\033[1m' + "Hinge Loss" + '\033[0m')
print(
f"Precision score : {precision_score(y_test_pred_hinge,y_test,average='weighted')}")
print(
f"Recall score : {recall_score(y_test_pred_hinge,y_test,average='weighted')}")
print("Confusion Matrix")
confusion_matrix(y_test_pred_hinge, y_test)
|
Hinge Loss
Precision score : 0.98125
Recall score : 0.98
Confusion Matrix
array([[19, 0, 0],
[ 0, 15, 1],
[ 0, 0, 15]])
Model training and evaluation using Modified Huber Loss
- We have imported SGD Classifier from scikit-learn and specified the loss function as ‘modified_huber’.
- Model uses the training data and corresponding labels to classify data based on modified huber loss function.
Python3
clf_huber = SGDClassifier(loss="modified_huber", max_iter=1000)
clf_huber.fit(X_train, y_train)
y_test_pred_huber = clf_huber.predict(X_test)
print('\033[1m' + 'Modified Huber' + '\033[0m')
print(
f"Precision score : {precision_score(y_test_pred_huber,y_test,average='weighted')}")
print(
f"Recall score : {recall_score(y_test_pred_huber,y_test,average='weighted')}")
print("Confusion Matrix")
confusion_matrix(y_test_pred_huber, y_test)
|
Modified Huber
Precision score : 0.9520000000000001
Recall score : 0.76
Confusion Matrix
array([[19, 3, 0],
[ 0, 3, 0],
[ 0, 9, 16]])
Model training and evaluation using Log Loss
- We have imported SGD Classifier from scikit-learn and specified the loss function as ‘log_loss’.
- Model uses the training data and corresponding labels to classify data based on log loss function.
Python3
clf_log = SGDClassifier(loss="log_loss", max_iter=1000)
clf_log.fit(X_train, y_train)
y_test_pred_log = clf_log.predict(X_test)
print('\033[1m' + 'Log Loss' + '\033[0m')
print(
f"Precision score : {precision_score(y_test_pred_log,y_test,average='weighted')}")
print(
f"Recall score : {recall_score(y_test_pred_log,y_test,average='weighted')}")
print("Confusion Matrix")
confusion_matrix(y_test_pred_log, y_test)
|
Log Loss
Precision score : 0.9413333333333332
Recall score : 0.78
Confusion Matrix
array([[19, 2, 0],
[ 0, 4, 0],
[ 0, 9, 16]])
Regression
Import necessary libraries
Python3
from sklearn.linear_model import SGDRegressor
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn import metrics
import sklearn.datasets
|
Load California Housing Dataset
Python3
california = sklearn.datasets.fetch_california_housing()
|
Split the dataset in test and train
Python3
X_train, X_test, y_train, y_test = train_test_split(
california['data'], california['target'], test_size=0.33, random_state=42)
|
Scaling the training and test set
Python3
scaler = StandardScaler().fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
|
Model training and evaluation using Squared Error Loss function - We have specified the loss function as squared error. The model uses the training data and the corresponding target values. The model will learn to predict the target values using squared error loss.
- We will use the trained model to make predictions on the test data.
- Coefficient of determination measures how well the model fits the data. It represents the proportion of the variance in the target variable that is explained by the model.
Python3
clf = SGDRegressor(loss="squared_error", max_iter=10000)
clf.fit(X_train, y_train)
y_test_pred_squarederror = clf.predict(X_test)
print('\033[1m' + 'Squared Error' + '\033[0m')
print('Coefficient of determination: ',
metrics.r2_score(y_test, y_test_pred_squarederror))
|
Squared Error
Coefficient of determination: 0.9128269523492567
Model training and evaluation using Huber Loss function
- We have specified the loss function as Huber loss function. The model uses the training data and the corresponding target values.
Python3
clf_huber = SGDRegressor(loss="huber", max_iter=10000)
clf_huber.fit(X_train, y_train)
y_test_pred_huber = clf_huber.predict(X_test)
print('\033[1m' + 'Huber Error' + '\033[0m')
print('Coefficient of determination: ',
metrics.r2_score(y_test, y_test_pred_huber))
|
Huber Error
Coefficient of determination: 0.9148961830574946
Model training and evaluation using Epsilon Insensitive Loss function - We have specified the loss function as Epsilon Insensitive function. The model uses the training data and the corresponding target values.
Python3
clf_epsilon = SGDRegressor(loss="epsilon_insensitive",
epsilon=1, max_iter=10000)
clf_epsilon.fit(X_train, y_train)
y_test_pred_epsilon = clf_epsilon.predict(X_test)
print('\033[1m' + 'Epsilon Insensitive Loss Function' + '\033[0m')
print('Coefficient of determination: ',
metrics.r2_score(y_test, y_test_pred_epsilon))
|
Epsilon Insensitive Loss Function
Coefficient of determination: 0.4266448634462471
Different Loss functions in SGD
In machine learning, optimizers and loss functions are two components that help improve the performance of the model. A loss function measures the performance of a model by measuring the difference between the output expected from the model and the actual output obtained from the model. Mean square loss and log loss are some examples of loss functions. The optimizer helps to improve the model by adjusting its parameters so that the loss function value is minimized. SGD, ADAM, and RMSProp are some examples of optimizers. The focus of this article will be the various loss functions supported by the SGD module of Sklearn. Sklearn provides two classes of SGD: SGDClassifier for classification tasks and SGDRegressor for regression tasks.
Contact Us