F1 Score

F1 Score = (2 x Precision x Recall) / (Precision + Recall)

The F1 score is an evaluation metric used in classification tasks that provides a balanced measure of a model’s performance by taking into account both precision and recall. It is the harmonic mean of precision and recall, combining the two metrics into a single value. The F1 score ranges from 0 to 1, where a value of 1 represents perfect precision and recall, while a value of 0 indicates poor performance.

The F1 score is particularly useful when the dataset is imbalanced or when there is an uneven distribution between the positive and negative classes. It helps assess the model’s ability to achieve a balance between correctly identifying positive instances (precision) and capturing all actual positive instances (recall).The F1 score provides a single metric that balances precision and recall. It is useful when there is a trade-off between the two metrics, such as in situations where both false positives and false negatives are important.

In imbalanced datasets, where the majority class heavily outweighs the minority class, accuracy alone may not provide an accurate representation of the model’s performance. The F1 score considers both false positives and false negatives, making it a more reliable metric for evaluating the model’s effectiveness in such scenarios.

Specificity (True Negative Rate)

Specificity = TN / (TN + FP)

Specificity, also known as the true negative rate, is an evaluation metric used in binary classification tasks to measure a model’s ability to correctly identify negative instances. It quantifies the proportion of true negative predictions out of all actual negative instances in the dataset.

In simpler terms, specificity represents the ratio of correctly identified negative instances (true negatives) to the total number of instances that are actually negative, regardless of whether the model predicted them as negative or positive. For example, suppose we have a binary classification problem with 100 negative instances, out of which the model correctly identifies 80 as negative (true negatives) but incorrectly predicts 20 as positive (false positives). In this case, the specificity would be 80 / (80 + 20) = 0.8 or 80%.

Specificity is particularly important in scenarios where the cost or impact of false positive predictions is high. False positives occur when the model incorrectly predicts a negative instance as positive, which can result in unnecessary actions or consequences. In certain domains, such as medical screenings or quality control, a high specificity is desirable. In medical screenings, a high specificity ensures that healthy individuals are correctly identified as negative, reducing the chances of unnecessary tests or treatments. In quality control, a high specificity ensures that products meeting the required standards are not mistakenly classified as defective.

Kappa score

Kappa = (observed - expected) / (100% - expected)

Kappa score, also known as Cohen’s kappa coefficient, is an evaluation metric used in machine learning and statistics to measure the level of agreement between the predicted and actual class labels beyond what would be expected by chance. It is particularly useful when dealing with imbalanced datasets or when the class distribution is not uniform.

The kappa score ranges from -1 to 1, with 1 indicating perfect agreement, 0 indicating agreement by chance, and negative values indicating agreement worse than chance.

The kappa score is commonly used in situations where the class distribution is imbalanced or when there is a significant class skew. It provides a more robust measure of agreement by taking into account the agreement that can be expected by chance.

We will use the “iris” dataset for the programming example. It contains three categories of 50 instances each. We will use the random forest to classify the data into categories and then print the above-mentioned evaluation metrics in R.

Classification evaluation metrics are quantitative measures used to assess the performance and accuracy of a classification model. These metrics provide insights into how well the model can classify instances into predefined classes or categories.

The commonly used classification evaluation metrics are: