Handling Missing Data in Logistic Regression using Missingness Indicator
In this approach, we incorporate the missingness mechanism into the analysis by including variables that indicate whether values are missing. This approach allows the model to learn from the missingness pattern and make more accurate predictions.
Pros of Handling Missing Data in Logistic Regression using Missingness Indicator
- Information Preservation: The Missingness Indicator method maintains information regarding missing data, enabling the model to address potential patterns or biases linked with missing values.
- Ease of Implementation: Implementing the Missingness Indicator is relatively simple, involving the addition of a binary variable to denote missingness, seamlessly integrating into logistic regression models.
- Avoidance of Imputation Assumptions: Unlike imputation methods, the Missingness Indicator approach sidesteps the need for assumptions about missing data mechanisms or value estimation, mitigating the risk of bias.
Cons Handling Missing Data in Logistic Regression using Missingness Indicator
- Dimensionality Increase: Incorporating Missingness Indicators raises dataset dimensionality, potentially leading to computational complexities, especially with large datasets or numerous missing values.
- Efficiency Reduction: The inclusion of Missingness Indicators may reduce model efficiency by introducing noise, particularly if missingness patterns lack informativeness or if many values are missing.
- Interpretation Complexity: Interpreting coefficients associated with Missingness Indicators can be more intricate compared to imputed values, as they represent missingness impact on outcomes rather than the missing values themselves, necessitating careful analysis and explanation of results.
Implementation
- Generate Synthetic Dataset with Missing Values:
- Generate a synthetic dataset (
X) with 1000 samples and 5 features. - Create a binary target variable (
y). - Introduce missing values (20% missing) into the dataset.
- Generate a synthetic dataset (
- Split Data into Training and Testing Sets:
- Split the dataset into training and testing sets (80-20 split).
- Modeling Method:
- Create indicator variables for missing values in the training set (
X_train_modeled) using pandas DataFrame. - Impute missing values in the training set with the mean of each feature.
- Train a logistic regression model (
model_modeled) on the training set (X_train_modeled).
- Create indicator variables for missing values in the training set (
- Evaluate Model on Test Set:
- Create indicator variables for missing values in the test set (
X_test_modeled) using pandas DataFrame. - Impute missing values in the test set with the mean of each feature.
- Evaluate the trained model (
model_modeled) on the test set (X_test_modeled) and calculate the accuracy (accuracy_modeled)..
- Create indicator variables for missing values in the test set (
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
import pandas as pd
# Step 1: Generate Synthetic Dataset with Missing Values
np.random.seed(2)
n_samples = 1000
n_features = 5
X = np.random.rand(n_samples, n_features)
y = np.random.randint(0, 2, n_samples) # binary target variable
missing_mask = np.random.rand(n_samples, n_features) < 0.2 # 20% missing values
X[missing_mask] = np.nan
# Step 2: Split Data into Training and Testing Sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Modeling Method:
# Create indicator variables for missing values
X_train_modeled = pd.DataFrame(X_train).copy()
X_train_modeled.columns = [f"Feature_{i}" for i in range(n_features)]
for col in X_train_modeled.columns:
X_train_modeled[str(col) + '_missing'] = X_train_modeled[col].isnull().astype(int)
X_train_modeled = X_train_modeled.fillna(X_train_modeled.mean()) # Impute missing values with mean
# Train logistic regression model
model_modeled = LogisticRegression()
model_modeled.fit(X_train_modeled, y_train)
# Evaluate model on test set
X_test_modeled = pd.DataFrame(X_test).copy()
X_test_modeled.columns = [f"Feature_{i}" for i in range(n_features)] # Preserve feature names
for col in X_test_modeled.columns:
X_test_modeled[str(col) + '_missing'] = X_test_modeled[col].isnull().astype(int)
X_test_modeled = X_test_modeled.fillna(X_test_modeled.mean()) # Impute missing values with mean
accuracy_modeled = model_modeled.score(X_test_modeled, y_test)
print("Accuracy with Modeling Method:", accuracy_modeled)
Output:
Accuracy with Modeling Method: 0.46
The output “Accuracy with Modeling Method: 0.46” indicates that the logistic regression model trained using the specified method achieved an accuracy of approximately 0.46 (46%) on the testing set. This means that the model correctly predicted the target variable (binary outcome) for about 46% of the instances in the testing set.
How to Handle Missing Data in Logistic Regression?
Logistic regression is a robust statistical method employed to model the likelihood of binary results. Nevertheless, real-world datasets frequently have missing values, presenting obstacles while fitting logistic regression models. Dealing with missing data effectively is essential to prevent skewed estimates and maintain the model’s accuracy. In this article, we have discussed how can we handle missing data in logistic regression.
Table of Content
- How to Handle Missing Data in Logistic Regression?
- 1. Handling Missing Data in Logistic Regression by Deletion
- 2. Handling Missing Data in Logistic Regression by Imputation
- 3. Handling Missing Data in Logistic Regression using Missingness Indicator
Contact Us