Implementation of Restricted Boltzmann Machine
Let’s dive deep into the implementation of unsupervised neural network models Restricted Boltzmann machine that is given by the Sklearn or Scikit-learn
RBMs are nonlinear feature learners that extract features from the input and are often used in conjunction with linear classifiers like linear SVMs or perceptron.
It basically makes assumptions from the distribution of inputs. Right now, Scikit-learn provides BernoulliRBM. It assumes input as binary values. It tries to maximize the likelihood of the data using graphical models. BernoulliRBM can also takes value between 0 and 1 which signifies the probability that the visible unit would turn off or on. It is useful in character recognition, where the interest is on which pixels are active and which aren’t.
Importing Libraries
Python3
# importing Libraries import numpy as np import matplotlib.pyplot as plt from sklearn.neural_network import BernoulliRBM from sklearn.datasets import fetch_openml from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import classification_report, accuracy_score |
The libraries and modules required to create a Bernoulli Restricted Boltzmann Machine (RBM) on the MNIST dataset are imported by this code. Data loading, preprocessing, RBM feature extraction, and Logistic Regression classification are all completed by it. After that, it assesses the model’s accuracy and produces a classification report.
Loading Dataset
Python3
# Load the MNIST dataset mnist = fetch_openml( "mnist_784" ) # Scale pixel values to [0, 1] X = mnist.data / 255.0 y = mnist.target.astype( int ) |
This code imports the MNIST dataset by retrieving the labels (y) and pixel data (X) using scikit-learn’s fetch_openml method. The pixel values are then scaled to the range [0, 1] by dividing by 255.0, making sure that the format is consistent for subsequent processing. When dealing with image data, this preprocessing step is typical.
Splitting dataset into train and test sets
Python3
# Split the dataset into training and test sets X_train, X_test, y_train, y_test = train_test_split( X, y, test_size = 0.2 , random_state = 42 ) |
This code uses the train_test_split tool from scikit-learn to divide the MNIST dataset into training and test sets. It allocates 20% of the data to the test set (X_test and y_test) and 80% of the data to the training set (X_train and y_train). Reproducibility is guaranteed via the random_state argument.
Model Development
Python3
# Create and configure the BernoulliRBM rbm = BernoulliRBM(n_components = 64 , learning_rate = 0.1 , n_iter = 10 , random_state = 0 , verbose = True ) # Fit the RBM to the training data rbm.fit(X_train) |
Output:
[BernoulliRBM] Iteration 1, pseudo-likelihood = -106.26, time = 4.14s
[BernoulliRBM] Iteration 2, pseudo-likelihood = -105.04, time = 4.90s
[BernoulliRBM] Iteration 3, pseudo-likelihood = -104.94, time = 4.60s
[BernoulliRBM] Iteration 4, pseudo-likelihood = -98.99, time = 5.14s
[BernoulliRBM] Iteration 5, pseudo-likelihood = -103.12, time = 5.00s
[BernoulliRBM] Iteration 6, pseudo-likelihood = -100.73, time = 5.00s
[BernoulliRBM] Iteration 7, pseudo-likelihood = -98.94, time = 5.42s
[BernoulliRBM] Iteration 8, pseudo-likelihood = -99.63, time = 4.95s
[BernoulliRBM] Iteration 9, pseudo-likelihood = -96.17, time = 4.95s
[BernoulliRBM] Iteration 10, pseudo-likelihood = -97.82, time = 4.94s
This code builds a Bernoulli Restricted Boltzmann Machine (RBM) using certain setup parameters, including 10 iterations, 64 hidden components, and a learning rate of 0.1. In order to learn how to extract features from the input data, the RBM is fitted to the training data (X_train) via the fit technique. Progress updates are enabled during training with the verbose=True option set.
Feature Transformation
Python3
# Transform the training and test data into the hidden representations X_train_encoded = rbm.transform(X_train) X_test_encoded = rbm.transform(X_test) # Train a classifier (Logistic Regression) on the encoded data classifier = LogisticRegression(max_iter = 100 ) classifier.fit(X_train_encoded, y_train) |
This function uses the trained RBM to turn the test and training data into hidden representations. Both sets’ encoded data is kept in X_train_encoded and X_test_encoded. The model is then taught to make predictions based on the compressed features that the RBM collected by training a Logistic Regression classifier on these encoded representations.
Model Evaluation and Prediction
Python3
# Evaluate the classifier on the test data y_pred = classifier.predict(X_test_encoded) accuracy = accuracy_score(y_test, y_pred) print ( "Classifier Accuracy:" , accuracy) |
Output:
Classifier Accuracy: 0.9064285714285715
This code predicts (y_pred) on the test data’s hidden representations (X_test_encoded) using the learned Logistic Regression classifier. It provides an indicator of the model’s performance on the test data by calculating the accuracy of the classifier’s predictions and printing the accuracy score.
Classification Report
Python3
classification_rep = classification_report(y_test, y_pred) print ( "Classification Report:" ) print (classification_rep) |
Output:
Classification Report:
precision recall f1-score support
0 0.95 0.95 0.95 1343
1 0.96 0.98 0.97 1600
2 0.91 0.91 0.91 1380
3 0.88 0.88 0.88 1433
4 0.87 0.85 0.86 1295
5 0.91 0.87 0.89 1273
6 0.94 0.96 0.95 1396
7 0.93 0.90 0.92 1503
8 0.88 0.89 0.88 1357
9 0.82 0.86 0.84 1420
accuracy 0.91 14000
macro avg 0.91 0.90 0.91 14000
weighted avg 0.91 0.91 0.91 14000
This code uses scikit-learn’s classification_report function to create a classification report by comparing the true labels (y_test) with the predicted labels (y_pred). A thorough evaluation of the model’s performance on the test data is provided by the report, which is output to the console and contains metrics like precision, recall, and F1-score for each class.
Unsupervised Neural Network Models
Unsupervised learning is an intriguing area of machine learning that reveals hidden structures and patterns in data without requiring labelled samples. Because it investigates the underlying relationships in data, it’s an effective tool for tasks like anomaly identification, dimensionality reduction, and clustering. There are several uses for unsupervised learning in domains like computer vision, natural language processing, and data analysis. Through self-sufficient data interpretation, it provides insightful information that enhances decision-making and facilitates comprehension of intricate data patterns.
There are many types of unsupervised learning, but here in this article, we will be focusing on Unsupervised neural network models.
Table of Content
- Unsupervised Neural Network
- Autoencoder
- Restricted Boltzmann Machine
- Self-Organizing maps (SOM)
- Generative Adversarial Networks (GANs)
- Implementation of Restricted Boltzmann Machine
- Advantages of Unsupervised Neural network models
- Disadvantages of Unsupervised Neural network models
Contact Us