Model Development and Evaluation
We will implement a Sequential model which will contain the following parts:
- Three Embedding Layers to learn a featured vector representations of the input vectors.
- A Bidirectional LSTM layer to identify useful patterns in the sequence.
- Then we will have one fully connected layer.
- We have included some BatchNormalization layers to enable stable and fast training and a Dropout layer before the final layer to avoid any possibility of overfitting.
- The final layer is the output layer which outputs soft probabilities for the three classes.
Python3
model = keras.models.Sequential([ layers.Embedding(max_words, 32, input_length=max_len), layers.Bidirectional(layers.LSTM(16)), layers.Dense(512, activation='relu', kernel_regularizer='l1'), layers.BatchNormalization(), layers.Dropout(0.3), layers.Dense(3, activation='softmax')])model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])model.summary() |
Output:
Summary of the model architecture
While compiling a model we provide these three essential parameters:
Python3
keras.utils.plot_model( model, show_shapes=True, show_dtype=True, show_layer_activations=True) |
Output:
Change in input as it passes in the model
Callback
Callbacks are used to check whether the model is improving with each epoch or not. If not then what are the necessary steps to be taken like ReduceLROnPlateau decreases learning rate further. Even then if model performance is not improving then training will be stopped by EarlyStopping. We can also define some custom callbacks to stop training in between if the desired results have been obtained early.
Python3
from keras.callbacks import EarlyStopping, ReduceLROnPlateaues = EarlyStopping(patience=3, monitor = 'val_accuracy', restore_best_weights = True)lr = ReduceLROnPlateau(patience = 2, monitor = 'val_loss', factor = 0.5, verbose = 0) |
So, finally, we have reached the step when we will train our model.
Python3
history = model.fit(X_train, Y_train, validation_data=(X_val, Y_val), epochs=50, verbose=1, batch_size=32, callbacks=[lr, es]) |
Output:
To get a better picture of the training progress we should plot the graph of loss and accuracy epoch-by-epoch.
Python3
history_df = pd.DataFrame(history.history)history_df.loc[:, ['loss', 'val_loss']].plot()history_df.loc[:, ['accuracy', 'val_accuracy']].plot()plt.show() |
Output:
Graph of loss and accuracy epoch by epoch
Hate Speech Detection using Deep Learning
There must be times when you have come across some social media post whose main aim is to spread hate and controversies or use abusive language on social media platforms. As the post consists of textual information to filter out such Hate Speeches NLP comes in handy. This is one of the main applications of NLP which is known as Sentence Classification tasks.
In this article, we will learn how to build an NLP-based Sequence Classification model which can predict Tweets as Hate Speech, Offensive Language, and Normal.
Contact Us