Deep Learning Methods for Register Classification - Mahato, Prashant (My University, 2021)
For this project the data used is the one collected by, Biber and Egbert (2018) related to various language articles from the internet. I am using BERT model (Bidirectional Encoder Representations from Transformers), which is a deep neural network and FastText, which is a shallow neural network, as a baseline to perform text classiﬁcation. Also, I am using Deep Learning models like XLNet to see if classiﬁcation accuracy is improved. Also, it has been described by Biber and Egbert (2018) what is register. We can think of register as genre. According to Biber (1988), register is varieties deﬁned in terms of general situational parameters. Hence, it can be inferred that there is a close relation between the language and the context of the situation in which it is being used. This work attempts register classiﬁcation using deep learning methods that use attention mechanism. Working with the models, dealing with the imbalanced datasets in real life problems, tuning the hyperparameters for training the models was accomplished throughout the work. Also, proper evaluation metrics for various kind of data was determined. The background study shows that how cumbersome the use classical Machine Learning approach used to be. Deep Learning, on the other hand, can accomplish the task with ease. The metric to be selected for the classiﬁcation task for different types of datasets (balanced vs imbalanced), dealing with overﬁtting was also accomplished.
Keywords: Register, Genre, Register Classiﬁcation, FastText, BERT, XLNet, Deep Learning, Attention Mechanism, Supervised Learning, Regularization