Text Classification is one of the most familiar use cases of NLP. The most common type of text classification problem includes spam identification, news text categorization, movie genre category prediction, sentiment analysis, etc. There can be a variety of use cases for every domain. The major disadvantages of the seq2seq model are we lose the dependency information, difficulty remembering the lengthy conversation, exploding gradient problems, etc while transformer-based models pay attention to the sequential words, as well as words far away from each other, their ability of learning, is more rigorous and better than seq2seq models giving higher prediction accuracy. This paper focuses on the multiple transformer pre-trained models that can be leveraged for text classification problems.
