GRENZE International Journal of Engineering and Technology
Vol. 10
(2024), Issue 2
Recent Advances in Text Documents Clustering
Authors
Manisha gaikwad, G.R.Shinde, P.arikshit Mahalle, Nilesh Sable, Namrata Kharate
Abstract
This paper explores transformative models in document clustering, focusing on practical applications of prominent transformer-based approaches. BERT, a successful model in natural language processing (NLP), captures contextual information for both syntactical and semantic analysis. Pre-trained BERT embeddings, applied as feature vectors, prove effective in document clustering by capturing intricate syntactical and semantic relationships. BERTopic introduces a technique that combines BERT embeddings with topic modeling for document clustering, enabling comprehensive syntactical and semantic understanding of content. XLNet and RoBERTa, alternative transformer-based models, hold potential for clustering tasks, with experimental use offering valuable insights into their effectiveness. Attention mechanisms, popularized by transformers, enhance clustering models by enabling a more nuanced analysis of document content through the focused attention on different input sequence parts. Graph-based methods, deep clustering, self-supervised learning, transfer learning, ensemble clustering, and Unsupervised Learning from Pre-training (ULP) contribute to effective syntactical clustering, showcasing diverse strategies to improve the robustness and generalizability of clustering models.
Pages:
4991 - 4997