The Categorization of Documents Using Support Vector Machines
Keywords:
Classification, supervised learning, support vector machines (SVM), linear kernel, polynomial kernel, Gaussian radial basis function (RBF) kernelAbstract
Support vector machine (SVM) is a popular machine learning algorithm. It first converts the input data into a higher dimensional space using a collection of mathematical operations known as kernels, and then it classifies the data points into discrete clusters. In this paper we use a dataset that correlates to scientific article abstracts. The articles fall into four categories: astro-physics, computer science, math, physics. This paper`s goal is to use support vector machines (SVM) to predict a given document`s category based on its text. Then, it will compare the accuracy and F-Score of each SVM`s performance using a linear, polynomial, Gaussian radial basis function kernel. Subsequently, SVM is utilized as a binary classifier for classification tasks, and the classification algorithms` performance is assessed using the confusion matrix. The trade-off between the model`s sensitivity and specificity is compared and visualized using the receiver operating characteristic (ROC) curve, which is a measure of a classifier`s prediction quality.
References
C. M. Bishop, “Pattern Recognition and Machine Learning,” Secaucus, NJ, USA: Springer- Verlag, pp.325–358, 2006.
T. Hofmann, B. Schölkopf, and A. J. Smola, “Kernel methods in machine learning,” The An- nals of Statistics, Vol.36, No.3, pp.1171–1220, 2008.
C. Cortes and V. Vapnik, “Support-vector networks,” J. of Mach. Learn., Vol.20, No.3, pp.273–297, 1995.
J. C. Platt, “Probabilistic Outputs for Support Vector Machines and Comparison to Regularized Likelihood Methods”, MIT Press, pp.61–74, 1999.
M. M. Hossain and M. S. Miah, “Evaluation of different SVM kernels for predicting customer churn,” in Proc. 18th Int. Conf. Computing. and Inform. Technol., Dhaka, Bangladesh, pp.1–4, 2015.
V. Vural and J. G. Dy, “A hierarchical method for multi-class support vector machines,” in Proc. Int. Conf. Mach. Learn., ICML 2004, Banff, Alberta, Canada, pp.831–838, 2004.
Y. Zhan and D. Shen, “Design efficient support vector machine for fast classification”, Pattern Recognition, Vol.38, pp.157-161, 2005.
Y. Tang, B. Jin, Y. Sun and Y. Zhang, “Granular support vector machines for medical binary classification problems”, Proceedings of 2004 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, CIBCB `04, pp.73-78, 2004.
S. Du and S. Chen, “Weighted support vector machine for classification”, IEEE International Conference on Systems, Man and Cybernetics, Vol.4, pp.3866-3871, 2005.
C. Lin, C. Yeh, S. Liang, J. Chung and N. Kumar, “Support-vector-based fuzzy neural network for pattern classification”, IEEE Transactions on Fuzzy Systems, Vol.14, No.1,pp.31-41, 2006.
Hammou BA, Lahcen AA, Mouline S. An effective distributed predictive model with matrix factorization and random forest for big data recommendation systems. Expert Syst Appl. 137:253–65, 2019.
Ullah F, Zhang B, Khan RU. Image-based service recommendation system: A JPEG-coeffcient RFs approach, IEEE Access.8:3308–18, 2020.
Ke G, Du HL, Chen YC. Cross-platform dynamic goods recommendation system based on reinforcement learning and social networks. Appl Soft Computing;104:107213, 2021.
Chen X, Liu D, Xiong Z, Zha ZJ. Learning and fusing multiple user interest representations for micro-video and movie recommendations. IEEE Trans Multimedia. 23:484–96, 2021.
Shimodaira H, Noma KI, Nakai M, and Sagayama S. “Dynamic time-alignment kernel in support vector machine. In: Touretzky DS, Mozer MC, and Hasselmo ME (Eds.),” Advances in neural information processing systems: 921-928. MIT Press, Cambridge, Massachusetts, USA, 2002.
B. Schölkopf, “The kernel trick for distances,” Technical Report, Microsoft Research, 2000.
Fan J, Heckman NE, and Wand MP “Local polynomial kernel regression for generalized linear models and quasi-likelihood functions,” Journal of the American Statistical Association, 90(429): pp.141-150, 1995.
I. S. Al-Mejibli, D. H. Abd, J. K.Alwan and A. J. Rabash, “Performance Evaluation of Kernels in Support Vector Machine,” in 1st Annual International Conference on Information and Sciences (AICIS), 2018.
Babaud J, Witkin AP, Baudin M, and Duda RO “Uniqueness of the Gaussian kernel for scale-space filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence,”PAMI-8(1): pp.26-33, 1986.
Q. Ding, Z. Li, S. Haeri, and Lj. Trajkovic´, “Application of machine learning techniques to detecting anomalies in communication networks: datasets and feature selection algorithms,” in Cyber Threat Intell., M. Conti, A. Dehghantanha, and T. Dargahi, Eds., Berlin: Springer, pp.47-70, 2018.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to this journal agree to publish their articles under the Creative Commons Attribution 4.0 International License, allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit and that in the event of reuse or distribution, the terms of this license are made clear.
