acc382ab-8202-472b-afb2-6905a0c84ed120220404065804257wseas:wseasmdt@crossref.orgMDT DepositWSEAS TRANSACTIONS ON MATHEMATICS2224-28801109-276910.37394/23206http://wseas.org/wseas/cms.action?id=40511520221520222110.37394/23206.2022.21https://wseas.com/journals/mathematics/2022.phpEfstathiosKarypidisDemocritus University of Thrace, Department of Electrical and Computer Engineering Xanthi, 67100 GREECEStylianos G.MouslechDemocritus University of Thrace, Department of Electrical and Computer Engineering Xanthi, 67100 GREECEKassianiSkoularikiDemocritus University of Thrace, Department of Production Engineering and Management Xanthi, 67100 GREECEAlexandrosGazisDemocritus University of Thrace, Department of Electrical and Computer Engineering Xanthi, 67100 GREECEThe purpose of the study is to analyse and compare the most common machine learning and deep learning techniques used for computer vision 2D object classification tasks. Firstly, we will present the theoretical background of the Bag of Visual words model and Deep Convolutional Neural Networks (DCNN). Secondly, we will implement a Bag of Visual Words model, the VGG16 CNN Architecture. Thirdly, we will present our custom and novice DCNN in which we test the aforementioned implementations on a modified version of the Belgium Traffic Sign dataset. Our results showcase the effects of hyperparameters on traditional machine learning and the advantage in terms of accuracy of DCNNs compared to classical machine learning methods. As our tests indicate, our proposed solution can achieve similar - and in some cases better - results than existing DCNNs architectures. Finally, the technical merit of this article lies in the presented computationally simpler DCNN architecture, which we believe can pave the way towards using more efficient architectures for basic tasks.3232022323202212213019https://wseas.com/journals/mathematics/2022/a385106-006(2022).pdf10.37394/23206.2022.21.19https://wseas.com/journals/mathematics/2022/a385106-006(2022).pdf10.1007/978-3-030-97874-7_92Wang, J. (2022). Influence of Big Data on Manufacturing Accounting Informatization. Cyber Security Intelligence and Analytics. CSIA 2022. Lecture Notes on Data Engineering and Communications Technologies, 125, 695-700. https://doi.org/10.1007/978-3-030-97874-7_92 10.1007/bf02478259McCulloch, W.S., Pitts, W. (1943) A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115–133. https://doi.org/10.1007/BF02478259 10.12988/asms.2017.753Lin, J. W. (2017). Artificial neural network related to biological neuron network: a review. Advanced Studies in Medical Sciences, 5(1), 55-62. https://doi.org/10.12988/asms.2017.753 10.37394/232010.2021.18.3Didaskalou, E., Manesiotis, P., Georgakellos, D. (2021). Smart Manufacturing and Industry 4.0: A preliminary approach in structuring a conceptual framework. WSEAS Transactions on Advances in Engineering Education, 18, 27-36. https://doi.org/10.37394/232010.2021.18.3 10.4018/978-1-7998-3479-3.ch132Walczak, S. (2021). Artificial Neural Networks in Medicine: Recent Advances. Encyclopedia of Information Science and Technology, Fifth Edition, 1901-1918. www.doi.org/10.4018/978-1-7998-3479- 3.ch132 10.1007/s00521-022-07027-5Dixit, E., Jindal, V. (2022). IEESEP: an intelligent energy efficient stable election routing protocol in air pollution monitoring WSNs. Neural Computing and Applications. https://doi.org/10.1007/s00521-022- 07027-5 10.1201/9781003140351-20Ayobami, A.O., Bunu, R., Jamal, U., Adedayo, O.O. (2020). Artificial Intelligence and its Applications: Current Trends and Challenges. International Journal of Innovative Science and Research Technology, 5(2), 144-147. https://ijisrt.com/artificial-intelligence-andits-applications-current-trends-and-challenges 10.1007/s13042-020-01096-5Wang, X., Zhao, Y., Pourpanah, F. (2020). Recent advances in deep learning. International Journal of Machine Learning and Cybernetics, 11(4), 747-750. https://doi.org/10.1007/s13042-020-01096-5 10.37394/232010.2020.17.10Paulo de Oliveira, A., Braga, H.F.T. (2020). Artificial Intelligence: Learning and Limitations. WSEAS Transactions on Advances in Engineering Education, 17, 80-86. https://doi.org/10.37394/232010.2020.17.10 10.35940/ijitee.c8472.019320Abd Halim, K.N., Jaya, A.S. M., Fadzil, A.F.A. (2020). Data pre-processing algorithm for neural network binary classification model in bank tele-Marketing. International Journal of Innovative Technology and Exploring Engineering (IJITEE), 9, 272-277. https://www.doi.org/10.35940/ijitee.C8472.019320 10.1016/j.procs.2020.06.019Tiwari, R., Srivastava, S., Gera, R. (2020). Investigation of artificial intelligence techniques in finance and marketing. Procedia Computer Science, 173, 149-157. https://doi.org/10.1016/j.procs.2020.06.019 10.4018/ijkss.2021040104Praneetpholkrang, P., Kanjanawattana, S. (2021). A Novel Approach for Determining Shelter LocationAllocation in Humanitarian Relief Logistics. International Journal of Knowledge and Systems Science (IJKSS), 12(2), 52-68. www.doi.org/10.4018/IJKSS.2021040104 10.37394/232010.2020.17.11Almutairi, A. F., Gegov, A., Adda, M., Arabikhan, F. (2020). Conceptual artificial intelligence framework to improving English as second language. WSEAS Transactions on Advances in Engineering Education, 17, 87-91. https://doi.org/10.37394/232010.2020.17.11 10.37394/232010.2020.17.9Yfantis, V., Ntalianis, K., Ntalianis, F. (2020). Exploring the Implementation of Artificial Intelligence in the Public Sector: Welcome to the Clerkless Public Offices. WSEAS Transactions on Advance in Engineering Education. https://doi.org/10.37394/232010.2020.17.9 10.1109/iotais50849.2021.9359704Zualkernan, I., Judas, J., Mahbub, T., Bhagwagar, A., Chand, P. (2021). An aiot system for bat species classification. IEEE International Conference on Internet of Things and Intelligence System, 155-160. www.doi.org/10.1109/IoTaIS50849.2021.9359704 10.3390/rs12091366Li, J., Lin, D., Wang, Y., Xu, G., Zhang, Y., Ding, C., Zhou, Y. (2020). Deep discriminative representation learning with attention map for scene classification. Remote Sensing, 12(9), 1366. http://dx.doi.org/10.3390/rs12091366 10.1109/m2vip.2016.7827292Noble, F.K. (2016). Comparison of OpenCV's feature detectors and feature matchers. IEEE International Conference on Mechatronics and Machine Vision in Practice (M2VIP), 1-6. https://doi.org/10.1109/M2VIP.2016.7827292 10.1145/3236024.3236052Song, L., Minku, L.L., Yao, X. (2018). A novel automated approach for software effort estimation based on data augmentation. In Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 468-479. https://doi.org/10.1145/3236024.3236052 10.23919/elmar.2018.8534641Jakubović, A., Velagić, J. (2018). Image feature matching and object detection using brute-force matchers. IEEE International Symposium ELMAR, 83- 86. https://doi.org/10.23919/ELMAR.2018.8534641 10.3390/s19040810Che, E., Jung, J., Olsen, M.J. (2019). Object recognition, segmentation, and classification of mobile laser scanning point clouds: A state of the art review. Sensors, 19(4), 810. https://doi.org/10.3390/s19040810 10.1109/iccv.2003.1238663Sivic, J., Zisserman, A. (2003). Video Google: A text retrieval approach to object matching in videos. IEEE International Conference on Computer Vision, 3, 1470-1470. https://doi.org/10.1109/ICCV.2003.1238663 Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C. (2004). Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, 1(1-22), 1-2. https://www.researchgate.net/publication/228602850_ Visual_categorization_with_bags_of_keypoints 10.1007/978-981-16-2123-9_36Shukla, J.S., Rastogi, K., Patel, H., Jain, G., Sharma, S. (2022). Bag of Visual Words Methodology in Remote Sensing—A Review. Proceedings of the International e-Conference on Intelligent Systems and Signal Processing, 475-486. https://doi.org/10.1007/978-981- 16-2123-9_36 BelgiumTS- Belgian Traffic Sign Dataset, ETH shared data. (Accesed on 31/03/2022). https://btsd.ethz.ch/shareddata/ Karypidis, E., Mouslech, S.G., Skoulariki, K., Gazis, A. BelgiumTS - Belgian Traffic Sign Dataset custom version, Mendeley Data (Accesed on 31/03/2022). www.doi.org/10.17632/kfngc5jxds.3 10.1023/b:visi.0000029664.99615.94Lowe, D.G. (2004). Distinctive image features from scale-invariant keypoints. International Journal Of Computer Vision, 60(2), 91-110. https://doi.org/10.1023/B:VISI.0000029664.99615.94 10.1002/eng2.12170Gazis, A., Katsiri, E. (2020). A wireless sensor network for underground passages: Remote sensing and wildlife monitoring. Engineering Reports, 2(6), e12170. https://doi.org/10.1002/eng2.12170 MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, 1(14), 281- 297. https://www.semanticscholar.org/paper/Somemethods-for-classification-and-analysis-ofMacQueen/ac8ab51a86f1a9ae74dd0e4576d1a019f5e6 54ed 10.1109/tit.1967.1053964Cover, T., Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964 10.2307/1403797Fix, E., Hodges, J. L. (1989). Discriminatory analysis. Nonparametric discrimination: Consistency properties. International Statistical Review/Revue Internationale de Statistique, 57(3), 238-247. https://apps.dtic.mil/sti/citations/ADA800276 10.1016/j.patcog.2017.09.038Gallego, A.J., Calvo-Zaragoza, J., Valero-Mas, J.J., Rico-Juan, J.R. (2018). Clustering-based k-nearest neighbor classification for large-scale data with neural codes representation. Pattern Recognition, 74, 531- 543. https://doi.org/10.1016/j.patcog.2017.09.038 10.1016/j.procs.2017.12.099Nerurkar, P., Shirke, A., Chandane, M., Bhirud, S. (2018). Empirical analysis of data clustering algorithms. Procedia Computer Science, 125, 770-779. https://doi.org/10.1016/j.procs.2017.12.099 10.5120/11430-6785Singh, A., Yadav, A., Rana, A. (2013). K-means with Three different Distance Metrics. International Journal of Computer Applications, 67(10). http://dx.doi.org/10.5120/11430-6785 10.1088/1757-899x/725/1/012128Mughnyanti, M., Efendi, S., Zarlis, M. (2020). Analysis of determining centroid clustering x-means algorithm with davies-bouldin index evaluation. In IOP Conference Series: Materials Science and Engineering, 725(1), 012128. https://doi.org/10.1088/1757- 899X/725/1/012128 10.1007/bf00994018Cortes, C., Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297. https://doi.org/10.1007/BF00994018 10.1007/3-540-44673-7_12Evgeniou T., Pontil M. (2001) Support Vector Machines: Theory and Applications. In: Paliouras G., Karkaletsis V., Spyropoulos C.D. (eds) Machine Learning and Its Applications. ACAI 1999. Lecture Notes in Computer Science, 2049. https://doi.org/10.1007/3-540-44673-7_12 10.1016/j.neucom.2017.07.070Tsekouras, G.E., Trygonis, V., Maniatopoulos, A., Rigos, A., Chatzipavlis, A., Tsimikas, J., Velegrakis, A. F. (2018). A Hermite neural network incorporating artificial bee colony optimization to model shoreline realignment at a reef-fronted beach. Neurocomputing, 280, 32-45. https://doi.org/10.1016/j.neucom.2017.07.070 10.1016/j.ins.2020.12.042Zhang, X., Wang, J., Wang, T., Jiang, R., Xu, J., Zhao, L. (2021). Robust feature learning for adversarial defense via hierarchical feature alignment. Information Sciences, 560, 256-270. https://doi.org/10.1016/j.ins.2020.12.042 10.1109/acpr.2015.7486599Liu, S., Deng, W. (2015). Very deep convolutional neural network based image classification using small training sample size. In IAPR Asian conference on pattern recognition (ACPR), 730-734. https://arxiv.org/abs/1409.1556 10.1007/s00138-011-0391-3Timofte, R., Zimmermann, K., Van Gool, L. (2014). Multi-view traffic sign detection, recognition, and 3D localisation. Machine vision and applications, 25(3), 633-647. https://doi.org/10.1007/s00138-011-0391-3 Misra, D. (2019). Mish: A self regularized nonmonotonic neural activation function. arXiv preprint arXiv:1908.08681. https://arxiv.org/abs/1908.08681 Goodfellow, I., Bengio, Y., Courville, A. (2017). Deep learning (adaptive computation and machine learning series). Cambridge Massachusetts, 321-359. Zhai, S., Cheng, Y., Lu, W., Zhang, Z. (2016). Doubly convolutional neural networks. arXiv preprint arXiv:1610.09716. https://arxiv.org/abs/1610.09716 Graham, B. (2014). Fractional max-pooling. arXiv preprint arXiv:1412.6071. https://arxiv.org/abs/1412.6071 Nair, V., Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. Icml. https://openreview.net/forum Glorot, X., Bordes, A., Bengio, Y. (2011). Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 315-323. https://proceedings.mlr.press/v15/glorot11a.html 10.3390/info12120513Maniatopoulos, A., Mitianoudis, N. (2021). Learnable Leaky ReLU (LeLeLU): An Alternative AccuracyOptimized Activation Function. Information, 12(12), 513. http://dx.doi.org/10.3390/info12120513 Ramachandran, P., Zoph, B., Le, Q.V. (2017). Swish: a self-gated activation function. arXiv preprint arXiv:1710.05941. https://arxiv.org/abs/1710.05941v1 Mish, M.D. (2020). A Self Regularized NonMonotonic Activation Function. arXiv preprint arXiv:1908.08681. https://arxiv.org/abs/1908.08681 Kingma, D.P., Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://arxiv.org/abs/1412.6980 Ioffe, S., Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, 448-456. http://proceedings.mlr.press/v37/ioffe15.html Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The Journal Of Machine Learning Research, 15(1), 1929-1958. https://jmlr.org/papers/v15/srivastava14a.html 10.1007/3-540-44503-x_27Aggarwal, C.C., Hinneburg, A., Keim, D.A. (2001). On the surprising behavior of distance metrics in high dimensional space. In International conference on database theory, 420-434. https://doi.org/10.1007/3- 540-44503-X_27 10.1007/978-3-642-15702-8_38Hou J., Kang J., Qi N. (2010) On Vocabulary Size in Bag-of-Visual-Words Representation. In: Qiu G., Lam K.M., Kiya H., Xue XY., Kuo CC.J., Lew M.S. (eds) Advances in Multimedia Information Processing - PCM 2010. Lecture Notes in Computer Science, 6297. https://doi.org/10.1007/978-3-642-15702-8_38 10.1109/access.2020.3046999Kuru, K., Khan, W. (2020). A framework for the synergistic integration of fully autonomous ground vehicles with smart city. IEEE Access, 9, 923-948. https://doi.org/10.1109/ACCESS.2020.3046999 10.1109/access.2022.3145972Butt, F. A., Chattha, J. N., Ahmad, J., Zia, M.U., Rizwan, M., Naqvi, I.H. (2022). On the Integration of Enabling Wireless Technologies and Sensor Fusion for Next-Generation Connected and Autonomous Vehicles. IEEE Access, 10, 14643-14668. https://doi.org/10.1109/ACCESS.2022.3145972 10.37394/23201.2021.20.13Yeomans, J.S. (2021). A Multicriteria, Bat Algorithm Approach for Computing the Range Limited Routing Problem for Electric Trucks. WSEAS Transactions on Circuits and Systems, 20, 96-106. www.doi.org/10.37394/23201.2021.20.13 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, 5998-6008. https://papers.nips.cc/paper/2017/hash/3f5ee243547de e91fbd053c1c4a845aa-Abstract.html Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., Dosovitskiy, A. (2021). Do vision transformers see like convolutional neural networks?. Advances in Neural Information Processing Systems, 34. https://proceedings.neurips.cc/paper/2021/hash/652cf3 8361a209088302ba2b8b7f51e0-Abstract.html Li, S., Chen, X., He, D., Hsieh, C.J. (2021). Can Vision Transformers Perform Convolution?. arXiv preprint arXiv:2111.01353. https://arxiv.org/abs/2111.01353 10.1109/cvpr42600.2020.00737Newell, A., Deng, J. (2020). How useful is selfsupervised pretraining for visual tasks? Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7345-7354.