Multivariate inlier and outlier data detection by using of data mining algorithms  Case study: Geochemical data at 1:100000 Roum sheet in South Khorasan

Geranian, Hamid; Khajeh Miry, Zahra

doi:10.22034/ijme.2020.37386

Multivariate inlier and outlier data detection by using of data mining algorithms Case study: Geochemical data at 1:100000 Roum sheet in South Khorasan

Document Type : research - paper

Authors

Hamid Geranian ¹

Zahra Khajeh Miry ²

¹ Assistant of Professor, Department of Mining Engineering, Birjand University of Technology

² Expert of Industry, Mine & Trade Organization of South Khorasan Province

10.22034/ijme.2020.37386

Abstract

In this paper, four data mining algorithms, namely, kernel density estimation, local outlier factor, OPTICS-OF and SVDD are used to determine multivariate outlier data. So, stream sediment geochemical data, in 1:100000 Roum sheet, with 902⨉41 matrix dimensions have been utilized. Replacing censored data, converting the data set to an open number system and finally standardizing them are used as pre-processing methods. Results show that in error sample detection approach, 10 samples that have the highest outlier probability, and are present in equal numbers in the four mentioned algorithms, can be considered for more study as replicate sampling. In non-normal sample detection approach, form 150 selected samples, 74.5% of samples are detected as outliers in the four mentioned algorithms, and 16.1 and 9.4 percent are recognized as the outlier data in one and two of the aforementioned algorithms, respectively. Determining of replicate sampling, calculating location and scatter matrices in multivariate robust statistics after eliminating non-normal samples and geochemical anomaly detection are suggested as the applications of these algorithms.

Keywords

Kernel Density Estimation

Local Outlier Factor

OPTICS-OF Algorithm

SVDD Algorithm

Roum Sheet

20.1001.1.17357616.1398.14.45.6.3

منابع
1. روشنی‌رودسری، پریسا؛ مختاری، احمدرضا؛ طباطبائی، سید حسن؛ 1393؛ "بررسی آنالیز ژئوشیمیایی عناصر در سیتم عدیی باز و بسته؛ مطالعه موردی: کانسار مس کوه‌پنچ (کرمان)"، نشریه علمی- پژوهشی روش‌های تحلیلی و عددی در مهندسی معدن، دوره دوم، شماره 4، صفحه 46 تا 58.
2. کیانپوریان، ص.؛ اسدی هارونی، ه.؛ افشاری، س.؛ فرهمندیان، م.، 1393؛ "جداسازی داده‌های خارج از رده به روش تک متغیره و چند متغیره در داده‌های ژئوشیمی محدوده طلای اپی‌ترمال ساری‌گونای"، نشریه مهندسی معدن، دوره 9، شماره 25، صفحه 85 تا 96.
3. گرانیان، حمید؛ خواجه‌میری، زهرا؛ 1396؛ "کاربرد برآوردگرهای مقاوم در تعیین داده‌های خارج از ردیف؛ مثال موردی: داده‌های ژئوشیمیایی منطقه شاه سلیمان علی در استان خراسان جنوبی"، نـشریه علمی- پژوهشی روش‌های تحلیلی و عددی در مهندسی معدن، شماره 14، صفحه 73 تا 85.
  1. Aggarwal, C. C., 2016. "Outlier analysis", Second Edition. Springer, New York, 54p.
  2. Ahmed, T., 2009. "Online Anomaly Detection using KDE". IEEE "GLOBECOM" 2009 proceedings, p. 4244-4148.
  3. Ahn, J., Lee, M.H., Lee, J.A., 2019. "Distance-based outlier detection for high dimension, low sample size data". Journal of Applied Statistics 46, 13-29.
  4. An, W., Liang, M., Liu, H., 2014. "An improved one-class support vector machine classifier for outlier detection". Journal of Mechanical Engineering Science, 1-9.
  5. Ankerst, M., Breunig, M.M., Kriegel, H.P., Sander, J., 1999. "OPTICS: Ordering Points to Identify the Clustering Structure", in Proceedings of the ACM SIGMOD International Conference on Management of Data, vol. 28, no. 2, pp. 49-60.
  6. Maronna, R.A., Martin, R.D., Yohai, V.J., Salibian-Barrera, M., 2019. "Robust Statistics: Theory and Methods", John Wiley & Sons., 464 p.
  7. Behera, S., Rani, R., 2016. "Comparative analysis of density-based outlier detection techniques on breast cancer data using Hadoop and map reduce". International Conference on Inventive Computation Technologies, India.
  8. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J., 2000. "LOF: identifying density-based local outliers". In Proc. of ACM SIGMOD International Conference on Management of Data, pages 93–104.
  9. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J., 1999. "OPTICS-OF: Identifying Local Outliers". in Proceedings of the Third European Conference on Principles of Data Mining and Knowledge Discovery, 262-270.
  10. Deng, X., Wang, L., 2018. "Modified kernel principal component analysis using double-weighted local outlier factor and its application to nonlinear process monitoring". ISA Transactions 72, 218-228.
  11. Febriana, N.L., Sitanggang, I.S., 2017. "Outlier Detection on Hotspot Data in Riau Province using OPTICS Algorithm". IOP Conference Series: Earth and Environmental Science 58 (2017) 012004.
  12. Filzmoser, P., Garrett, R.G., Reimann, C., 2005. "Multivariate outlier detection in exploration geochemistry". Computers & Geosciences 31, 579–587.
  13. Filzmoser, P., Hron, K., Reimann, C., 2012. "Interpretation of multivariate outliers for compositional data". Computers & Geosciences 39, 77–85.
  14. Filzmoser, P., Hron, K., Reimann, C., 2009. "Principal component analysis for compositional data with outliers". Environmetrics 20, 621–632.
  15. Filzmoser, P., Hron, K., Reimann, C., Garrett, R., 2009. "Robust factor analysis for compositional data". Computers & Geosciences 35, 1854–1861.
  16. Han, J., Kamber, M., Pei, J., 2012. "Data Mining: Concepts and Techniques", Morgan Kaufmann, 740 p.
  17. Latecki, L.J., Lazarevic, A., Pokrajac, D., 2007. "Outlier Detection with Kernel Density Functions". In: Perner P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science, vol 4571. Springer, Berlin, Heidelberg.
  18. Lima, R., 2013. "Outlier detection with kernel density functions in monitoring the Istat Lfs data production processes", Electronic Journal of Applied Statistical Analysis 6(1), 118 – 129.
  19. Ma, Y., Shi, H., Ma, H., Wang, M., 2013. "Dynamic process monitoring using adaptive local outlier factor". Chemometrics and Intelligent Laboratory Systems 127, 89–101.
  20. Ranga Suri, N.N.R., Murty, N., Athithan, G., 2019. "Outlier Detection: Techniques and Applications: A Data Mining Perspective", Springer International Publishing, 216 p.
  21. Schölkopf, B., Williamson, R., Smola, A., Shawe-Taylor, J., 1999. "SV estimation of a distribution's support", in: Advances in Neural Information Processing Systems, Colorado, USA, pp.582–588.
  22. Sreevani, R., Murthy, C.A., 2016. "On bandwidth selection using minimal spanning tree for kernel density estimation". Computational Statistics and Data Analysis 102, 67–84.
  23. Sudha, P., Krithigadevi, K., 2014. "Outlier detection using high dimensional dataset for comparison of clustering algorithms". International Journal of Advanced Research in Computer Science & Technology 2(3), 283-288.
  24. Tax, D.M.J., Duin, R.P., 1999. "Support vector domain description". Pattern Recognition Letters 20(11-13), 1191-1199.
  25. Terrell, G.R., Scott, D.W., 1992. "Variable kernel density estimation". The Annals of Statistics 20(3), 1236-1265.
  26. Wang, Y.F., Yu, J., Su, G.P., Qian, Y.R., 2019. "New outlier detection method based on OPTICS". Sustainable Cities and Society 45, 197-212.
  27. Xu, Y., Xu, N., Feng, X., 2016. "A New Outlier Detection Algorithm Based on Kernel Density Estimation for ITS". The IEEE International Conference on Internet of Things, Green Computing and Communications, Cyber, Physical and Social Computing and Smart Data. Chengdu, China.
  28. Zheng, S., 2016. "Smoothly approximated support vector domain description". Pattern Recognition 49, 55–64.
  29. Zhou, S., Zhou, K., Wang, J., Yang, G., Wang, S., 2017. "Application of cluster analysis to geochemical compositional data for identifying ore-related geochemical anomalies", Frontiers of Earth Science 12(3), 491–505.