Introducing New Outlier Detection Method Using Robust Statistical Distance in Water Quality Data |
---|
학술지명 7th IWA-ASPIRE Conference (Malaysia)
저자 채선하,김성수,윤석민,박노석
발표일 2017-09-12
|
Currently, various water qualities are being measured in real time, to monitor source water and the drinking and waste water processed by treatment plants. However, there are likely to be various potential outliers in the water quality dataset due to replacement of consumables and equipment calibration; and missing data from mechanical malfunctions, etc. Outlier detection method based on multivariate analysis, which has been used generally, is an approach to detecting outliers using the chi-squared distribution and the mahalanobis distance derived from the multivariate Gaussian distribution. However, mahalanobis distance is sensitive to the effects of potential outliers and extreme values distributed away from the cluster mean. Accordingly, we adopted robust distance based on minimum covariance determinant estimators to minimize the effects of the potential outliers and extreme values. In addition, the modified threshold of the chi-squared distribution and the threshold calculation methodology were applied to reduce the effects of data size on detecting outliers using robust distance and the chi-squared distribution. |