All Issue

2025 Vol.24, Issue 2

Research Article

A Study on the Determination of Outlier Detection Techniques for the Automation of Dam Monitoring Data Analysis by Instrument Type

댐 계측데이터 분석의 자동화를 위한 계측기기별 이상치 탐지 방법 결정을 위한 연구

Seong-Kyu Yun1
,
Hyeonsu Yun2
,
Jae-Soon Park3
,
Tae-Hyung Lee4
,
Hye-Geun Jin5
,
Gichun Kang6*
윤 성규1
,
윤 현수2
,
박 재순3
,
이 태형4
,
진 혜근5
,
강 기천6*
1Member, Research Professor, Engineering Research Institute, Gyeongsang National University, 501 Jinjudae-Ro, Jinju 52828, Republic of Korea
2Member, Ph.D. Candidate, Department of Civil Engineering, College of Engineering, Gyeongsang National University, 501 Jinjudae-Ro, Jinju 52828, Republic of Korea
3Manager, Institute Technology, Korea Authority of Land & Infrastructure Safety, 24 Ena-ro, 128 beon-gil, Jinju 52856, Republic of Korea
4Principal Research Manager, Institute Technology, Korea Authority of Land & Infrastructure Safety, 24 Ena-ro, 128 beon-gil, Jinju 52856, Republic of Korea
5Senior Manager, Infrastructure Management Division, Korea Authority of Land & Infrastructure Safety, 24 Ena-ro, 128 beon-gil, Jinju 52856, Republic of Korea
6Member, Associate Professor, Department of Civil Engineering, College of Engineering, Gyeongsang National University, 501 Jinjudae-Ro, Jinju 52828, Republic of Korea
*Corresponding Author
30 June 2025. pp. 1-13
PDF XML Citation
Abstract

Dams, special bridges, tunnels, and other social infrastructure play a crucial role in national water resource management, maintaining traffic flow, ensuring safe transportation, and forming the backbone of the national economy and public life. These infrastructures require periodic precise safety inspections to continuously assess their structural stability and functional reliability. However, the analysis of measurement data required for preparing detailed safety inspection reports involves a vast amount of data and high complexity, necessitating significant manpower and time. In particular, the preprocessing stage of detecting and removing outliers in measurement data is prone to errors, reducing work efficiency. To address these challenges, the automation of measurement data analysis is necessary. An automated measurement data analysis system enhances the consistency and efficiency of report generation while reducing labor and costs. Therefore, this study focuses on techniques for detecting and removing outliers in raw data as a step toward establishing an automated measurement data analysis system. Univariate and multivariate outlier detection methods were reviewed and compared. Based on the characteristics of each instrument, the most appropriate detection technique was selected. For pore water pressure gauges, the Local Outlier Factor (LOF) method was identified as the most suitable approach, showing the greatest improvement in correlation after outlier removal. The proposed methodology can be applied to other instruments as well and will serve as a foundation for improving the reliability and precision of automated monitoring data analysis systems.

Keywords
Dam
Outlier
Measurement data
Univariate analysis
Multivariate analysis
Correlation

댐, 특수교, 터널 등 사회기반시설은 국가의 수자원 관리, 교통 흐름 유지, 안전한 이동 등을 담당하며 국가 경제와 국민 생활의 핵심을 이루는 중요한 인프라이다. 이러한 사회 기반시설은 주기적인 정밀안전진단을 통해 구조적 안정성과 기능적 신뢰성을 지속적으로 점검해야 한다. 하지만 정밀안전진단보고서 작성에 필요한 계측자료 분석 과정은 방대한 데이터량과 복잡성으로 인해 많은 인력과 시간이 소요되고, 특히 계측자료의 이상치를 탐지하고 제거하는 전처리과정에서 오류 가능성이 높아 작업의 효율성을 저하시키고 있다. 이러한 문제를 해결하기 위해 계측자료 분석의 자동화를 도입하는 것이 필요하다. 자동화된 계측자료 분석 시스템은 보고서 작성의 인력과 비용을 절감하며, 효율성과 일관성을 높일 수 있다. 따라서 본 연구에서는 자동화된 계측자료 분석 시스템을 구축하는 과정의 하나인 원시데이터의 이상치를 탐지 및 제거하는 기법에 대한 연구를 수행하였다. 본 연구에서는 단변량 및 다변량 이상치 탐지 기법을 비교 분석하고, 계측기별 특성에 따라 가장 적합한 기법을 도출하였다. 특히, 간극수압계를 대상으로 한 적용 결과, LOF(Local Outlier Factor) 기법이 가장 높은 상관성 증가를 보이며 최적 기법으로 도출되었고, 이는 자동화 시스템의 기본값으로 제안되었다. 이와 같은 방법론은 향후 다양한 계측기에도 동일하게 적용될 수 있으며, 계측자료 분석의 정밀성과 신뢰성을 향상시키는 기반이 될 수 있다.

References
1

Breunig, M. M., Kriegel, H. P., Ng, R. T. and Sander, J. (2000), "LOF: Identifying density-based local outliers", Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp.93-104.

10.1145/342009.335388
2

Evans, J. D. (1996), Straightforward Statistics for the Behavioral Sciences, Brooks, Cole Publishing.

3

Hodge, V. J. and Austin, J. (2004), "A survey of outlier detection methodologies", Artificial Intelligence Review, Vol.22, No.2, pp.85-126.

10.1023/B:AIRE.0000045502.10941.a9
4

Irazábal, J., Salazar, F., Silva-Cancino, N. and Vicente, D. J. (2025), "Detection of outliers in dam monitoring time series with autoencoders", Structural Health Monitoring, (in press).

10.1007/s13349-025-00910-4
5

Jeon, H. C., Yoon, S. G., Kim, J. S., Lim, E. S. and Kang, G. C. (2021), "A case study on optimal analysis techniques for key instrumentation data in earthfill dam safety management", Journal of the Korean Geo-Environmental Society, Vol.22, No.5, pp.43-53. (in Korean)

6

Jeon, J. S. (2018), "Integrated outlier evaluation and verification for multiple monitoring data", Journal of Korean Geo- Environmental Society, Vol.19, No.1, pp.5-14. (in Korean)

7

Jeon, J. S., Koo, J. G. and Park, C. M. (2015), "Outlier detection of time-series measurement data using rule-based and correlation analysis methods", Journal of Korean Geo- Environmental Society, Vol.16, No.5, pp.43-53. (in Korean)

10.14481/jkges.2015.16.5.43
8

Kim, D. H., Yoo, S. E., Lee, B. J., Kim, K. T. and Yoon, H. Y. (2019), "Efficient data preprocessing for machine learning", Proceedings of the 59th Winter Conference of the Korean Society of Computer Information, pp.123-130. (in Korean)

9

K-water (2024a). Dam instrumentation reliability assessment service report. Korea Water Resources Corporation. (in Korean)

10

K-water (2024b), Dam Instrumentation Reliability Assessment Service Report. Korea Water Resources Corporation. (in Korean)

11

​Lee, Y. H. (2021), "A study on effective machine learning methods applying data discretization and hierarchical clustering algorithms", Proceedings of the Korean Institute of Information Scientists and Engineers Conference, pp.45-52.

12

Liu, C., Pan, J. and Wang, J. (2023), "An LSTM-based anomaly detection model for the deformation of concrete dams", Automation in Construction, Vol.153, No.104971.

13

Liu, F. T., Ting, K. M. and Zhou, Z. H. (2008), "Isolation forest", Proceedings of the 8th IEEE International Conference on Data Mining, pp.413-422.

10.1109/ICDM.2008.17
14

Razali, N. M. and Wah, Y. B. (2011), "Power Comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson- Darling Tests", Journal of Statistical Modeling and Analytics, Vol.2, No.1, pp.21-33.

15

Xiang, H., Zhu, J., Zhang, Y. and Zhang, Y., et al. (2023), "Application of Transformer-based anomaly detection in dam structural strong motion monitoring data", Sensors, Vol.23, No.8, No.3501.

16

Yun, S. K., Kim, J. S., Im, E. S. and Kang, G. C. (2022), "Behavior of Porewater Pressures in an Earth Dam by Principal Component Analysis", Water, Vol.14, No.4. Article No.672.

10.3390/w14040672
17

Zhang, Y. and Jiang, L. (2022), "Machine learning methods for outlier detection in structural health monitoring: A review", Structural Health Monitoring, Vol.21, No.5, pp.1594-1612.

Information
  • Publisher :Korean Geosythetics Society
  • Publisher(Ko) :한국지반신소재학회
  • Journal Title :Journal of the Korean Geosynthetics Society
  • Journal Title(Ko) :한국지반신소재학회 논문집
  • Volume : 24
  • No :2
  • Pages :1-13
  • Received Date : 2025-03-24
  • Revised Date : 2025-04-15
  • Accepted Date : 2025-04-25
  • DOI :https://doi.org/10.12814/jkgss.2025.24.2.001
Journal Informaiton Journal of the Korean Geosynthetics Society Journal of the Korean Geosynthetics Society
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close