Detecting point anomalies in energy consumption data using unsupervised machine learning methods

Cover Page

Cite item

Full Text

Abstract

The paper describes studies on detecting point anomalies in energy consumption data using two different data sets as an example. Methods for constructing typical energy consumption patterns are considered and the authors' method for constructing a typical daily energy consumption profile is presented. To conduct numerical experiments, the authors selected 21 unsupervised machine learning methods suitable for detecting point anomalies. Based on the results of numerical experiments, the methods that most successfully coped with the task of detecting point anomalies were noted. Particular attention in the work was paid to methods that do not require additional parameters and modern, promising methods based on artificial neural networks. According to the test results, the best algorithms were statistical algorithms based on constructing histograms. One of the main problems addressed in the work is the problem of setting the contamination parameter for each considered algorithm. One of the solutions to this problem is the use of threshold algorithms. It is shown that if the original algorithm does not detect anomalies well enough (the contamination parameter is not configured), then the use of threshold algorithms can significantly improve the accuracy of anomaly detection. Threshold algorithms are noted, the use of which for the tasks of analyzing anomalies in energy consumption data, most often ensures an increase in accuracy. Threshold algorithms can be applied both to the results of individual anomaly detection algorithms and to the results of ensembles of algorithms obtained using various combination strategies.

About the authors

Oleg Yur'evich Maryasin

Yaroslavl State Technical University

Email: maryasin2003@list.ru
Yaroslavl

Leonid Igorevich Tihomirov

Yaroslavl State Technical University

Email: lenusscik@yandex.ru
Yaroslavl

References

  1. МАРЬЯСИН О.Ю., ЛУКАШОВ А.И., ТИХОМИРОВ Л.И.Обнаружение аномальных отклонений от типового профи-ля энергопотребления // Математические методы в техникеи технологиях. – 2022. – №8. – P. 108–113.
  2. О внесении изменений в некоторые акты ПравительстваРоссийской Федерации по вопросам совершенствования ор-ганизации учета электрической энергии. ПостановлениеПравительства Российской Федерации от 29 июня 2020 г. –№950. – 32 с.
  3. ALMARDENY Y., BOUJNAH N., CLEARY F. A NovelOutlier Detection Method for Multivariate Data // IEEE Trans.on Knowledge and Data Engineering. – Vol. 32. – 2020. –P. 1–13.
  4. BURGESS C.P., HIGGINS I., PAL A. et al. Understandingdisentangling in β-VAE // arXiv:1804.03599v1. – 2018. –P. 1–11.
  5. CHANDOLA V., BANERJEE A., KUMAR V. AnomalyDetection: A Survey // ACM Computing Surveys. – 2009. –Vol. 41. – P. 1–58.
  6. DUMOULIN V., BELGHAZI I., POOLE B. et al. Adversariallylearned inference // arXiv:1606.00704v3. – 2017. – P. 1–18.
  7. GitHub – KulikDM/pythresh: Outlier Detection Thresholding. –URL: https://github.com/KulikDM/pythresh (дата обращения:30.08.2024).
  8. GitHub – yzhao062/pyod: A Python Library for Outlier andAnomaly Detection, Integrating Classical and Deep LearningTechniques. – URL: https://github.com/yzhao062/pyod (датаобращения: 30.08.2024).
  9. GOLDSTEIN M., DENGEL A. Histogram-based Outlier Score(HBOS): A fast Unsupervised Anomaly Detection Algorithm //Conference KI-2012. – 2012. – P. 1–6.
  10. GULATI M., ARJUNAN P. LEAD1.0: A large-scale annotateddataset for energy anomaly detection in commercial buildings //arXiv:2203.17256v1. – 2022. – P. 1–5.
  11. HABEN S., SINGLETON C., GRINDROD P. Analysis andclustering of residential customers energy behavioral demandusing smart meter data // IEEE Trans. on Smart Grid. – 2016. –Vol. 1. – P. 136–144.
  12. HIMEUR Y., GHANEM K., ALSALEMI A. et al. Artificialintelligence based anomaly detection of energy consumptionin buildings: A review, current trends and new perspectives //Applied Energy. – 2021. – Vol. 287. – P. 1–26.
  13. KINGMA D.P., WELLING M. Auto-Encoding VariationalBayes // Int. Conf. on Learning Representations. – 2013. –P. 1–14.
  14. LI C., LIU H., CHEN C. et al. ALICE: Towards UnderstandingAdversarial Learning for Joint Distribution Matching //arXiv:1709.01215v2. – 2017. – P. 1–22.
  15. LI K., MA Z., ROBINSON D. et al. Identification of typicalbuilding daily electricity usage profiles using Gaussian mixturemodel-based clustering and hierarchical clustering // AppliedEnergy. – 2018. – Vol. 231. – P. 331–342.
  16. LI K., YANG R.J., ROBINSON D. et al. An agglomerativehierarchical clustering-based strategy using Shared NearestNeighbours and multiple dissimilarity measures to identifytypical daily electricity usage profiles of university librarybuildings // Energies. – 2019. – Vol. 174. – P. 735–748.
  17. LINDEMANN B., MASCHLER B., SAHLAB N. et al. Asurvey on anomaly detection for technical systems using LSTMnetworks // Computers in Industry. – 2021. – Vol. 131. – P. 1–11.
  18. LI S., HAN Y., YAO X. et al. Electricity theft detection inpower grids with deep learning and random forests // Journalof Electrical and Computer Engineering. – 2019. – P. 1–12.
  19. LIU X., DING Y., TANG H. et al. A data mining-based framework for the identification of daily electricityusage patterns and anomaly detection in building electricityconsumption data // Energy and Buildings. – 2021. – Vol. 231. –P. 1–22.
  20. LI Z., ZHAO Y., BOTTA N. et al. COPOD: Copula-BasedOutlier Detection // arXiv:2009.09463v1. – 2020. – P. 1–6.
  21. LI Z., ZHAO Y., HU X. et al. ECOD: UnsupervisedOutlier Detection Using Empirical Cumulative DistributionFunctions // arXiv:2201.00382v3. – 2022. – P. 1–13.
  22. MARYASIN O.YU., TIHOMIROV L. Analysis of Point andCollective Anomalies in Energy Consumption Data // Int.Russian Automation Conf. (RusAutoCon–2023). – 2023. –P. 431–436.
  23. PEVNY T. Loda: Lightweight on-line detector of anomalies //Mach Learn. – 2016. – Vol. 102. – P. 275–304.
  24. RAJABI A., ESKANDARI M., GHADI M.J. et al. Acomparative study of clustering techniques for electrical loadpattern segmentation // Renewable and Sustainable EnergyReviews. – 2020. – Vol. 120. – P. 1–20.
  25. RASANEN T., KOLEHMAINEN M. Feature-Based Clusteringfor Electricity Use Time Series Data // Int. Conf. on Adaptiveand Natural Computing Algorithms. – 2009. – P. 401–412.
  26. RUFF L., VANDERMEULEN R.A., GORNITZ N. et al.Deep One-Class Classification // 35th Int. Conf. on MachineLearning. – 2018. – P. 1–10.
  27. SCHLEGL T., SEEBOCK P., WALDSTEIN S.M. et al.Unsupervised anomaly detection with generative adversarialnetworks to guide marker discovery // Int. Conf. on InformationProcessing in Medical Imaging. – 2017. – P. 146–157.
  28. SIAL A., SINGH A., MAHANTI A. Detecting anomalousenergy consumption using contextual analysis of smart meterdata // Wireless Networks. – 2019. – P. 1–18.
  29. Thresholding Outlier Detection Scores with PyThresh. –URL: https://towardsdatascience.com/thresholding-outlier-detection-scores-with-pythresh-f26299d14fa (дата обращения:30.08.2024).
  30. VERLEYSEN M., FRANCOIS D. The curse of dimensionalityin data mining and time series prediction // Int. Work-Conf. onArtificial Neural Networks. – 2005. – P. 758–770.
  31. WANG X., ZHAO T., LIU H., HE R. Power consumptionpredicting and anomaly detection based on long short-termmemory neural network // IEEE 4th Int. Conf. on CloudComputing and Big Data Analysis. – 2019. – P. 487–491.
  32. ZENATI H., ROMAIN M., FOO C.S. et al. AdversariallyLearned Anomaly Detection // arXiv:1812.02288v1. – 2018. –P. 1–11.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).