Knowledge from water data
Tools to support the operation and management of water supply services.
More
Tasks
The project consists of five major tasks:
analytics
exploratory data analysis
timeline
water consumption predictive analysis
share
pipe bursts spatial location
multiline_chart
anomalous events identification and anticipation
widgets
user guidelines and decision support tools
Partners
Participant Water Utilities:
-
Infraquinta
Urban tipology
Floating population
75 km of pipes
-
CM Barreiro
Urban tipology
About 80 000 inhabitants
310 km of pipes
-
EMAS Beja
Rural tipology
About 34 000 inhabitants
272 km of pipes
Team
The following Academic Institutions participated in the WISDom project:
Instituto Politécnico de Setúbal
Prof. Ana Mendes
Prof. Cédric Grueau
Prof. Nelson Carriço
Prof. Raquel Barreira
Prof. Sérgio Fernandes
André Antunes
Bruno Ferreira
Carlos Ascensão
Diogo Alves
Francisco Santos
Instituto Superior Técnico
Prof. Conceição Amado
Prof. Didia Covas
Prof. José Rui Figueira
Laura Monteiro
Bruna Mason
Gilberto Silva
Miguel Capelo
Ricardo Sousa
Tiago Dias
Tiago Santos
INESC-ID
Prof. Jacinto Estima
Prof.Rui Henriques
Prof.Susana Vinga
Margarida Costa
Susana Gomes
Publications
Relevant publications in the scope of the WISDom project.
Water Quality Analysis in Water Distributions Systems using Multivariate Statistics and Water Quality Indexes (2021)
Rodrigo Mendes Ribeiro
Thesis to obtain the Master of Science Degree in Mathematics and Applications. Supervisor(s): Prof. Conceição Amado, Dra. Laura Monteiro
Over the years water utilities have gathered information regarding the state of the quality of the water in its distribution systems. This quality is evaluated by a number of water parameters tested along the year in different sites, assuring the quality of the water provided to the customers is in conformity to the legislation. Investigating three case studies (Barreiro, Beja and Infraquinta), the goal is to perceive relations between the water parameters, detecting correlations and trends in the parameters variations' over time and along the water network. Another objective was to quantify this quality through water quality indexes (WQIs), where three newly created indexes are proposed. In order to detect simple correlations between the parameters the Pearson correlation matrix is used. As it has become popular in water distribution system analysis, an unsupervised Artifical Neural Network (ANN) class, the Kohonen self-organizing maps (SOMs) are a data mining tool being used in this field that allows a better understanding and a clearer view of the data through dimensionality reduction of the feature space to a 2D plane keeping the topology of the original data. Some other unsupervised methods such as Principal Component Analysis (PCA) and clustering techniques (KMeans and Hierarchical Clustering) were employed. As to the WQIs, two of them are adaptations from already existing work and the third one is a completely new approach using differences of the actual concentration of the parameter to its parametric value. The results obtained provided further insight of the water quality in each network, concluding that the water quality is of a very high standard in all cases.
Descriptive and Predictive Modeling of Water Distribution Network Dynamics using Multivariate Time Series Data (2021)
Susana Chambel Gonçalves Correia Gomes
Thesis to obtain the Master of Science Degree in Information Systems and Computer Engineering. Supervisor(s): Prof. Rui Miguel Carrasqueiro Henriques, Prof. Susana de Almeida Mendes Vinga Martins
Water distribution networks (WDNs) are hydraulic infrastructures that provide a continuous supply of pressurized safe drinking water to all consumers, playing an important role in public health. Leakages cause service interruptions, waste resources, and compromise water quality. Although we can find many methods to support the monitoring and control of WDNs, they exhibit limited ability to detect anomalies and are not yet consistently applied to Portuguese WDNs. We show that it is possible to (1) describe the dynamics of a WDN through spatiotemporal correlation analysis of pressure and volumetric flowrate sensors, and (2) analyze disruptions on the expected correlation to detect burst leakage dynamics using standard classifiers. Our approach is promising in a synthetic setting and offers initial support towards leakage detection in real WDNs despite the presence of highly irregular consumption patterns, a limited number of recorded leakages, and highly heterogeneous leakage profiles. We discovered that the disruption caused by leakages is higher shortly after the burst. Furthermore, a comprehensive pairing of heterogeneous sensors and data balancing in the real setting is also promising. Our results suggest that it is important to access data from WDNs with good sensor coverage and complete information about leakages. Accordingly, we believe that our WDN would benefit a lot from sensor expansion and relocation. Lastly, given the simplicity, novelty, and accuracy of the proposed correlation-based principles for anomaly detection in heterogeneous and georeferenced time series, we anticipate our work to contribute to the study and development of automated leakage detection in Portuguese WDNs.
Autonomous Time Series Data Processing on Historical and Real-Time Settings (2021)
Ricardo Filipe Lourenço Nunes de Sousa
Thesis to obtain the Master of Science Degree in Computer Science and Engineering. Supervisors: Prof. Maria da Conceicçao Esperancça Amado, Prof. Rui Miguel Carrasqueiro Henriques
Heterogeneous sensor networks, including water distribution systems and traffic monitoring systems, produce abundant time series data with an arbitrarily-high multivariate order for monitoring network dynamics and detecting events of interest. Nevertheless, errors and failures in the calibration, data storage or acquisition can occur on some of the sensors installed in those systems, producing missing and/or anomalous values. This thesis proposes a computational system, for the fully autonomous cleaning of multivariate time series data using strict quality criteria assessed against ground truth extracted from the targeted series data, on historical and real time data contexts. The proposed methodology is parameterfree as it relies on robust principles for the assessment, hyperparameterization and selection of methods. This work supports an extensive set state-of-the-art methods for (multivariate) time series imputation and outlier detection-and-treatment, considering both point and segment/serial occurrences. A comprehensive evaluation of system is accomplished using heterogeneous sensors from two water distribution systems with varying sampling rates, water consumption patterns, and inconsistencies. Results confirm the relevance of the proposed autonomous processing approach, and its extensibility towards real-time settings under tigtht optimality guarantees.
Water Distribution Networks Design with Mixed Integer Non-Linear Programming 2021)
Gilberto Silva
Thesis to obtain the Master of Science Degree in Mathematics and Applications. Supervisors: Prof. Conceição Amado, Prof. Raquel Barreira
Water.
Unevenly time series modelling of water flow data: a first approach (2021)
Tiago Santos
Thesis to obtain the Master of Science Degree in Mathematics and Applications. Supervisors: Prof. Conceição Amado, Prof. Raquel Barreira
Water.
Unsupervised Anomaly Detection in Water System Networks using Recurrent Neural Networks (2020)
Ana Margarida Pataca da Costa
Thesis to obtain the Master of Science Degree in Computer Science and Engineering. Supervisors: Prof. Susana de Almeida Mendes Vinga Martins, Prof. Rui Miguel Carrasqueiro Henriques
Water is an indispensable resource for society. A major task associated with the proper management of water distribution systems is to detect anomalies to support decision-making and make contingency plans. Interesting anomalies can be bursts, unusual demands, and illegal consumption. Water leakage detection and location is a very difficult problem, due to the lack of information about the water system, and a leak might not be easily detected or confused with other events. The methodology proposed detects anomalies in the water system distributions, with a focus on bursts, through the use of deep learning architectures, in particular, encoder-decoder architectures based on LSTM such as LSTM autoencoder, CNN-LSTM, CNN-BiLSTM, and SCB-LSTM. Predictions are adjusted by using a temperature correction model. The water system studied is Quinta do Lago, located in Portugal, Algarve region. Infraquinta is the entity that manages the infrastructures. Simulation analysis and experimental results in real data show the pitfalls of the unsupervised anomaly detection task in water distribution systems. It also highlights that the proposed methodology, although yielding some properties of interest, needs to be complemented with additional principles to the targeted end. Finally, it pinpoints meaningful differences between recurrent architectures.
Generalized Additive Model and Artificial Neural Networks for Water Demand (2019)
Bruna Alexandra Gregório Mason
Thesis to obtain the Master of Science Degree in Mathematics and Applications. Supervisors: Prof. Maria da Conceicçao Esperancça Amado, Dr. Dália Susana dos Santos da Cruz Loureiro
The growing instability of the weather patterns threats the availability of potable water, becoming increasingly more important the realisation of an efficient management of current and future supplies. The WISDom (Water Intelligence System Data) Project aims at the development of algorithms and models that allow the extraction of knowledge from the data, supporting the water utility in the decision-making. In the context of this project, the present study is aimed at characterizing, modelling and forecasting water flow demand measured at the entry of District Metered Areas (DMA). We describe and compare a series of predictive models for forecasting hourly water demand, namely, we have considered Seasonal Autoregressive Integrated Moving Average (SARIMA) Models, Generalized Additive Models and Artificial Neural Networks (ANNs). Models are evaluated based on the Mean Absolute Error (MAE) and Mean Absolute Scaled Error (MASE). The results show that ANNs can outperform all other methods. We demonstrate that by using time series water flow data, ANN models can provide excellent fit and forecasts without reliance upon the explicit inclusion of climate factors. In addition, an exploratory analysis of water billing data from a southern region of Portugal is done. This analysis aims at improving resource management as well as studying the impact of an ageing meter park on readings' quality.
Not available (Not available)
Not available
Not available
Not available
Not available
Comparison of reconstruction methods for water supply systems flow rate time series (2021)
Carlos Ascensão; Ferreira, Bruno; Barreira, Raquel; Carriço, Nelson
International Conference on Water Energy Food and Sustainability, Leiria
Not available
Rehabilitation of an Industrial Water Main Using Multicriteria Decision Analysis (2021)
Carriço, N.; Covas, D.; Almeida, M.d.C.
Water 2021
This paper demonstrates the application of multicriteria decision analysis (MCDA) methodology in a real case study aiming to choose the best rehabilitation intervention of an industrial water main. This methodology is composed of three main stages: problem identification, structuring, and evaluation. Problem structuring, a crucial stage for the outcomes, includes identifying objectives, selecting scenarios within the analysis period, defining problem alternatives, and defining the problematic type. Problem evaluation includes the selection of assessment metrics, selection of the aggregation method, application of the method, and a sensitivity and robustness analysis. Two scenarios, nine assessment metrics, and seven alternatives are established, and two ranking methods (the additive model and ELECTRE III method) are used to compare the alternatives. The results show that the best solution corresponds to building a new pipe and deactivating the existing one, as it significantly reduces the pipe failure risk and O&M costs, whereas the worst solution is always the status quo case, since it is the only alternative that does not improve the system performance, independently of the ranking method and the analyzed scenario.
Enhanced Water Age Performance Assessment in Distribution Networks (2021)
Monteiro, L.; Algarvio, R.; Covas, D.
Water 2021
Water age is frequently used as a surrogate for water quality in distribution networks and is often included in modelling and optimisation studies, though there are no reference values or standard performance functions for assessing the network behaviour regarding water age. This paper presents a novel methodology for obtaining enhanced system-specific water age performance assessment functions, tailored for each distribution network. The methodology is based on the establishment of relationships between the chlorine concentration at the sampling nodes and simulated water age. The proposed methodology is demonstrated through application to two water distribution systems in winter and summer seasons. Obtained results show a major improvement in comparison with those obtained by published performance functions, since the water age limits of the performance functions used herein are tailored to the analysed networks. This demonstrates that the development of network-specific water age performance functions is a powerful tool for more robustly and reliably defining water age goals and evaluating the system behaviour under different operating conditions.
Spatiotemporal Correlation Feature Spaces to Support Anomaly Detection in Water Distribution Networks (2021)
Gomes, S.C.; Vinga, S.; Henriques, R.
Water 2021
Monitoring disruptions to water distribution dynamics are essential to detect leakages, signal fraudlent and deviant consumptions, amongst other events of interest. State-of-the-art methods to detect anomalous behavior from flowarate and pressure signal show limited degrees of success as they generally neglect the simultaneously rich spatial and temporal content of signals produced by the multiple sensors placed at different locations of a water distribution network (WDN). This work shows that it is possible to (1) describe the dynamics of a WDN through spatiotemporal correlation analysis of pressure and volumetric flowrate sensors, and (2) analyze disruptions on the expected correlation to detect burst leakage dynamics and additional deviant phenomena. Results gathered from Portuguese WDNs reveal that the proposed shift from raw signal views into correlation-based views offers a simplistic and more robust means to handle the irregularity of consumption patterns and the heterogeneity of leakage profiles (both in terms of burst volume and location). We further show that the disruption caused by leakages can be detected shortly after the burst, highlighting the actionability of the proposed correlation-based principles for anomaly detection in heterogeneous and georeferenced time series. The computational approach is provided as an open-source tool available at GitHub.
Near-Real Time Burst Location and Sizing in Water Distribution Systems Using Artificial Neural Networks (2021)
Capelo, M.; Brentan, B.; Monteiro, L.; Covas, D.
Water 2021
The current paper proposes a novel methodology for near-real time burst location and sizing in water distribution systems (WDS) by means of Multi-Layer Perceptron (MLP), a class of artificial neural network (ANN). The proposed methodology can be systematized in four steps: (1) construction of the pipe-burst database, (2) problem formulation and ANN architecture definition, (3) ANN training, testing and sensitivity analyses, (4) application based on collected data. A large database needs to be constructed using 24 h pressure-head data collected or numerically generated at different sensor locations during the pipe burst occurrence. The ANN is trained and tested in a real-life network, in Portugal, using artificial data generated by hydraulic extended period simulations. The trained ANN has demonstrated to successfully locate 60-70% of the burst with an accuracy of 100 m and 98% of the burst with an accuracy of 500 m and to determine burst sizes with uncertainties lower than 2 L/s in 90% of tested cases and lower than 0.2 L/s in 70% of the cases. This approach can be used as a daily management tool of water distribution networks (WDN), as long as the ANN is trained with artificial data generated by an accurate and calibrated WDS hydraulic models and/or with reliable pressure-head data collected at different locations of the WDS during the pipe burst occurrence.
Optimal Number of Pressure Sensors for Real-Time Monitoring of Distribution Networks by Using the Hypervolume Indicator (2021)
Ferreira, B.; Carriço, N.; Covas., D.
Water
This article proposes a novel methodology to determine the optimal number of pressure sensors for the real-time monitoring of water distribution networks based on a quality hypervolume indicator. The proposed methodology solves the optimization problem for different numbers of pressure sensors, assesses the gain of installing each set of sensors by means of the hypervolume indicator and determines the optimal number of sensors by the variation of the hypervolume indicator. The methodology was applied to a real case study. Several robustness analyses were carried out. The results demonstrate that the methodology is hardly influenced by the method parameters and that a reasonable estimation of the optimal number of sensors can be easily achieved.
Using a segmenting description approach in multiple criteria decision aiding (2020)
Kadzínski, M.; Badura, J.; Figueira, J.R.
Expert Systems With Applications 147
Not available
News
The WISDom project is funded by the Fundaçáo para a Ciência e a Tecnologia (Contract No. DSAIPA/DS/0089/2018)
