CN120579067A

CN120579067A - A water resources monitoring method based on big data

Info

Publication number: CN120579067A
Application number: CN202511074865.7A
Authority: CN
Inventors: 牛艺博; 勾晓华; 杨海江; 王宝; 宋晓谕
Original assignee: Lanzhou University; Northwest Institute of Eco Environment and Resources of CAS
Current assignee: Lanzhou University; Northwest Institute of Eco Environment and Resources of CAS
Priority date: 2025-08-01
Filing date: 2025-08-01
Publication date: 2025-09-02
Anticipated expiration: 2045-08-01
Also published as: CN120579067B

Abstract

The invention provides a water resource monitoring method based on big data, which is suitable for surface water body, drainage basin scheduling and dynamic management of a municipal water system; the method comprises the steps of collecting multi-source data such as water quality and water level by utilizing an Internet of things and a remote sensing technology to generate structured water resource data, constructing a spectrum feedback-TCN model to conduct multi-variable prediction on a water quality state, dynamically adjusting a model structure through convolution weight spectrum expansion and power law feature extraction to improve prediction accuracy and interpretability, constructing a reinforced learning driven water level abnormality detection model, combining a punishment objective function, semiDICE strategy and CORSDICE confidence correction mechanism, outputting a strategy correction result as a dynamic priori embedded Bayesian change point detection model to achieve sensitive response and non-steady state identification on sudden water level abnormality, and effectively improving the self-adaptability, stability and real-time early warning capability of a water resource monitoring system.

Description

Water resource monitoring method based on big data

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a water resource monitoring method based on big data.

Background

Although a plurality of intelligent water resource monitoring methods are widely applied to actual scenes at present, the sensing equipment of the Internet of things, edge computing nodes and a deep learning model are fused, and the automatic acquisition and trend prediction of key hydrologic parameters can be realized, the following technical bottlenecks still exist in the aspect of coping with complex hydrologic dynamic and non-steady state identification:

Firstly, most systems adopt an end-to-end training mechanism, mainly pay attention to the mapping relation of input and output and lack structural feedback adjustment on the evolution rule of the internal features of a model, for example, when a convolutional neural network processes time sequence data, spectrum structure information reflected by a weight tensor often contains important dynamic change features, but the modeling and utilization of the spectrum structure features are generally ignored by the existing method, so that the model has insufficient interpretation and weaker robustness, the parameter adjustment is seriously dependent on manual experience, and the prediction performance is difficult to keep stable in a variable environment;

Secondly, although some systems introduce technical means such as a Bayesian method and a change point detection algorithm to identify water level mutation, most methods rely on fixed parameter configuration or static priori distribution, lack the adjustment capability of dynamic matching with environmental states, are difficult to adapt to hydrologic processes with non-stationarity characteristics, and are faced with problems of response delay, low detection accuracy and the like when sudden water level abnormality, landslide fluctuation or extreme hydrologic events are caused, so that the effectiveness and reliability of an early warning mechanism are seriously affected.

Disclosure of Invention

The invention aims to solve the defects of the existing intelligent water resource monitoring system in deep characteristic feedback modeling and non-stationary hydrologic anomaly identification, and provides a water resource monitoring method based on big data; the method integrates a spectrum feedback-TCN model and a reinforcement learning driven water level abnormality detection model, constructs an intelligent monitoring and early warning mechanism with self-adaptability, interpretability and high response sensitivity, is suitable for various application scenes such as complex surface water body, drainage basin scheduling, reservoir operation and urban water management, and particularly, the spectrum feedback-TCN model forms a spectrum structure index by carrying out spectrum structure expansion and power law characteristic extraction on each convolution layer weight tensor, dynamically adjusts network structure parameters according to the spectrum structure index, realizes self-adaptive modeling and accurate prediction on a multivariable water quality state, improves structural perceptibility and stability of the model on complex water quality time sequence data, and the water level abnormality detection model integrates a punishment objective function, semiDICE behavior correction strategy and CORSDICE confidence factor optimization mechanism, and on the basis, embeds strategy correction results as dynamic priori into a Bayesian online change point detection model to realize sensitive identification and dynamic calibration on non-steady sudden water level change.

The invention provides a water resource monitoring method based on big data, which comprises the following steps:

The method comprises the steps of S1, accessing water on-line monitoring equipment by using an Internet of things interface protocol, acquiring water quality parameter data in real time, acquiring water profile data and sediment content data by using a remote sensing image, and realizing data synchronous acquisition by using an ETL scheduling tool to obtain water resource multi-source data;

s2, preprocessing the water resource multi-source data to generate structured water resource data;

step 3, constructing a spectrum feedback-TCN model, processing the structured water resource data through the spectrum feedback-TCN model, and predicting the water quality state to generate a multivariable prediction result;

S4, constructing a water level abnormality detection model, inputting the structured water resource data into the water level abnormality detection model, detecting water level abnormality, and generating a water level mutation detection result;

and S5, combining the multivariable prediction result and the water level mutation detection result, and performing early warning response and visual analysis.

Further, the process of generating a multivariate prediction result by a spectral feedback-TCN model specifically comprises the following steps:

s31, performing sliding window slicing operation on structured water resource data to construct a training sample pair set;

Step S32, configuring causal convolution, void ratio and residual connection of a TCN model, inputting a training sample pair set into the TCN model for training, and recording a convolution kernel weight matrix of each layer in the TCN model to obtain a trained TCN model and a convolution kernel weight matrix set;

Step S33, performing dimension reconstruction on each weight tensor in the convolution kernel weight matrix set, unfolding the weight tensor into a two-dimensional weight matrix form to obtain a two-dimensional weight matrix set, segmenting each two-dimensional weight matrix in the two-dimensional weight matrix set into submatrices by using a sliding window strategy with a fixed length-width ratio, calculating a correlation matrix of each submatrix and extracting a characteristic value sequence, splicing all characteristic value sequences to construct average spectrum distribution, performing power law fitting on the average spectrum distribution, and obtaining corresponding PL Alpha indexes by using Hill Estimator;

And step S34, judging the PL Alpha index state by combining the spectrum structure index set, dynamically adjusting the corresponding layer super parameters of the TCN model to obtain an optimized TCN model, and generating a multivariable prediction result by combining the optimized TCN model.

Further, the water level abnormality detection model performs water level abnormality detection, and a process of generating a water level mutation detection result specifically includes the following steps:

S41, slicing a water level monitoring value in structural water resource data according to a sliding window to form a time sequence state, constructing hydrologic behavior actions, including adjusting a sluice, starting drainage and alarming, pairing the time sequence state and the hydrologic behavior actions to form a state-action pair, carrying out standardization and noise reduction treatment on the state-action pair, and extracting time sequence statistical characteristics to obtain a state-action sequence;

Step S42, calculating instant rewards of each time point based on a state-action sequence, introducing a cost function for describing potential threats of the state-action combination to system safety, introducing a punishment factor, constructing a punishment objective function for punishing high-risk behaviors and guiding strategies to converge towards low-risk trajectories, constructing a cost function estimation model based on the state-action sequence, training the cost function estimation model through the punishment objective function, evaluating action rewards under the current strategy, introducing a dual function and a regular factor, and calculating a behavior layer correction factor sequence through a SemiDICE semi-gradient strategy, thereby describing the deviation degree of the current strategy relative to the history strategy, realizing sensitive response to the high-risk behaviors, and reflecting state value potential energy of each state by the dual function;

Step S43, inputting a behavior layer correction factor sequence into a state layer, constructing a state balance constraint, and learning a correction term of a current strategy in a state dimension through a state dual function and an auxiliary estimation function so as to characterize steady-state offset distributed with historical data;

And S44, constructing a Bayesian online change point detection model, embedding a state layer confidence correction factor sequence into the model as priori dynamic weight to realize dynamic calibration of mutation priori probability, and improving sensitivity to non-stationary water level change through the mechanism to output a water level mutation detection result.

By adopting the scheme, the beneficial effects obtained by the invention are as follows:

The method realizes structural understanding and self-adaptive prediction of multivariable water quality time sequence data by introducing a spectrum feedback-TCN model, integrates a convolutional layer weight spectrum expansion and power law characteristic extraction mechanism on the basis of a traditional TCN architecture, establishes a spectrum structure feedback loop, dynamically adjusts model super-parameters according to PL Alpha indexes of each layer, improves the adaptability of a system to the evolution rule of water quality parameters and the capture capability of abnormal fluctuation by the mechanism, solves the problems of rigidity and poor prediction stability of the existing system, and remarkably enhances the water quality trend identification capability under a complex hydrologic environment.

The method realizes the accurate identification and dynamic response of sudden non-stationary water level change by constructing the reinforcement learning driven water level abnormality detection model, combines a punishment objective function and SemiDICE strategy correction behavior deviation, further introduces a CORSDICE mechanism to extract a state layer confidence correction factor, embeds a Bayes online change point detection model as a dynamic priori, and realizes the probability self-calibration of mutation detection.

In conclusion, the spectrum feedback-TCN model and the water level abnormality detection model form a cooperative mechanism in the dynamic monitoring of water resources, so that the prediction capability of the water quality change trend is improved, the intelligent recognition capability of the water level mutation risk is enhanced, the whole system has high interpretability, self-adaptability and instantaneity, can be widely applied to river basin management, reservoir scheduling and urban water service systems, and provides high-precision and high-reliability intelligent technical support for the water resource safety management.

Drawings

FIG. 1 is a graph of a water level prediction curve and a mutation probability hot zone proposed in the sixth embodiment;

FIG. 2 is a water quality multivariable radar chart presented in the sixth embodiment;

fig. 3 is a graph of the TCN model spectral structure index line presented in example six.

In FIG. 1, blue lines represent water level prediction, red columns represent mutation probability at each time, in FIG. 2, the multivariate includes ammonia nitrogen, dissolved oxygen, COD and PH, in FIG. 3, the lower side represents the number of model layers, and the left side represents the PL Alpha value.

Detailed Description

The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.

In a first embodiment, the present invention provides a water resource monitoring method based on big data, the method comprising the following steps:

S3, constructing a spectrum feedback-TCN model, processing structured water resource data through the spectrum feedback-TCN model, and predicting the water quality state to generate a multivariable prediction result;

S4, constructing a water level abnormality detection model, inputting the structured water resource data into the water level abnormality detection model to perform water level abnormality detection, and generating a water level mutation detection result;

In the second embodiment, the process of generating a multivariate prediction result by using a spectral feedback-TCN model in the first embodiment specifically includes the following steps:

Step S32, constructing a TCN model, configuring causal convolution, void ratio and residual connection, inputting a training sample pair set into the TCN model for training, and recording a convolution kernel weight matrix of each layer in the TCN model to obtain a trained TCN model and a convolution kernel weight matrix set;

Step S33, performing dimension reconstruction on each weight tensor in a convolution kernel weight matrix set, unfolding the weight tensor into a two-dimensional weight matrix form to obtain a two-dimensional weight matrix set, segmenting each two-dimensional weight matrix in the two-dimensional weight matrix set into submatrices by using a sliding window strategy with a fixed length-width ratio, calculating a correlation matrix of each submatrix and extracting a characteristic value sequence, splicing all characteristic value sequences to construct average spectrum distribution, performing power law fitting on the average spectrum distribution, obtaining corresponding PL Alpha indexes by using Hill Estimator, and summarizing PL Alpha of all layers into a spectrum structure index set by using the following formula:

Hill Estimator power law index estimation formula:

;

wherein, the Representing the power law exponent,Representing the layer index of the TCN model,Represent the firstThe power law index estimation value corresponding to the layer, namely the heavy tail index; represents the number of power law tail fitting points, Representing the tail sample index,Representing the total number of feature values; Represent the first Is used for the characteristic value of the (c),Represent the firstIs a characteristic value of (2); Representing a natural logarithmic function;

Hill Estimator (Hill estimator) is a method for estimating the tail index (i.e., degree of heavy tail) of a power law distribution, is commonly used in extremum theory and heavy tail distribution analysis, and is particularly suitable for analyzing empirical data distribution with long tail or heavy tail characteristics;

In a third embodiment, the present embodiment is based on the first embodiment, and the process of generating the multivariate prediction result in the present embodiment specifically includes the following steps:

e1, performing sliding window slicing operation on structured water resource data to construct a training sample pair set;

e2, constructing a TCN model, configuring causal convolution, void ratio and residual connection, inputting a training sample pair set into the TCN model for training, and obtaining a trained TCN model;

Step E3, reasoning an input sample by using the trained TCN model, extracting characteristic representation of each time step, and generating a multidimensional characteristic output sequence;

and E4, based on the feature expression vector, predicting by using the trained TCN model, and outputting corresponding multivariable prediction results including future trend prediction results of water level, water temperature and sediment content.

In the fourth embodiment, based on the second embodiment, in the present embodiment, the water level abnormality detection model performs water level abnormality detection, and a process of generating a water level mutation detection result specifically includes the following steps:

Step S42, calculating instant rewards of each time point based on a state-action sequence, introducing a cost function for describing potential threats of the state-action combination to system safety, introducing a punishment factor, constructing a punishment objective function for punishing high-risk behaviors and guiding strategies to converge towards low-risk trajectories, constructing a cost function estimation model based on the state-action sequence, training the cost function estimation model through the punishment objective function, evaluating action rewards under the current strategy, introducing a dual function and a regular factor, calculating a behavior layer correction factor sequence through a SemiDICE semi-gradient strategy, describing the deviation degree of the current strategy relative to a history strategy, realizing sensitive response to the high-risk behaviors, and reflecting state value potential energy of each state by the dual function, wherein the following formula is adopted:

Punishment objective function formula:

;

wherein, the The time-series state is indicated,Representing the action of the hydrologic behavior,Representing a punished bonus function,Representing the original bonus function,A penalty factor is indicated and is indicated,Representing a cost function;

Behavior modification ratio formula in SemiDICE half gradient strategy:

;

wherein, the The behavior modification ratio is represented as such,Indicating that the current policy is in stateDown selection actionProbability of (2); a set of historical data is represented and, Representing historical dataset behavior policies in stateDown selection actionProbability of (2); Derivative representing f-divergence Is used as a function of the inverse function of (c),The dual function is represented by a function of the dual,Representing the regularization factor,Representing a model of the estimation of the cost function,Representing a non-negative correction;

Step S43, inputting a behavior layer correction factor sequence to a state layer, constructing a state balance constraint, learning a correction term of a current strategy in a state dimension through a state dual function and an auxiliary estimation function so as to characterize steady-state offset with historical data distribution, solving the correction term by utilizing a double-stage optimization mechanism of CORSDICE, respectively minimizing an expected fitting error of the auxiliary estimation function and regularization cost of the state dual function, and acquiring a state layer confidence correction factor sequence by using the following formula:

The first stage formula:

;

wherein, the The next state is indicated to be the next state,The auxiliary estimation function is represented as such,Representing state dual functionsAt the value of the current state of the device,Representing state dual functionsAt the value of the next state,Representing the discount factor(s),Representing a state transition dual residual term; representing empirical expectations; representing a minimized auxiliary estimation function;

the second stage formula:

;

wherein, the Representing an objective function defined by a minimized state dual function,Representation ofIs used as a target function of the external target function of the (c),The representation is defined as; representing an initial state Is used to determine the dual value of (c),Representing initial state distributionMiddle samplingIs used as a means for controlling the speed of the vehicle,A derivative representing the f-divergence conjugate function;

The construction of the state balance constraint provides structural conditions to be met, and the optimized state dual function and the auxiliary estimation function are efficient strategies and technical paths for realizing the constraint, and form the unification of targets and means;

In the fifth embodiment, the step S4 specifically includes constructing a Bayesian online change point detection model, taking structured water resource data as input, extracting water level time sequence characteristics, carrying out online posterior inference and mutation detection based on the model, and outputting a water level mutation detection result, wherein the method is suitable for standard monitoring scenes without prior strengthening information by continuously evaluating the change trend of the water level data to realize quick response and real-time detection of the mutation points.

In the sixth embodiment, according to fig. 1,2 and 3, the present embodiment is based on the fifth embodiment, and in the present embodiment, step S5, early warning response and visual analysis are performed by combining the multivariate prediction result and the water level mutation detection result;

in this embodiment:

table 1 shows the multivariate prediction results:

TABLE 1

;

The analysis result shows that COD exceeds the water quality limit value of III class of the surface water by more than one time, the water level rises from 3.85m to 5.33m, and the continuous rising situation and the trend danger are displayed;

Table 2 shows the water level mutation detection results:

TABLE 2

;

Starting at 03:00, the mutation probability obviously rises to 74%, 86% is reached at 04:00, the section is marked as a 'water level mutation interval', and a mutation alarm is triggered;

And (3) visual analysis, namely a water level prediction curve, a mutation probability hot zone diagram, a water quality multivariable radar diagram and a TCN model spectrum structure index line diagram.

The present invention and its embodiments have been described above with no limitation, and the embodiments of the present invention are shown in the drawings, and the actual structure is not limited thereto, so that those skilled in the art who have the ordinary skill in the art who have the benefit of the present invention will not creatively design similar structures and examples to those of the present invention without departing from the gist of the present invention.

Claims

1. A water resources monitoring method based on big data, characterized in that the method comprises the following steps:

Step S1: Collect multi-source data of water resources;

Step S2: preprocessing multi-source water resource data to generate structured water resource data;

Step S3: constructing a spectral feedback-TCN model, processing structured water resource data through the spectral feedback-TCN model, performing water quality state prediction, and generating multivariate prediction results; the spectral feedback-TCN model includes a TCN model;

Step S4: constructing a water level anomaly detection model, inputting the structured water resource data into the water level anomaly detection model to perform water level anomaly detection and generate water level mutation detection results; the water level anomaly detection model includes a state construction unit, a correction calculation unit, a state correction unit, and a mutation detection unit;

Step S5: Combine the multivariate prediction results and the water level mutation detection results to perform an early warning response.

2. The water resources monitoring method based on big data according to claim 1 is characterized in that the process of generating multivariate prediction results through the spectral feedback-TCN model specifically includes the following steps:

Step S31: constructing a training sample pair set based on structured water resources data;

Step S32: inputting the training sample pair set into the TCN model for training to obtain a convolution kernel weight matrix set;

Step S33: combining the convolution kernel weight matrix set, and obtaining the spectrum structure index set through matrix feature extraction and power law fitting method of average spectrum distribution;

Step S34: Combine the spectral structure indicator set to generate a multivariate prediction result.

3. A water resources monitoring method based on big data according to claim 2 is characterized in that: step S33 specifically includes: reconstructing the dimension of each weight tensor in the convolution kernel weight matrix set to obtain a two-dimensional weight matrix set; using a sliding window strategy with a fixed aspect ratio to divide each two-dimensional weight matrix in the two-dimensional weight matrix set into sub-matrices; calculating the correlation matrix of each sub-matrix and extracting the eigenvalue sequence; splicing all eigenvalue sequences, constructing an average spectral distribution, performing power law fitting on the average spectral distribution, using Hill Estimator to obtain the corresponding PL Alpha index, and summarizing it into a spectral structure index set.

4. A water resources monitoring method based on big data according to claim 1, characterized in that: the state construction unit constructs time-series states and hydrological behavior actions according to structured water resources data to obtain a state-action sequence.

5. A water resources monitoring method based on big data according to claim 4 is characterized in that: a correction calculation unit calculates the instantaneous reward at each time point in the state-action sequence, introduces a cost function and a penalty factor, and constructs a penalty objective function; a value function estimation model is constructed based on the state-action sequence, the value function estimation model is trained by the penalty objective function, and a dual function and a regularization factor are introduced, and a behavior layer correction factor sequence is obtained through the SemiDICE semi-gradient strategy.

6. A water resources monitoring method based on big data according to claim 5 is characterized in that: the state correction unit constructs a state balance constraint according to the behavior layer correction factor sequence, and learns the correction term of the current strategy in the state dimension through the state dual function and the auxiliary estimation function; the correction term is solved using the two-stage optimization mechanism of CORSDICE, and the expected fitting error of the auxiliary estimation function and the regularization cost of the state dual function are minimized respectively to obtain the state layer confidence correction factor sequence.

7. A water resource monitoring method based on big data according to claim 6 is characterized in that: the mutation detection unit constructs a Bayesian online change point detection model, combined with the state layer confidence correction factor sequence, to obtain the water level mutation detection result.