CN114936620B - Bias correction method for sea surface temperature numerical forecast based on attention mechanism - Google Patents

Abstract

The invention discloses a sea surface temperature numerical forecast deviation correcting method based on an attention mechanism, which comprises the following basic steps: preprocessing the prediction data of the mode to be corrected, and constructing an input sequence; constructing a sea surface temperature correction model; correcting the model data by using the constructed model; and evaluating the correction precision of the model. The method not only considers the influence of the space distribution of training data, adds characteristic factors such as salinity and the like, but also considers the importance of history information. The model can effectively extract the space-time dependency relationship between the sea temperature field data, thereby realizing high-precision SST correction.

Description

Bias correction method for sea surface temperature numerical forecast based on attention mechanism

Technical Field

The present invention belongs to the technical field of marine meteorological forecast correction, and in particular relates to a method for correcting sea surface temperature numerical forecast deviations based on an attention mechanism.

Background technique

The ocean occupies more than 70% of the earth's surface area and is closely related to human activities. In global climate research, marine ecosystem research and ocean-related applications, sea surface temperature is an important physical quantity. Changes in sea temperature have a very important impact on activities such as navigation, marine disaster prevention and mitigation, and marine fisheries. Therefore, accurate observation and forecasting of sea surface temperature is of great significance. Numerical model forecasting is a common forecasting method in marine forecasting. However, since the numerical forecast model cannot fully describe various physical processes in the ocean, the model still has problems such as uncertainty in the initial field and unavoidable calculation errors in the numerical solution process of the model, the forecast results still have certain errors, which makes the forecast results of numerical forecast products need further correction. The forecast model based on machine learning is a purely data-driven forecast model, although it has great advantages in capturing the nonlinear relationship between forecast factors and forecast targets. However, with the advancement of science and technology and society, element forecasting based only on deep learning methods is difficult to meet the needs of current business forecasting, and the forecast results based on numerical models also have unavoidable errors. Both data-driven machine learning methods and theory-driven numerical model methods cannot meet the needs of marine element forecast accuracy. Therefore, it is urgent to fuse the theory-driven numerical model method with the data-driven deep learning method to forecast ocean elements. The forecast errors of numerical forecast products can be corrected by post-processing the forecast results of numerical models using machine learning methods to improve the accuracy of forecasts of various elements. In terms of how to fuse the two, it is urgent to use deep learning methods to post-process the model results to correct the forecast errors of numerical forecast products to improve the accuracy of forecasts of various elements.

Summary of the invention

In view of this, the present invention proposes a sea surface temperature numerical forecast deviation correction method based on attention mechanism, and a sea surface temperature correction method with attention mechanism based on ConvLSTM and CBAM, including data preprocessing, constructing input sequence, constructing correction model, correcting pattern data and evaluating correction accuracy, etc.

The present invention discloses a method for correcting deviations of sea surface temperature numerical forecasts based on an attention mechanism, comprising the following steps:

Step 1: Preprocess the model forecast data and remote sensing satellite sea temperature data to construct the input sequence, including:

(1) Obtain model forecast data and remote sensing satellite sea temperature data as reference values, and extract marine environment data;

(2) Construct a time column, obtain the number of time features through a sliding window, and then add ocean feature influencing factors, including sea temperature, salinity, and the U and V vectors of water currents, to obtain the input historical SST sequence of the model;

(3) Standardize the historical SST series data obtained in the above steps;

Step 2: Construct a revised model, including:

(1) Spatial feature extraction: Using 3D convolution, the 3D kernel is convolved with a cube formed by superimposing multiple continuous matrices to extract the spatial dependency characteristics of SST data and the relationship between multiple environmental variables;

(2) Using the 3D-CBAM attention mechanism on the data sequence after 3D convolution to improve the utilization rate of the spatial features of the 3D convolution network and show the importance of different environmental variables to the results; the 3D-CBAM attention mechanism consists of two parts: a channel attention module and a spatial attention module;

Multiply Mc and the input feature matrix F to get the input matrix Ms of the spatial attention module;

Multiply Ms by the input F’ of the module to obtain the final generated feature matrix;

(3) Input the feature matrix data sequence obtained in the above steps into the ConvLSTM part of the model; use the Attention mechanism to add a custom attention layer after the ConvLSTM network, and use the hidden layer state of each step of the ConvLSTM model to assign the temporal attention weight to the hidden layer state of each time step;

Adjust the final ConvLSTM output to get the final corrected result;

(4) Building and training the correction model obtained in the above steps, continuously adjusting the parameters, and selecting the best parameters to obtain the sea surface temperature correction model;

Step 3: Use the correction model to correct the model forecast data;

(1) Inputting the standardized time series forecast data into the sea surface temperature correction model obtained in step 2 to obtain the corrected output result;

(2) De-standardize the model output results. The de-standardization method corresponds to the standardization method in step 1, and the corrected SST value is obtained;

Step 4: Use MAE, MAPE, MSE and RMSE evaluation indicators to evaluate the accuracy of the model correction results.

Furthermore, the bilinear interpolation method is used to unify the temporal and spatial resolutions of the forecast data and remote sensing satellite observation data.

Furthermore, for the historical data sequence X, where the SST data at any time t and other marine environmental variables Xt are grid data of W×H×C specifications, the input of the entire model is a five-dimensional tensor, expressed as B×T×C×W×H, where B is the number of training samples in a batch, T is the length of the sequence data, W and H are the length and width of the SST field, and C is the number of marine environmental variables added.

Furthermore, the standardization formula is as follows:

Among them, X _max and X _min are the maximum and minimum values in the sequence data respectively.

Furthermore, the calculation formula of the channel attention module is as follows:

Mc(F)＝σ(MLP(MaxPool3D(F))+MLP(AvgPool3D(F)))

Where MLP represents multi-layer perceptron, MaxPool3D represents maximum pooling, AvgPool3D represents average pooling, and F is the input feature matrix;

The input matrix Ms of the spatial attention module is:

Ms(F) = σ(f ^3×3 ([MaxPool3D(F); AvgPool3D(F)]))

Among them, f is a 3×3 convolution operation.

Furthermore, the calculation formula of the ConvLSTM network is as follows:

_it =σ( _wxi * _xt + _whi * _ht-1 + _wci · _ct-1 +b _i )

_ft = σ( _wxf * _xt + _whf * _ht-1 + _wcf · _ct-1 + _bf )

c _t = f _t · c _t-1 + _it · tanh(w _xc * x _t + w _hc * h _t-1 + b _c )

o _t =σ(w _xo *x _t +w _ho *c _t-1 +w _co ·c _t +b _o )

h _t = o _t ·tanh(c _t )

Where _it represents the input gate, _ft represents the forget gate, _ot represents the output gate, _ct represents the unit state, w is the weight matrix, _wxi is the weight matrix of the input gate _xt , _wxf is the weight matrix of the forget gate _xt , _wxc is the weight matrix of _xt in the unit state calculation, _who is the weight matrix of the output gate _xt , b is the bias term, _bi is the bias term of the input gate, _bf is the bias term of the forget gate, _bc is the bias term of the unit state, _bo is the bias term of the output gate, _xt is the input matrix, _ht-1 is the hidden layer state at time t-1, and _ct-1 is the memory state at time t-1.

Furthermore, the attention layer takes the output _ht of each iteration of ConvLSTM as input; then the weight AT( _ht ) of each output vector is obtained through Softmax operation; finally, the attention weight AT( _ht ) is multiplied by the hidden layer state _ht to obtain the final correction result _Yt , which is calculated as follows:

Among them, W is the weight matrix.

Furthermore, the mean square error is used as the loss function, and the calculation formula of the loss function is:

Where n represents the number of grid points in the grid data, is the true value data of the grid, _yi is the network corrected data; after building the model, set the input dimension of the model and the time step of the input data; set the model optimizer and learning rate; set the number of hidden layer neural nodes; set the number of model iterations; continuously adjust the parameters, check the degree of model convergence with the model loss, select the convergence parameters at the best, and form the final sea temperature correction model.

Furthermore, the calculation methods of the mean square error MSE, root mean square error RMSE, mean absolute error MAE and mean absolute percentage error MAPE are as follows:

Where _yi is the true value, is the estimated value, and n is the number of samples.

The beneficial effects of the present invention are as follows:

Mining the variation patterns of sea surface temperature data based on spatiotemporal series can achieve better correction results with high correction accuracy.

The CBAM module of the present invention is very small, so that the model training time is reduced, the number of parameters is also small, the training speed is faster, and good performance is achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a flow chart of the present invention;

Figure 2 is a diagram of the revised model framework;

Figure 3 is a diagram showing the correction effect of the correction model at different time steps;

Figure 4 is a diagram showing the correction effect of the correction model at different learning rates;

Figure 5 shows the correction effect of the correction model under different training times.

Detailed ways

The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

The present invention is a sea surface temperature correction method with an attention mechanism based on ConvLSTM and CBAM, including data preprocessing, constructing an input sequence, constructing a correction model, correcting the pattern data and evaluating the correction accuracy, etc. The specific flow chart is shown in Figure 1.

Step 1 The specific steps are:

1. Preprocess the model forecast data and the remote sensing satellite sea temperature data used as label values to construct the input sequence. Using the bilinear interpolation method, the spatial and temporal resolution of the unified forecast data and remote sensing satellite observation data is 0.25°×0.25° per day.

2. For the historical data sequence X, Xt composed of SST data and other marine environmental variables at any time t is grid data of W×H×C specifications, so the input of the entire model is a five-dimensional tensor, expressed as B×T×C×W×H. Here B is the number of training samples in a batch, and T is the length of the sequence data. W and H are the length and width of the SST field, and C is the number of added marine environmental variables. In the experiment, the length H and width W are longitude and latitude. The number of time features is obtained by sliding the window. For example, if the SST of the day is corrected with the historical data of the past 3 days, the length of the time series is 4, which is the value of T. In the experiment, in addition to the sea temperature, three influencing factors, U and V vectors of salinity and water flow, are added, so C is 4 here. Finally, a 5-dimensional tensor is obtained as the input sequence of the model. U is the direction pointing to the east, and V is the direction pointing to the north. Sometimes, U refers to the zonal velocity and V refers to the radial velocity.

3. Standardize the obtained input data sequence and use the standardized data sequence as the input data of the CBAM-ConvLSTM model with an input dimension of B×T×C×W×H.

Step 2 The specific steps are:

Establish the correction model CBAM-ConvLSTM. The framework of the CBAM-ConvLSTM method is shown in Figure 2.

1. Spatial feature extraction. In this step, three-dimensional convolution is used to extract the spatial dependence characteristics of sea temperature data and the relationship between various environmental variables. Three-dimensional convolution is developed on the basis of two-dimensional convolution. Three-dimensional convolution is to convolve a cube formed by superimposing a three-dimensional kernel with multiple continuous matrices.

2. The 3D-CBAM attention mechanism is used to improve the utilization rate of the spatial features of the 3D convolutional network for the data sequence after 3D convolution, and the importance of different environmental variables to the results is shown. The input feature matrix F (B×T×H×W×C) is subjected to global max pooling and global average pooling based on width and height respectively to obtain two B×T×1×1×C feature maps. Then, they are sent to a two-layer neural network (MLP) respectively. The number of neurons in the first layer is C/r (r is the reduction rate), the activation function is Relu, and the number of neurons in the second layer is C. The two layers of the neural network are shared. Then, the features output by the MLP are added and then activated by sigmoid to generate the final channel attention matrix, i.e. Mc. Finally, Mc is multiplied by the input feature matrix F to generate the input feature F' required by the spatial attention module. The feature matrix F' output by the channel attention module is used as the input feature matrix of this module. First, perform a channel-based global max pooling and global average pooling to obtain two B×T×H×W×1 feature matrices, and then perform a channel-based concatenation operation (concat) on these two feature maps. Then, after a convolution operation, the dimension is reduced to 1 channel. Then, a sigmoid activation function is used to generate a spatial attention matrix, Ms. Finally, the Ms is multiplied by the input F’ of the module to obtain the final generated feature matrix X.

3. Input X obtained in the above steps into the ConvLSTM part of the model. Since the interval of sea temperature correction is one day, the hidden layer state at the previous moment is h _t-1 and the memory state at the previous moment is c _t-1 . ConvLSTM includes forget gate f _t , input gate _it , and output gate o _t . At the current moment t, the forget gate f _t is responsible for controlling how much of the previous moment's c _t-1 is saved to the current moment's c _t ; the input gate _it is responsible for controlling how much of the current moment's instant state is input to the current unit state c _t ; the output gate o _t is responsible for controlling how much of the current unit state o _t is used as the hidden layer output h _t at the current moment. The calculation formulas are:

_it =σ( _wxi * _xt + _whi * _Ht-1 + _wci · _Ct-1 + _bi )

f _t =σ(w _xf *x _t +w _hf *H _t-1 +w _cf ·C _t-1 +b _f )

o _t =σ(w _xo *x _t +w _ho *H _t-1 +w _co ·C _t +b _o )

Among them, w _xi , w _xf , w _xo are the weight matrices of the input gate, forget gate and output gate respectively, _bi , b _f , b _o are the bias terms of the input gate, forget gate and output gate respectively, and σ is the sigmoid function.

The current cell state c _t is determined by the forget gate f _t , the cell state c _t-1 at the previous moment, the input gate i _t and the current input cell state. The calculation formula is:

C _t = f _t ·C _t-1 + _it ·tanh(w _xc *x _t +w _hc *H _t-1 +b _c )

Among them, w _xc is the weight matrix of the input unit state, b _c is the bias term of the input unit state, and tanh is the hyperbolic tangent function.

The hidden layer output value _ht of ConvLSTM at the current moment is determined by the output gate _ot and the current unit state _ct , and its calculation formula is:

H _t = o _t ·tanh(C _t )

At the same time, this stage uses the Attention mechanism to add a custom attention layer after the ConvLSTM network, making full use of the hidden layer state of each step of the ConvLSTM model and assigning the time attention weight to the hidden layer state of each time step. The attention layer takes the output h _t of each iteration of ConvLSTM as input; then the weight AT(h _t ) of each output vector is obtained through the Softmax operation; finally, the attention weight AT(h _t ) is multiplied by the hidden layer state h _t to obtain the final correction result Y _t , and the calculation formula is as follows:

Among them, W is the weight matrix.

The entire SST forecast correction model can be expressed as:

The present invention adopts mean square error (MSE) as loss function (LOSS), and its calculation formula is:

Where n represents the number of grid points in the grid data, is the true value data of the grid points, and _yi is the network-corrected data. As a further optimization, after building the model, set the model input dimension and the time step of the input data; set the model optimizer and learning rate; set the number of hidden neural nodes; set the number of model iterations; continuously adjust the parameters, check the model convergence degree with the model loss, select the parameters with high convergence degree, and form the final sea temperature correction model.

Step 3: Use the correction model to correct the model forecast data;

1. Input the normalized time series forecast data into the sea surface temperature correction model obtained in step 2 to obtain the corrected output result; historical data is mainly used to train the model and is the data of a time series, while the daily data is used to correct the data and is the data at the current moment. Usually the daily data is included in the historical data, but it can also be corrected when it is not included.

2. De-standardize the model output results. The de-standardization method corresponds to the standardization method in step 1. The corrected SST values are obtained with an accuracy of 0.25°×0.25° per day.

Step 4: To verify the effectiveness of the CBAM-ConvLSTM model, four indicators, namely mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE), are used to evaluate the model. MAE reflects the overall error of the estimate, and RMSE reflects the estimation sensitivity and extreme value effect of the sample data. The smaller the values of the two, the better the effect. The calculation methods of each indicator are as follows:

Among them, _yi represents the true observation value, represents the average of the true observations, /> Represents the predicted value.

The following example illustrates the effect of the sea surface temperature correction method based on 3DCBAM-ConvLSTM. In the experiment, the global 1/12.5° hybrid coordinate ocean model HYCOM forecast data released by the National Oceanic and Atmospheric Administration (NOAA) of the United States is used as the forecast data to be corrected, and the NOAA 1/4° Daily OI SST Analysis satellite remote sensing observation data is used as the true value to evaluate the correction accuracy. The experiment takes the 0.25°×0.25° range of 110°E-114°E and latitude 8°N-12°N as an example. The sea area is located in the South China Sea. The historical SST data within the above range is extracted, and the time range is from 2019-01-01 to 2019-12-25, a total of 360 days.

HYCOM is a three-hour forecast. The daily average temperature of HYCOM is calculated, and the bilinear interpolation method is used to interpolate the HYCOM forecast data to the grid points of OI SST Analysis to unify the spatiotemporal resolution of the forecast data and remote sensing satellite observation data. Since the numerical differences of the data are obvious, normalization is required. The normalization operation can improve the convergence speed of the model on the one hand, improve the accuracy of the model on the other hand, and prevent the gradient explosion of the model. According to the data time, the corresponding time column is generated, and all the processed SST data are divided. 75% of the data is used as the training set to train the parameters of the CBAM-ConvLSTM forecast model, and the remaining 25% of the data is used as the validation set to verify the learning effect of the model.

The model is built by Pytorch, and the shapes of training data and input data are adjusted and converted into the required tensor format in the Pytorch framework. Then the parameters of the CBAM-ConvLSTM model are defined, including the input step length, i.e. the length of the input sequence, the number of hidden layers, the length of the output sequence, and the number of neurons in each layer. In this experiment, the convolution part of the model includes a Conv3D layer and a BN layer. The main function of the BN layer is to make the distribution of input data in each layer of the network relatively stable, accelerate the learning speed of the model, alleviate the gradient disappearance problem, and have a certain regularization effect. When setting the network, the convolution kernel size in the Conv3D layer is 3×3×3. The convolution kernel size in the convolution attention in the CBAM part of the model is 3×3×3. In the ConvLSTM part of the model, since the sequence selected for the experiment is short, only a single-layer ConvLSTM is selected, the number of hidden layer neurons is 32, and the number of neurons in the output layer is 1. After defining the model parameters, define the loss function and optimizer, select the MSE loss function and Adam optimizer, and then select the appropriate number of training times to start training the model. After the model training is completed, the test data is input into the model for testing, and the deviation correction value of the sea temperature is obtained after the model output result is denormalized. The correction effect is tested by comparing the indicators before and after the deviation correction.

In order to prove the effectiveness of the CBAM-ConvLSTM hybrid model proposed in this paper, the experimental results are compared with two traditional machine learning algorithms in sea surface temperature correction. They are linear regression and SVM support vector machine. The linear regression analysis model has the advantages of strong anti-interference ability and fast training speed. The disadvantage is that it cannot simulate nonlinear relationships, the accuracy is not very high, and it is easy to underfit. The SVM model has better generalization performance, is not easy to overfit, and can achieve good performance with less data. However, SVM is sensitive to missing data, parameters, and kernel functions. In the past, in order to match the input form of these algorithms, SST and ocean variables were generally used as independent features, so that the spatiotemporal relationship between variables could not be considered. The process of implementing these two algorithms in this paper is to first expand all samples into a form that the algorithm can handle, and call the sklearn machine learning algorithm package for predictive analysis. In addition, we compared a series of models and compared them with the CBAM-ConvLSTM model, because there were fewer methods used to calibrate two-dimensional sea temperatures. This includes an LSTM model that only considers temporal relationships but not spatial relationships, an improved ConvLSTM model that combines convolution, a ConvLSTM model that only adds 3dCnn, a ConvLSTM model that only adds the temporal attention mechanism, and a CONVLSTM hybrid model that adds both CNN and AT. Here, we set the experimental parameters learning rate LR = 0.01, the number of iterations Epoch = 300, and use 3 days of historical data for SST correction. We will discuss the parameter settings in detail later. The model evaluation uses RMSE, MSE, MAE and MAPE. The experimental results of the formation prediction of different correction methods are shown in Table 1.

Table 1 Comparison of correction results of various correction methods

From the comparison in Table 1, we can find that when using traditional machine learning methods for correction, SVR has a higher accuracy than linear regression. Among other deep learning models, the 3DCNN-CONVLSTM-AT model has the best results. However, the CBAM-CONVLSTM model we proposed can achieve an MSE value of 0.3520, a MAE value of 0.2641, and a MAPE of 0.9546% in the correction experiment, which is better than the other models.

As can be seen from Table 1, the effect of LSTM is better than that of using traditional machine learning methods for correction, which shows the importance of temporal correlation to SST correction. The results of the existing LSTM improved method ConvLSTM are better than LSTM, which verifies the importance of spatial correlation to SST correction. At the same time, the experimental results of ConvLSTM-AT with added attention mechanism show that the different effects of historical SST on the predicted SST are considered by assigning different weights, so the model performs better. The prediction results of 3DCNN-ConvLSTM-AT and ConvLSTM-AT are compared, where ConvLSTM-AT does not have a convolution layer and 3DCNN-ConvLSTM-AT has a convolution layer. Under the same parameters and the same data set, it is obvious that the correction accuracy of 3DCNN-ConvLSTM-AT is higher than that of ConvLSTM-AT. The experimental results show that adding convolution layers has a certain effect on improving the accuracy of SST forecasts. The main reason for this is that the local features extracted from SST data by the convolution operation of ConvLSTM itself are not obvious enough. Adding another convolutional layer before the model improves the feature extraction capability of the model, making the spatial characteristics of the data more obvious in the ConvLSTM model, which is conducive to improving the accuracy of SST forecast.

Our CBAM-ConvLSTM model adds the CBAM attention mechanism to the 3DCNN-ConvLSTM-AT model, with an RMSE index of 0.35 and the best correction effect. The CBAM-ConvLSTM model further extracts spatial features and adds weights to environmental information and spatial features to improve information utilization, making the model closer to reality and containing more comprehensive information, ultimately improving the accuracy of SST forecasts. In summary, compared with traditional machine learning correction methods such as LR and SVR, as well as deep learning methods such as LSTM, ConvLSTM, and ConvLSTM-AT, CBAM-ConvLSTM has the best performance in correcting SST, verifying the effectiveness of this method.

The time step is an important parameter in the process of learning time series information of the model. Here, the time step refers to the time of input data advance when training the model. For example, timestep = 1 means that the historical data of the previous day is used to correct the SST. Generally, the longer the time step, the more information can be used for correction, but the accumulation of errors will also increase. Therefore, timestep = 1, 3, 5, 7, 10, and 15 are used to correct the SST. Through experimental comparison of indicators such as RMSE and MAPE, the appropriate timestep is determined for correcting the SST. Figure 3 reflects the comparison of evaluation indicators of CBAM-ConvLSTM correcting SST under different time steps. Timestep symbolizes the information of the time dimension, and its value has an impact on the performance of the model. When correcting, comparing the RMSE indicators under different timesteps, the correction effect when timestep = 3 is better than the evaluation indicators when timestep = 1, 5, 7, 10, and 15. After timestep is greater than 10, the results tend to be stable, and the influence of time information on the correction results becomes smaller. It can be seen that when forecasting SST, the amount of information in the spatial and temporal dimensions should be moderate. Too much or too little will affect the performance of the model, so the above experiments are necessary. In summary, timestep = 3 is used to correct SST.

In order to determine the best number of training epochs for the dataset, different epochs were set for experiments. As shown in Figure 4, RMSE reaches a stable state at 300 time points. Therefore, considering the accuracy and performance of the model, 300 training epochs were used in our experiments.

The learning rate is an important hyperparameter that determines whether and when the objective function can converge to the local minimum. A suitable learning rate enables the objective function to converge to the local minimum in a suitable time. It is found that at the beginning of the training process, when using an adaptive optimization algorithm such as the "Adam" algorithm, the model is in a non-convergent state. Then adjust lr and other hyperparameters within the fixed model framework. The first step is to reduce it from 0.1 to 0.001, with a rate of 10. Then, when the learning rate is at the level of 10 ^-2 , the training and validation losses of the model will be in a stable decline. The learning rate is adjusted for experiments, and the experimental results are shown in Figure 5. The figure shows that indicators such as RMSE/MAPE change with lr. According to the root mean square error (RMSE), the optimal learning rate is 0.01. According to MAPE, the optimal learning rate is 0.004. Therefore, in our data set, the best learning rate is between 10-2 and 10-3.

Complexity and training time analysis,The experimental environment is Windows 10, Intel Core i5 11, 2.4GHz, 16GRAM, and the algorithm is implemented using python3.

Table 2 Number of network parameters and training time for each model

Parameters Train(s) Test(s) LSTM 13601 271 1 CONVLSTM 44993 236 1 CONVLSTM-AT 46079 437 1 3DCNN-CONVLSTM-AT 13197 272 1 CBAM-CONVLSTM 13560 223 1

Table 2 lists the training time and number of model parameters of the models used in the experiment. It can be found that the training parameters of the CBAM-CONVLSTM model are about 10 times less than those of the ConvLSTM, which makes the training faster and more suitable for practical applications. The number of parameters of the 3DCNN-CONVLSTM-AT model is close to that of the CBAM-CONVLSTM model, indicating that the CBAM module is very small and will also reduce the model training time. The CBAM-CONVLSTM model we proposed consumes the least time and has fewer parameters, and the model has good performance.

The beneficial effects of the present invention are as follows:

Mining the variation patterns of sea surface temperature data based on spatiotemporal series can achieve better correction results with high correction accuracy.

The CBAM module of the present invention is very small, so that the model training time is reduced, the number of parameters is also small, the training speed is faster, and good performance is achieved.

As used herein, the word "preferred" is intended to be used as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as being more advantageous than other aspects or designs. On the contrary, the use of the word "preferred" is intended to present concepts in a specific way. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" means any one of the naturally included permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

Moreover, although the present disclosure has been shown and described with respect to one or implementations, those skilled in the art will think of equivalent variations and modifications based on the reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations, and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if the structure is not equivalent to the disclosed structure of the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that may be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

The functional units in the embodiments of the present invention may be integrated into a processing module, or each unit may exist physically separately, or multiple or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc. The above-mentioned devices or systems may execute the storage method in the corresponding method embodiment.

To sum up, the above embodiment is an implementation mode of the present invention, but the implementation mode of the present invention is not limited by the embodiment, and any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims (7)

1. A method for correcting the deviation of sea surface temperature numerical forecast based on attention mechanism, characterized in that it comprises the following steps: Step 1: Preprocess the model forecast data and remote sensing satellite sea temperature data to construct the input sequence, including: (1) Obtain model forecast data and remote sensing satellite sea temperature data as reference values, and extract marine environment data; (2) Construct a time column, obtain the number of time features through a sliding window, and then add ocean feature influencing factors, including sea temperature, salinity, and the U and V vectors of water currents, to obtain the input historical SST sequence of the model; (3) Standardize the historical SST series data obtained in the above steps; Step 2: Construct a revised model, including: (1) Spatial feature extraction: Using 3D convolution, the 3D kernel is convolved with a cube formed by superimposing multiple continuous matrices to extract the spatial dependency characteristics of SST data and the relationship between multiple environmental variables; (2) Using the 3D-CBAM attention mechanism on the data sequence after 3D convolution to improve the utilization rate of the spatial features of the 3D convolution network and show the importance of different environmental variables to the results; the 3D-CBAM attention mechanism consists of two parts: a channel attention module and a spatial attention module; Multiply the channel attention module Mc and the input feature matrix F to obtain the input matrix Ms of the spatial attention module; Multiply the spatial attention module Ms and the input F’ of the module to obtain the final generated feature matrix; (3) Input the feature matrix data sequence obtained in the above steps into the ConvLSTM part of the model; use the Attention mechanism to add a custom attention layer after the ConvLSTM network, and use the hidden layer state of each step of the ConvLSTM model to assign the temporal attention weight to the hidden layer state of each time step; Adjust the final ConvLSTM output to get the final corrected result; The attention layer takes the output _ht of each iteration of ConvLSTM as input; then the weight AT( _ht ) of each output vector is obtained through Softmax operation; finally, the attention weight AT( _ht ) is multiplied by the hidden layer state _ht to obtain the final correction result _Yt , which is calculated as follows: Among them, W is the weight matrix (4) Building and training the correction model obtained in the above steps, continuously adjusting the parameters, and selecting the best parameters to obtain the sea surface temperature correction model; Step 3: Use the correction model to correct the model forecast data; (1) Inputting the standardized time series forecast data into the sea surface temperature correction model obtained in step 2 to obtain the corrected output result; The mean square error is used as the loss function, and the calculation formula of the loss function is: Where n represents the number of grid points in the grid data, is the true value data of the grid points, _yi is the network-corrected data; after building the model, set the model input dimension and the time step of the input data; set the model optimizer and learning rate; set the number of hidden neural nodes; set the number of model iterations; continuously adjust the parameters, check the model convergence degree with the model loss, select the convergence parameters optimally, and form the final SST correction model; (2) De-standardize the model output results. The de-standardization method corresponds to the standardization method in step 1, and the corrected SST value is obtained; Step 4: Use the mean absolute error (MAE), mean absolute percentage error (MAPE), mean square error (MSE) and root mean square error (RMSE) evaluation indicators to evaluate the accuracy of the results after model correction. 2. According to the attention mechanism-based sea surface temperature numerical forecast deviation correction method described in claim 1, it is characterized in that a bilinear interpolation method is used to unify the temporal and spatial resolutions of forecast data and remote sensing satellite observation data. 3. According to the method for correcting the deviation of numerical sea surface temperature forecast based on the attention mechanism in claim 1, it is characterized in that, for the historical data sequence X, the SST data at any time t and other marine environmental variables Xt are all grid data of W×H×C specifications, and the input of the entire model is a five-dimensional tensor, expressed as B×T×C×W×H, where B is the number of a batch of training samples, T is the length of the sequence data, W and H are the length and width of the SST field, and C is the number of marine environmental variables added. 4. The method for correcting the deviation of sea surface temperature numerical forecast based on the attention mechanism according to claim 1 is characterized in that the formula for the standardization process is as follows: Among them, X _max and X _min are the maximum and minimum values in the sequence data respectively. 5. The method for correcting the deviation of sea surface temperature numerical forecast based on the attention mechanism according to claim 1 is characterized in that the calculation formula of the channel attention module is as follows: Mc(F)＝σ(MLP(MaxPool3D(F))+MLP(AvgPool3D(F))) Where MLP represents multi-layer perceptron, MaxPool3D represents maximum pooling, AvgPool3D represents average pooling, and F is the input feature matrix; The input matrix Ms of the spatial attention module is: Ms(F) = σ(f ^3×3 ([MaxPool3D(F); AvgPool3D(F)])) Among them, f is a 3×3 convolution operation. 6. The method for correcting the deviation of sea surface temperature numerical forecast based on the attention mechanism according to claim 1, characterized in that the calculation formula of the ConvLSTM network is as follows: _it =σ( _wxi * _xt + _whi * _ht-1 + _wci · _ct-1 +b _i ) _ft = σ(w _xf *x _t +w _hf *h _t-1 +w _cf -c _t-1 +b _f ) c _t = f _t · c _t-1 + _it · tanh(w _xc * x _t + w _hc * h _t-1 + b _c ) o _t =σ(w _xo *x _t +w _ho *c _t-1 +w _co ·c _t +b _o ) h _t = o _t ·tanh(c _t ) Where _it represents the input gate, _ft represents the forget gate, _ot represents the output gate, _ct represents the unit state, w is the weight matrix, _wxi is the weight matrix of the input gate _xt , _wxf is the weight matrix of the forget gate _xt , _wxc is the weight matrix of _xt in the unit state calculation, _who is the weight matrix of the output gate _xt , b is the bias term, _bi is the bias term of the input gate, _bf is the bias term of the forget gate, _bc is the bias term of the unit state, _bo is the bias term of the output gate, _xt is the input matrix, _ht-1 is the hidden layer state at time t-1, and _ct-1 is the memory state at time t-1. 7. The method for correcting the deviation of sea surface temperature numerical forecast based on the attention mechanism according to claim 1 is characterized in that the calculation method of the mean square error MSE, root mean square error RMSE, mean absolute error MAE and mean absolute percentage error MAPE is as follows: Where _yi is the true value, is the estimated value, and n is the number of samples.