CN116068511A - Deep learning-based InSAR large-scale system error correction method - Google Patents
Deep learning-based InSAR large-scale system error correction method Download PDFInfo
- Publication number
- CN116068511A CN116068511A CN202310220889.3A CN202310220889A CN116068511A CN 116068511 A CN116068511 A CN 116068511A CN 202310220889 A CN202310220889 A CN 202310220889A CN 116068511 A CN116068511 A CN 116068511A
- Authority
- CN
- China
- Prior art keywords
- data
- phase
- longitude
- neural network
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000012937 correction Methods 0.000 title claims abstract description 20
- 238000013135 deep learning Methods 0.000 title claims abstract description 14
- 238000003062 neural network model Methods 0.000 claims abstract description 34
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 238000001914 filtration Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000005305 interferometry Methods 0.000 claims description 5
- 230000009897 systematic effect Effects 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 238000007670 refining Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 description 7
- 230000009286 beneficial effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 241001420622 Meris Species 0.000 description 1
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000005433 ionosphere Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- CCEKAJIANROZEO-UHFFFAOYSA-N sulfluramid Chemical group CCNS(=O)(=O)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)C(F)(F)F CCEKAJIANROZEO-UHFFFAOYSA-N 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B15/00—Measuring arrangements characterised by the use of electromagnetic waves or particle radiation, e.g. by the use of microwaves, X-rays, gamma rays or electrons
- G01B15/06—Measuring arrangements characterised by the use of electromagnetic waves or particle radiation, e.g. by the use of microwaves, X-rays, gamma rays or electrons for measuring the deformation in a solid
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
- G01S13/90—Radar or analogous systems specially adapted for specific applications for mapping or imaging using synthetic aperture techniques, e.g. synthetic aperture radar [SAR] techniques
- G01S13/9021—SAR image post-processing techniques
- G01S13/9023—SAR image post-processing techniques combined with interferometric techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S7/417—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section involving the use of neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
本发明公开了一种基于深度学习的InSAR大尺度系统误差改正方法,包括以下步骤:收集覆盖目标区域数据并对其进行数据预处理,获得地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据;根据相关数据之间的关系,搭建深度神经网络模型;将经过预处理获得的数据输入深度神经网络模型中,获得模拟的大尺度系统误差解缠相位;获得改正后的解缠相位;反演形变参数,获得改正后的形变结果。本发明提供了一种能够同时对地形相关大气相位、轨道残余相位及部分的湍流相位进行改正的模型,且在高陡山区、丘陵、平地区域均可实现InSAR大尺度系统误差改正。
The invention discloses an InSAR large-scale system error correction method based on deep learning, which includes the following steps: collecting and preprocessing the data covering the target area, obtaining unwrapped phase data after geocoding, and DEM data after clipping and longitude and latitude data; according to the relationship between relevant data, build a deep neural network model; input the preprocessed data into the deep neural network model, and obtain the simulated large-scale system error unwrapping phase; obtain the corrected unwrapping phase ; Invert the deformation parameters to obtain the corrected deformation results. The invention provides a model capable of simultaneously correcting terrain-related atmospheric phases, orbital residual phases, and partial turbulent phases, and can realize InSAR large-scale system error correction in high and steep mountainous areas, hills, and flat areas.
Description
技术领域Technical Field
本发明涉及合成孔径雷达干涉测量技术领域,尤其涉及一种基于深度学习的InSAR大尺度系统误差改正方法。The present invention relates to the field of synthetic aperture radar interferometry technology, and in particular to an InSAR large-scale system error correction method based on deep learning.
背景技术Background Art
合成孔径雷达干涉测量技术是近三十年来迅速发展的一种基于星载传感器对地观测测量技术。具有大范围、全天时和全天候的形变监测能力。随着SAR影像的时间分辨率与空间分辨率的不断提升、多源多时相的SAR影像可供选择与InSAR时序算法的不断发展与进步,InSAR技术在地质灾害隐患早期识别与监测领域得到了广泛的应用,如:火山监测、地面沉降监测、滑坡监测、地震监测以及冰川运动。该技术可以有效的弥补传统测量方法的不足,然而由于大尺度相位误差的存在,如:电离层、地形相关大气相位、轨道残余相位。这些误差会给形变测量精度带来严重影响。如:当大气出现20%的相对湿度时空变化会对形变结果造成几分米的误差;轨道相位与大尺度形变的空间特征类似,会严重影响大尺度形变的监测,如地震、火山运动。因此,综合去除大尺度的相位误差具有重要意义。Synthetic aperture radar interferometry is a ground observation measurement technology based on satellite-borne sensors that has developed rapidly in the past three decades. It has the ability to monitor deformation over a large range, all day and all weather. With the continuous improvement of the temporal and spatial resolution of SAR images, the availability of multi-source and multi-temporal SAR images, and the continuous development and progress of InSAR time series algorithms, InSAR technology has been widely used in the field of early identification and monitoring of geological disaster hazards, such as volcano monitoring, ground subsidence monitoring, landslide monitoring, earthquake monitoring, and glacier movement. This technology can effectively make up for the shortcomings of traditional measurement methods. However, due to the existence of large-scale phase errors, such as ionosphere, terrain-related atmospheric phase, and orbital residual phase. These errors will have a serious impact on the accuracy of deformation measurement. For example, when the relative humidity of 20% changes in the atmosphere in time and space, it will cause an error of several decimeters in the deformation results; the orbital phase is similar to the spatial characteristics of large-scale deformation, which will seriously affect the monitoring of large-scale deformation, such as earthquakes and volcanic movements. Therefore, it is of great significance to comprehensively remove large-scale phase errors.
为了去除大尺度InSAR系统误差,在过去的几十年中,不少学者提出了大量针对不同大尺度系统误差成分的方法。针对地形相关大气相位提出了基于外部数据的改正方法,如数值气象产品(Weathermodels)、GNSS data、光谱分析法(MODIS, MERIS)、亦或是它们之间的组合;其次是基于改正模型的方法,如:线性改正法与幂率改正法。而针对轨道残余相位提出了基于卫星轨道轨迹补偿模型,对InSAR成像几何中的基线误差进行精确估计;其次是基于轨道相位在空间域与频域的特征;然后是基于外部数据的辅助,如GNSS;最后是一种数学拟合的方法,主要有线性拟合、多项式拟合等模型。In order to remove large-scale InSAR system errors, many scholars have proposed a large number of methods for different large-scale system error components in the past few decades. For terrain-related atmospheric phase, a correction method based on external data is proposed, such as numerical meteorological products (Weather models), GNSS data, spectral analysis (MODIS, MERIS), or a combination of them; followed by methods based on correction models, such as linear correction method and power-rate correction method. For orbital residual phase, a satellite orbit trajectory compensation model is proposed to accurately estimate the baseline error in InSAR imaging geometry; followed by the characteristics of orbital phase in the spatial domain and frequency domain; then based on the assistance of external data, such as GNSS; and finally a mathematical fitting method, mainly including linear fitting, polynomial fitting and other models.
上述提出的改正方法已被证明在去除地形相关大气相位和轨道残余相位取得了一定程度的成功,但是这些方法都受限于其固有的局限。传统的基于线性模型、多项式等模型的方法不能完全描述每幅干涉对与轨道特征,且会受到空间特征与轨道相位和地形相关大气相位相似形变的影响。The correction methods proposed above have been proven to be successful to some extent in removing terrain-related atmospheric phase and orbital residual phase, but these methods are subject to their inherent limitations. Traditional methods based on linear models, polynomial models, etc. cannot fully describe the characteristics of each interferometer pair and orbit, and will be affected by the similar deformation of spatial characteristics and orbital phase and terrain-related atmospheric phase.
发明内容Summary of the invention
针对现有技术中的上述不足,本发明提供的一种基于深度学习的InSAR大尺度系统误差改正方法,提供了一种能够同时对地形相关大气相位、轨道残余相位及部分的湍流相位进行改正的模型,解决了传统方法受限的问题。In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for correcting InSAR large-scale system errors based on deep learning, which provides a model that can simultaneously correct the terrain-related atmospheric phase, orbital residual phase and part of the turbulence phase, solving the problem of limitations of traditional methods.
为了达到上述发明目的,本发明采用的技术方案为:一种基于深度学习的InSAR大尺度系统误差改正方法,其特征在于,所述方法包括以下步骤:In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a method for correcting large-scale system errors of InSAR based on deep learning, characterized in that the method comprises the following steps:
S1:收集覆盖目标区域的SAR数据、DEM数据和经纬度数据,并对其进行数据预处理,剔除相干性差的区域,获得地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据;S1: Collect SAR data, DEM data and latitude and longitude data covering the target area, and perform data preprocessing to remove areas with poor coherence, and obtain geocoded unwrapped phase data, cropped DEM data and latitude and longitude data;
S2:根据相关数据之间的关系,搭建深度神经网络模型,所述相关数据包括DEM数据、经纬度数据、地形相关大气相位、轨道残余相位和湍流相位;S2: Building a deep neural network model based on the relationship between relevant data, wherein the relevant data includes DEM data, longitude and latitude data, terrain-related atmospheric phase, orbital residual phase and turbulence phase;
S3:将经过预处理获得的地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据输入深度神经网络模型中,获得模拟的大尺度系统误差解缠相位;S3: The geocoded unwrapped phase data, cropped DEM data and longitude and latitude data obtained after preprocessing are input into the deep neural network model to obtain the simulated large-scale system error unwrapped phase;
S4:将原始经过地理编码后的解缠相位与经过深度神经网络模型模拟后的大尺度系统误差解缠相位相减得到改正后的解缠相位;S4: Subtract the original geocoded unwrapped phase from the large-scale system error unwrapped phase simulated by the deep neural network model to obtain the corrected unwrapped phase;
S5:将改正后的解缠相位经过反地理编码到SAR坐标系,通过反演形变参数,获得改正后的形变结果。S5: The corrected unwrapped phase is de-geocoded to the SAR coordinate system, and the corrected deformation result is obtained by inverting the deformation parameters.
上述方案的有益效果是:通过上述技术方案,综合考虑了轨道相位与地形相关大气相位,得到其与解缠相位的函数关系,剔除地形相关大气相位与轨道相位的同时,考虑了湍流相位的空间特性,可以剔除部分的湍流相位,同时对地形相关大气相位、轨道残余相位及部分的湍流相位进行改正,解决了传统方法受限的问题,能够更加准确的识别形变。The beneficial effect of the above scheme is: through the above technical scheme, the orbital phase and the terrain-related atmospheric phase are comprehensively considered to obtain their functional relationship with the unwrapping phase. While eliminating the terrain-related atmospheric phase and the orbital phase, the spatial characteristics of the turbulent phase are considered, and part of the turbulent phase can be eliminated. At the same time, the terrain-related atmospheric phase, the orbital residual phase and part of the turbulent phase are corrected, which solves the problems limited by the traditional method and can identify deformation more accurately.
进一步地,S1中对SAR数据进行InSAR数据预处理,包括以下步骤:Furthermore, in S1, the SAR data is preprocessed into InSAR data, including the following steps:
S1-1:获取同一研究区按时间排序的两景SLC影像,并选择其中一景作为主影像;S1-1: Obtain two SLC images of the same study area in time order and select one of them as the main image;
S1-2:通过对影像在距离和方位上设置多视比10:2来减小斑点噪声的影响,并根据噪声的大小设置相应滤波参数,经滤波得到滤波后相干性数据;S1-2: The influence of speckle noise is reduced by setting the multi-view ratio of 10:2 in the distance and azimuth of the image, and the corresponding filtering parameters are set according to the size of the noise, and the filtered coherence data is obtained by filtering;
S1-3:通过滤波后相干性数据的数据分布设置一个解缠阈值,保证训练数据充足,防止解缠错误数据的影响;S1-3: Setting a disentanglement threshold based on the data distribution of the filtered coherence data to ensure sufficient training data and prevent the influence of disentanglement error data;
S1-4:根据解缠阈值,选择最小费用流法对所选影像干涉图计算得到的被包裹相位进行相位解缠;S1-4: According to the unwrapping threshold, the minimum cost flow method is selected to perform phase unwrapping on the wrapped phase calculated from the selected image interferogram;
S1-5:通过地理编码将SAR坐标系下的解缠相位数据转到与裁剪后的DEM数据和经纬度数据相同的地理坐标下,获得地理编码后的解缠相位数据。S1-5: The unwrapped phase data in the SAR coordinate system is transferred to the same geographic coordinates as the cropped DEM data and longitude and latitude data through geocoding to obtain the geocoded unwrapped phase data.
上述进一步方案的有益效果是:通过上述技术方案,对收集到的数据进行预处理,从而得到训练深度神经网络需要的数据。The beneficial effect of the above further scheme is: through the above technical scheme, the collected data is preprocessed to obtain the data required for training the deep neural network.
进一步地,S2中所述深度神经网络模型搭建包括以下步骤:Furthermore, the deep neural network model construction in S2 includes the following steps:
S2-1:将深度神经网络模型的全连接层网络作为提取输入DEM数据和经纬度数据信息的特征模块,结合DEM数据、经纬度数据与地形相关大气相位、轨道残余相位及湍流相位的关系,选择深度神经网络模型的通道注意力机制模块作为数据的加权特征提取模块;S2-1: The fully connected layer network of the deep neural network model is used as a feature module to extract the input DEM data and longitude and latitude data information. Combined with the relationship between DEM data, longitude and latitude data and terrain-related atmospheric phase, orbital residual phase and turbulence phase, the channel attention mechanism module of the deep neural network model is selected as the weighted feature extraction module of the data;
S2-2:通过加权特征提取模块进行特征压缩操作,将空间维度进行特征压缩,再通过还原操作,为每个相关数据特征维度生成相应的权重,包括以下公式:S2-2: Perform feature compression operation through weighted feature extraction module, compress the spatial dimension, and then generate corresponding weights for each relevant data feature dimension through restoration operation, including the following formula:
特征值压缩公式如下所示:The eigenvalue compression formula is as follows:
其中,为输入的数据,和分别为输入数据的长和宽,为压缩操作,为原始数据压缩后第个二维矩阵,和均为图像像素坐标;in, For the input data, and are the length and width of the input data respectively, For compression operation, The original data is compressed A two-dimensional matrix, and All are image pixel coordinates;
权重计算公式如下所示:The weight calculation formula is as follows:
其中,为还原操作,为RELU激活函数,为RELU激活函数参数的降维层,为RELU激活函数参数的升维层,为特征通道权重,为原始数据压缩后的结果,为全局平均池化;in, To restore the operation, is the RELU activation function, is the dimension reduction layer of the RELU activation function parameters, is the dimension-raising layer of the RELU activation function parameters, is the feature channel weight, is the result after the original data is compressed. is global average pooling;
S2-3:将权重作用于原来的每个相关数据特征通道,公式如下所示:S2-3: Apply the weight to each original relevant data feature channel. The formula is as follows:
为特征映射,为尺度因子,为最终得到的图像数据,为特征映射与尺度因子在通道上的卷积。 is the feature map, is the scale factor, is the final image data, is the convolution of the feature map and the scale factor on the channel.
上述进一步方案的有益效果是:通过上述技术方案,顾及使用的数据,搭建深度神经网络模型,便于后续获得模拟的大尺度系统误差解缠相位,同时将权重作用于原来的每个相关数据特征通道,有利于学习到不同数据通道的重要性。The beneficial effect of the above further scheme is: through the above technical scheme, taking into account the data used, a deep neural network model is built to facilitate the subsequent acquisition of the simulated large-scale system error unwrapped phase, and at the same time, weights are applied to each of the original relevant data feature channels, which is conducive to learning the importance of different data channels.
进一步地,深度神经网络模型选用MSE作为损失函数进行参数评估,选择Adam优化算法作为参数优化算法。Furthermore, the deep neural network model selects MSE as the loss function for parameter evaluation and the Adam optimization algorithm as the parameter optimization algorithm.
上述进一步方案的有益效果是:考虑到地形相关大气相位与地形函数关系在空间上是变化的,即不同地区地形与大气相位的函数关系不同,通过上述技术方案,使得模型可以拟合一个较优的全局参数。The beneficial effect of the above further scheme is: taking into account that the relationship between the terrain-related atmospheric phase and the terrain function varies in space, that is, the functional relationship between the terrain and the atmospheric phase in different regions is different, through the above technical scheme, the model can fit a better global parameter.
进一步地,S3中包括以下步骤:Furthermore, S3 includes the following steps:
S3-1:将经过预处理获得的地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据输入至深度神经网络模型中,通过深度神经网络模型的全连接模块提取DEM数据和经纬度数据的空间特征,并通过通道注意力机制模块获得DEM数据、经纬度数据和解缠相位数据的权重;S3-1: The geocoded unwrapped phase data, cropped DEM data and longitude and latitude data obtained after preprocessing are input into the deep neural network model, the spatial features of the DEM data and longitude and latitude data are extracted through the fully connected module of the deep neural network model, and the weights of the DEM data, longitude and latitude data and unwrapped phase data are obtained through the channel attention mechanism module;
S3-2:根据DEM数据、经纬度数据和解缠相位数据的权重获得DEM数据、经纬度数据和解缠相位数据之间的函数关系,得到预训练权重;S3-2: Obtain a functional relationship between the DEM data, the longitude and latitude data, and the unwrapped phase data according to the weights of the DEM data, the longitude and latitude data, and the unwrapped phase data to obtain a pre-trained weight;
S3-3:根据预训练权重,通过深度神经网络模型获得最终的预测结果,得到模拟的大尺度系统误差解缠相位。S3-3: Based on the pre-trained weights, the final prediction result is obtained through the deep neural network model to obtain the simulated large-scale system error disentanglement phase.
上述进一步方案的有益效果是:通过上述技术方案,将数据输入深度神经网络模型中处理后,得到去除大尺度系统误差后的解缠相位。The beneficial effect of the above further scheme is: through the above technical scheme, after the data is input into the deep neural network model for processing, the unwrapped phase is obtained after removing the large-scale system error.
进一步地,S5中包括以下步骤:Furthermore, S5 includes the following steps:
S5-1:将改正后的解缠相位重新编码到SAR坐标下,进行DInSAR反演形变操作,通过对稳定的地面控制点的选择和细化进行轨道相位精炼和重去平,消除差分干涉中的轨道误差相位,再反演形变参数;S5-1: Re-encode the corrected unwrapped phase to SAR coordinates, perform DInSAR inversion deformation operation, refine and re-level the orbit phase by selecting and refining stable ground control points, eliminate the orbit error phase in differential interferometry, and then invert the deformation parameters;
S5-2:将反演形变结果进行地理编码,获得系统误差改正后的形变结果。S5-2: Geocode the inverted deformation results to obtain the deformation results after correcting the systematic errors.
上述进一步方案的有益效果是:将得到的改正后的解缠相位重新编码到SAR坐标下,继续相应的DInSAR反演形变操作,获得系统误差改正后的形变结果图,经对比,可以明显看到本发明的优势。The beneficial effect of the above further scheme is: the obtained corrected unwrapped phase is re-encoded to the SAR coordinates, and the corresponding DInSAR inversion deformation operation is continued to obtain the deformation result map after the system error is corrected. After comparison, the advantages of the present invention can be clearly seen.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为一种基于深度学习的InSAR大尺度系统误差改正方法流程图。Figure 1 is a flow chart of a method for correcting large-scale system errors of InSAR based on deep learning.
图2为一种基于深度学习的InSAR大尺度系统误差改正方法技术流程图。Figure 2 is a technical flow chart of a deep learning-based InSAR large-scale system error correction method.
图3为FC-Net部分InSAR大尺度误差改正结果图。Figure 3 shows some of the InSAR large-scale error correction results using FC-Net.
图4为FC-Net改正前后形变结果对比图。Figure 4 is a comparison of the deformation results before and after FC-Net correction.
具体实施方式DETAILED DESCRIPTION
下面结合附图和具体实施例对本发明做进一步说明。The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.
如图1所示,一种基于深度学习的InSAR大尺度系统误差改正方法,其特征在于,所述方法包括以下步骤:As shown in FIG1 , a method for correcting large-scale system errors of InSAR based on deep learning is characterized in that the method comprises the following steps:
S1:收集覆盖目标区域的SAR数据、DEM数据和经纬度数据,并对其进行数据预处理,剔除相干性差的区域,获得地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据;S1: Collect SAR data, DEM data and latitude and longitude data covering the target area, and perform data preprocessing to remove areas with poor coherence, and obtain geocoded unwrapped phase data, cropped DEM data and latitude and longitude data;
S2:根据相关数据之间的关系,搭建深度神经网络模型,所述相关数据包括DEM数据、经纬度数据、地形相关大气相位、轨道残余相位和湍流相位;S2: Building a deep neural network model based on the relationship between relevant data, wherein the relevant data includes DEM data, longitude and latitude data, terrain-related atmospheric phase, orbital residual phase and turbulence phase;
S3:将经过预处理获得的地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据输入深度神经网络模型中,获得模拟的大尺度系统误差解缠相位;S3: The geocoded unwrapped phase data, cropped DEM data and longitude and latitude data obtained after preprocessing are input into the deep neural network model to obtain the simulated large-scale system error unwrapped phase;
S4:将原始经过地理编码后的解缠相位与经过深度神经网络模型模拟后的大尺度系统误差解缠相位相减得到改正后的解缠相位;S4: Subtract the original geocoded unwrapped phase from the large-scale system error unwrapped phase simulated by the deep neural network model to obtain the corrected unwrapped phase;
S5:将改正后的解缠相位经过反地理编码到SAR坐标系,通过反演形变参数,获得改正后的形变结果。S5: The corrected unwrapped phase is de-geocoded to the SAR coordinate system, and the corrected deformation result is obtained by inverting the deformation parameters.
除此之外,S1中对SAR数据进行InSAR数据预处理,包括以下步骤:In addition, S1 performs InSAR data preprocessing on the SAR data, including the following steps:
S1-1:获取同一研究区按时间排序的两景SLC影像,并选择其中一景作为主影像;S1-1: Obtain two SLC images of the same study area in time order and select one of them as the main image;
S1-2:通过对影像在距离和方位上设置多视比10:2来减小斑点噪声的影响,并根据噪声的大小设置相应滤波参数,经滤波得到滤波后相干性数据;S1-2: The influence of speckle noise is reduced by setting the multi-view ratio of 10:2 in the distance and azimuth of the image, and the corresponding filtering parameters are set according to the size of the noise, and the filtered coherence data is obtained by filtering;
S1-3:通过滤波后相干性数据的数据分布设置一个解缠阈值,保证训练数据充足,防止解缠错误数据的影响;S1-3: Setting a disentanglement threshold based on the data distribution of the filtered coherence data to ensure sufficient training data and prevent the influence of disentanglement error data;
S1-4:根据解缠阈值,选择最小费用流法对所选影像干涉图计算得到的被包裹相位进行相位解缠;S1-4: According to the unwrapping threshold, the minimum cost flow method is selected to perform phase unwrapping on the wrapped phase calculated from the selected image interferogram;
S1-5:通过地理编码将SAR坐标系下的解缠相位数据转到与裁剪后的DEM数据和经纬度数据相同的地理坐标下,获得地理编码后的解缠相位数据。S1-5: The unwrapped phase data in the SAR coordinate system is transferred to the same geographic coordinates as the cropped DEM data and longitude and latitude data through geocoding to obtain the geocoded unwrapped phase data.
S2中所述深度神经网络模型搭建包括以下步骤:The deep neural network model construction described in S2 includes the following steps:
S2-1:将深度神经网络模型的全连接层网络作为提取输入DEM数据和经纬度数据信息的特征模块,结合DEM数据、经纬度数据与地形相关大气相位、轨道残余相位及湍流相位的关系,选择深度神经网络模型的通道注意力机制模块作为数据的加权特征提取模块;S2-1: The fully connected layer network of the deep neural network model is used as a feature module to extract the input DEM data and longitude and latitude data information. Combined with the relationship between DEM data, longitude and latitude data and terrain-related atmospheric phase, orbital residual phase and turbulence phase, the channel attention mechanism module of the deep neural network model is selected as the weighted feature extraction module of the data;
S2-2:通过加权特征提取模块进行特征压缩操作,将空间维度进行特征压缩,再通过还原操作,为每个相关数据特征维度生成相应的权重,包括以下公式:S2-2: Perform feature compression operation through weighted feature extraction module, compress the spatial dimension, and then generate corresponding weights for each relevant data feature dimension through restoration operation, including the following formula:
特征值压缩公式如下所示:The eigenvalue compression formula is as follows:
其中,为输入的数据,和分别为输入数据的长和宽,为压缩操作,为原始数据压缩后第个二维矩阵,和均为图像像素坐标;in, For the input data, and are the length and width of the input data, respectively. For compression operation, The original data is compressed A two-dimensional matrix, and All are image pixel coordinates;
权重计算公式如下所示:The weight calculation formula is as follows:
其中,为还原操作,为RELU激活函数,为RELU激活函数参数的降维层,为RELU激活函数参数的升维层,为特征通道权重,为原始数据压缩后的结果,为全局平均池化;in, To restore the operation, is the RELU activation function, is the dimension reduction layer of the RELU activation function parameters, is the dimension-raising layer of the RELU activation function parameters, is the feature channel weight, is the result after the original data is compressed. is global average pooling;
S2-3:将权重作用于原来的每个相关数据特征通道,公式如下所示:S2-3: Apply the weight to each original relevant data feature channel. The formula is as follows:
为特征映射,为尺度因子,为最终得到的图像数据,为特征映射与尺度因子在通道上的卷积。 is the feature map, is the scale factor, is the final image data, is the convolution of the feature map and the scale factor on the channel.
深度神经网络模型选用MSE作为损失函数进行参数评估,选择Adam优化算法作为参数优化算法。The deep neural network model uses MSE as the loss function for parameter evaluation and the Adam optimization algorithm as the parameter optimization algorithm.
S3中包括以下步骤:S3 includes the following steps:
S3-1:将经过预处理获得的地理编码后的解缠相位数据、裁剪后的DEM数据和经纬度数据输入至深度神经网络模型中,通过深度神经网络模型的全连接模块提取DEM数据和经纬度数据的空间特征,并通过通道注意力机制模块获得DEM数据、经纬度数据和解缠相位数据的权重;S3-1: The geocoded unwrapped phase data, cropped DEM data and longitude and latitude data obtained after preprocessing are input into the deep neural network model, the spatial features of the DEM data and longitude and latitude data are extracted through the fully connected module of the deep neural network model, and the weights of the DEM data, longitude and latitude data and unwrapped phase data are obtained through the channel attention mechanism module;
S3-2:根据DEM数据、经纬度数据和解缠相位数据的权重获得DEM数据、经纬度数据和解缠相位数据之间的函数关系,得到预训练权重;S3-2: Obtain a functional relationship between the DEM data, the longitude and latitude data, and the unwrapped phase data according to the weights of the DEM data, the longitude and latitude data, and the unwrapped phase data to obtain a pre-trained weight;
S3-3:根据预训练权重,通过深度神经网络模型获得最终的预测结果,得到模拟的大尺度系统误差解缠相位。S3-3: Based on the pre-trained weights, the final prediction result is obtained through the deep neural network model to obtain the simulated large-scale system error disentanglement phase.
S5中包括以下步骤:S5 includes the following steps:
S5-1:将改正后的解缠相位重新编码到SAR坐标下,进行DInSAR反演形变操作,通过对稳定的地面控制点的选择和细化进行轨道相位精炼和重去平,消除差分干涉中的轨道误差相位,再反演形变参数;S5-1: Re-encode the corrected unwrapped phase to SAR coordinates, perform DInSAR inversion deformation operation, refine and re-level the orbit phase by selecting and refining stable ground control points, eliminate the orbit error phase in differential interferometry, and then invert the deformation parameters;
S5-2:将反演形变结果进行地理编码,获得系统误差改正后的形变结果。S5-2: Geocode the inverted deformation results to obtain the deformation results after correcting the systematic errors.
在本发明的一个实施例中,如图2所示,收集覆盖目标区域的SAR影像数据和精密轨道数据,经配准处理获得SLC数据,形成原始干涉对,同时使用外部高程数据去除平地相位与地形相位,进而形成差分干涉对,经最小费用流解缠算法将缠绕相位解包裹并将其地理编码,将经过预处理的地理编码后的解缠相位、高程及经纬度数据输入至深度神经网络得到模拟的解缠相位,包括模拟的地形相关大气相位和模拟的轨道残余相位,然后将经地理编码后的解缠相位与网络模型模拟的解缠相位相减获得改正后的解缠相位,将改正后的解缠相位经反地理编码到SAR坐标系,反演形变参数,得到改正后的形变结果。本实施例中采用全连接通道注意力网络作为深度神经网络。In one embodiment of the present invention, as shown in FIG2 , SAR image data and precise orbit data covering the target area are collected, SLC data is obtained through registration processing to form an original interference pair, and external elevation data is used to remove the flat ground phase and terrain phase, thereby forming a differential interference pair, and the entangled phase is unwrapped and geocoded by the minimum cost flow unwrapping algorithm, and the pre-processed geocoded unwrapped phase, elevation and longitude and latitude data are input into the deep neural network to obtain a simulated unwrapped phase, including a simulated terrain-related atmospheric phase and a simulated orbital residual phase, and then the geocoded unwrapped phase is subtracted from the unwrapped phase simulated by the network model to obtain a corrected unwrapped phase, and the corrected unwrapped phase is inversely geocoded to the SAR coordinate system, and the deformation parameters are inverted to obtain the corrected deformation result. In this embodiment, a fully connected channel attention network is used as a deep neural network.
本发明提供了一种在高陡山区、丘陵和平地区域均可实现InSAR大尺度系统误差改正的方法,提出了一种能够同时对地形相关大气相位、轨道残余相位及部分的湍流相位进行改正的模型,该模型对InSAR获取实际形变速率和正确解译具有重要意义。图3提供了误差改正的结果,给出滤波后干涉对、原始的解缠相位、模拟的解缠相位和改正后的解缠相位结果对比图,由图3和图4(图4中a和c为改正前形变结果,b和d为改正后形变结果)可以看出,本方案在使用DInSAR进行形变监测时,通过使用该方法可以极大的减弱地形相关大气相位、轨道残余相位以及部分的小尺度的湍流相位的影响,从而可以识别从受到大气影响、轨道相位影响的区域识别出形变。The present invention provides a method for correcting large-scale InSAR system errors in steep mountainous areas, hills and plain areas, and proposes a model that can simultaneously correct terrain-related atmospheric phases, orbital residual phases and part of the turbulent phases. The model is of great significance for InSAR to obtain the actual deformation rate and correct interpretation. Figure 3 provides the results of error correction, and gives a comparison chart of the interference pair after filtering, the original unwrapped phase, the simulated unwrapped phase and the corrected unwrapped phase results. It can be seen from Figures 3 and 4 (a and c in Figure 4 are the deformation results before correction, and b and d are the deformation results after correction) that when using DInSAR for deformation monitoring, this scheme can greatly reduce the influence of terrain-related atmospheric phases, orbital residual phases and part of the small-scale turbulent phases by using this method, so that deformation can be identified from areas affected by the atmosphere and orbital phase.
本领域的普通技术人员将会意识到,这里所述的实施例是为了帮助读者理解本发明的原理,应被理解为本发明的保护范围并不局限于这样的特别陈述和实施例。本领域的普通技术人员可以根据本发明公开的这些技术启示做出各种不脱离本发明实质的其它各种具体变形和组合,这些变形和组合仍然在发明的保护范围内。Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the invention.
Claims (6)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310220889.3A CN116068511B (en) | 2023-03-09 | 2023-03-09 | A large-scale system error correction method for InSAR based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310220889.3A CN116068511B (en) | 2023-03-09 | 2023-03-09 | A large-scale system error correction method for InSAR based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116068511A true CN116068511A (en) | 2023-05-05 |
| CN116068511B CN116068511B (en) | 2023-06-13 |
Family
ID=86171652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310220889.3A Active CN116068511B (en) | 2023-03-09 | 2023-03-09 | A large-scale system error correction method for InSAR based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116068511B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112816983A (en) * | 2021-01-06 | 2021-05-18 | 中南大学 | Time sequence InSAR turbulence atmospheric delay correction method based on optimized interferogram set |
| CN119936921A (en) * | 2025-04-09 | 2025-05-06 | 内蒙古电力(集团)有限责任公司内蒙古电力经济技术研究院分公司 | A transmission line tower monitoring method and system based on Beidou positioning |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109541596A (en) * | 2018-11-28 | 2019-03-29 | 中国电子科技集团公司电子科学研究院 | InSAR image processing method and device based on deep learning algorithm |
| CN110332990A (en) * | 2019-05-24 | 2019-10-15 | 深圳大学 | A method and system for real-time compensation of atmospheric turbulence based on deep learning |
| CN111915570A (en) * | 2020-07-10 | 2020-11-10 | 云南电网有限责任公司带电作业分公司 | Atmospheric delay estimation method based on back propagation neural network |
| CN111998766A (en) * | 2020-08-31 | 2020-11-27 | 同济大学 | Surface deformation inversion method based on time sequence InSAR technology |
| CN112748430A (en) * | 2020-12-02 | 2021-05-04 | 北京理工大学 | Multi-base atmospheric phase joint compensation method based on space division |
| WO2021146775A1 (en) * | 2020-01-23 | 2021-07-29 | Ied Foundation Pty Ltd | Systems and methods for processing gnss data streams for determination of hardware and atmosphere-delays |
| CN113238227A (en) * | 2021-05-10 | 2021-08-10 | 电子科技大学 | Improved least square phase unwrapping method and system combined with deep learning |
| CN113311433A (en) * | 2021-05-28 | 2021-08-27 | 北京航空航天大学 | InSAR interferometric phase two-step unwrapping method combining quality map and minimum cost flow |
| CN114265062A (en) * | 2021-11-11 | 2022-04-01 | 电子科技大学 | InSAR phase unwrapping method based on phase gradient estimation network |
| CN114280608A (en) * | 2022-03-07 | 2022-04-05 | 成都理工大学 | Method and system for removing DInSAR elevation-related atmospheric effect |
| CN115270608A (en) * | 2022-07-09 | 2022-11-01 | 自然资源部第二海洋研究所 | Prediction method of coastal zone land subsidence based on ARIMA and LSTM |
| CN115272683A (en) * | 2022-07-25 | 2022-11-01 | 广西科技大学 | A phase unwrapping method for central differential information filtering based on deep learning |
| CN115293223A (en) * | 2022-05-17 | 2022-11-04 | 电子科技大学长三角研究院(湖州) | Method and system for optimizing detail loss of phase unwrapping in deep learning and storage medium |
| CN115423180A (en) * | 2022-08-31 | 2022-12-02 | 内蒙古大学 | Surface deformation monitoring and predicting method |
| CN115545100A (en) * | 2022-09-27 | 2022-12-30 | 重庆三峡学院 | GB-InSAR atmospheric phase compensation method based on LSTM |
| CN115629384A (en) * | 2022-12-08 | 2023-01-20 | 中南大学 | A correction method and related equipment for timing InSAR errors |
-
2023
- 2023-03-09 CN CN202310220889.3A patent/CN116068511B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109541596A (en) * | 2018-11-28 | 2019-03-29 | 中国电子科技集团公司电子科学研究院 | InSAR image processing method and device based on deep learning algorithm |
| CN110332990A (en) * | 2019-05-24 | 2019-10-15 | 深圳大学 | A method and system for real-time compensation of atmospheric turbulence based on deep learning |
| WO2021146775A1 (en) * | 2020-01-23 | 2021-07-29 | Ied Foundation Pty Ltd | Systems and methods for processing gnss data streams for determination of hardware and atmosphere-delays |
| CN111915570A (en) * | 2020-07-10 | 2020-11-10 | 云南电网有限责任公司带电作业分公司 | Atmospheric delay estimation method based on back propagation neural network |
| CN111998766A (en) * | 2020-08-31 | 2020-11-27 | 同济大学 | Surface deformation inversion method based on time sequence InSAR technology |
| CN112748430A (en) * | 2020-12-02 | 2021-05-04 | 北京理工大学 | Multi-base atmospheric phase joint compensation method based on space division |
| CN113238227A (en) * | 2021-05-10 | 2021-08-10 | 电子科技大学 | Improved least square phase unwrapping method and system combined with deep learning |
| CN113311433A (en) * | 2021-05-28 | 2021-08-27 | 北京航空航天大学 | InSAR interferometric phase two-step unwrapping method combining quality map and minimum cost flow |
| CN114265062A (en) * | 2021-11-11 | 2022-04-01 | 电子科技大学 | InSAR phase unwrapping method based on phase gradient estimation network |
| CN114280608A (en) * | 2022-03-07 | 2022-04-05 | 成都理工大学 | Method and system for removing DInSAR elevation-related atmospheric effect |
| CN115293223A (en) * | 2022-05-17 | 2022-11-04 | 电子科技大学长三角研究院(湖州) | Method and system for optimizing detail loss of phase unwrapping in deep learning and storage medium |
| CN115270608A (en) * | 2022-07-09 | 2022-11-01 | 自然资源部第二海洋研究所 | Prediction method of coastal zone land subsidence based on ARIMA and LSTM |
| CN115272683A (en) * | 2022-07-25 | 2022-11-01 | 广西科技大学 | A phase unwrapping method for central differential information filtering based on deep learning |
| CN115423180A (en) * | 2022-08-31 | 2022-12-02 | 内蒙古大学 | Surface deformation monitoring and predicting method |
| CN115545100A (en) * | 2022-09-27 | 2022-12-30 | 重庆三峡学院 | GB-InSAR atmospheric phase compensation method based on LSTM |
| CN115629384A (en) * | 2022-12-08 | 2023-01-20 | 中南大学 | A correction method and related equipment for timing InSAR errors |
Non-Patent Citations (2)
| Title |
|---|
| WENFEI MAO等: "An InSAR Ionospheric Correction Method Based on Variance Component Estimation With Integration of MAI and RSS Measurements", 《IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING》, vol. 14, pages 1423 - 1433, XP011830241, DOI: 10.1109/JSTARS.2020.3045267 * |
| 李立中等: "系统噪声对 InSAR 相位解缠影响的模拟分析", 《测绘》, vol. 45, no. 6, pages 243 - 246 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112816983A (en) * | 2021-01-06 | 2021-05-18 | 中南大学 | Time sequence InSAR turbulence atmospheric delay correction method based on optimized interferogram set |
| CN112816983B (en) * | 2021-01-06 | 2023-09-19 | 中南大学 | Time-series InSAR turbulent atmospheric delay correction method based on optimized interferometric atlas |
| CN119936921A (en) * | 2025-04-09 | 2025-05-06 | 内蒙古电力(集团)有限责任公司内蒙古电力经济技术研究院分公司 | A transmission line tower monitoring method and system based on Beidou positioning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116068511B (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113866764B (en) | Landslide susceptibility improved assessment method based on InSAR and LR-IOE models | |
| CN106772342B (en) | Time sequence differential radar interference method suitable for large-gradient ground surface settlement monitoring | |
| CN103454636B (en) | Differential interferometric phase estimation method based on multi-pixel covariance matrixes | |
| CN109558859B (en) | Method and system for mining distribution information extraction based on DInSAR and DCNN | |
| CN116068511B (en) | A large-scale system error correction method for InSAR based on deep learning | |
| CN116305902B (en) | Flood maximum submerged depth space simulation method based on multi-mode remote sensing | |
| CN117724098A (en) | Time sequence InSAR troposphere delay correction method for relieving atmospheric seasonal oscillation | |
| CN113378945A (en) | Method for reconstructing high-spatial-temporal-resolution ground settlement information based on machine learning | |
| CN116148855B (en) | Method and System for Time-series InSAR Atmospheric Phase Removal and Deformation Calculation | |
| CN111458709A (en) | A method and device for monitoring a two-dimensional deformation field of a wide-area surface of a spaceborne radar | |
| CN115980751A (en) | A Tropospheric Delay Correction Method for Power Law Model InSAR | |
| CN115079172A (en) | A kind of MTInSAR landslide monitoring method, equipment and storage medium | |
| CN114812491A (en) | Power transmission line earth surface deformation early warning method and device based on long-time sequence analysis | |
| CN110109105A (en) | A method of the InSAR technical monitoring Ground Deformation based on timing | |
| CN119027640B (en) | An InSAR atmospheric delay correction method and system based on adaptive window | |
| CN114239379A (en) | A method and system for analyzing geological hazards of transmission lines based on deformation detection | |
| CN106842199A (en) | It is a kind of to merge the method that different resolution SAR data monitors Ground Deformation | |
| CN117970332B (en) | InSAR urban surface deformation risk fine recognition method combining optical images | |
| CN113138388B (en) | Ground subsidence monitoring method integrating precise level and InSAR reliable subsidence value | |
| CN107144213A (en) | The big magnitude three-D sequential deformation method of estimation in mining area and device based on SAR intensity images | |
| CN108132468A (en) | A kind of more baseline polarimetric SAR interferometry depth of building extracting methods | |
| Lauknes et al. | A comparison of SBAS and PS ERS InSAR for subsidence monitoring in Oslo, Norway | |
| CN114280608B (en) | Method and system for removing DInSAR elevation-related atmospheric effect | |
| CN114046774A (en) | Ground deformation continuous monitoring method integrating CORS network and multi-source data | |
| CN119358818A (en) | Landslide susceptibility assessment method, device, equipment and medium based on InSAR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |