WO2021051593A1 - Image processing method and apparatus, computer device, and storage medium - Google Patents

Abstract

Disclosed are an image processing method and apparatus, a computer device, and a storage medium. The method comprises: randomly obtaining a preset number of multiple training images from a preset training image set; performing, according to a preset spatial transformation network, spatial transformation on the multiple training images to obtain corresponding registration images; performing, according to a preset convolutional noise reduction model, superimposition and noise reduction on the registration images to obtain corresponding noise reduction images; performing iterative training on the convolutional noise reduction model according to a preset gradient descent training model, the registration images, the noise reduction images, and the training image set to obtain a trained convolutional noise reduction model; and processing, according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superimposition rule, an image to be processed input by a user to obtain a corresponding optimized image.

Description

Image processing method, device, computer equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 19, 2019, the application number is 201910888024.8, and the application name is "Image processing methods, devices, computer equipment and storage media", the entire contents of which are incorporated by reference In this application.

Technical field

This application relates to the field of computer technology, and in particular to an image processing method, device, computer equipment, and storage medium.

Background technique

In the process of lesion judgment based on OCT images, due to the coherent imaging mode used, speckle noise is inevitably generated in the collected single image, which seriously affects the subsequent processing of OCT images and lesion judgment. In order to solve this problem, the commonly used image enhancement methods are to collect about 50 OCT images through OCT scanning equipment, and superimpose and reduce noise based on the collected images to obtain a clear fused image. However, this processing process needs to be in the same The area is repeatedly scanned about 50 times, which greatly increases the time required to acquire OCT images, and the larger number of images greatly increases the processing time, resulting in a long time-consuming process for obtaining clear images. Therefore, the existing method for image enhancement of OCT images has the problem of long processing time.

Summary of the invention

The embodiments of the present application provide an image processing method, device, computer equipment, and storage medium, aiming to solve the problem of long processing time when performing image enhancement on an OCT image in the image processing method in the prior art method.

In the first aspect, an embodiment of the present application provides an image processing method, which includes: randomly acquiring a preset number of training images from a preset training image set; The image is spatially transformed to obtain the corresponding registered image; the registered image is superimposed and denoised according to the preset convolutional denoising model to obtain the corresponding denoised image; according to the preset gradient descent training model, the The registration image, the denoising image, and the training image set are iteratively trained on the convolution denoising model to obtain the trained convolution denoising model; if the image to be processed input by the user is received, The image to be processed is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image.

In the second aspect, an embodiment of the present application provides an image processing device, which includes: a training image acquisition unit, which randomly acquires a preset number of multiple training images from a preset training image set; The preset spatial transformation network performs spatial transformation on a plurality of the training images to obtain corresponding registered images; a superimposed noise reduction processing unit for superimposing and denoising the registered images according to a preset convolutional noise reduction model To obtain the corresponding denoising image; a model training unit for iterating the convolution denoising model according to a preset gradient descent training model, the registration image, the denoising image, and the training image set Training to obtain the trained convolutional noise reduction model; an optimized image acquisition unit for receiving the image to be processed input by the user, according to the spatial transformation network, the trained convolutional noise reduction model, and The preset superposition rule processes the to-be-processed image to obtain a corresponding optimized image.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer The program implements the image processing method described in the first aspect.

In a fourth aspect, the embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the above-mentioned first One aspect of the image processing method.

The embodiments of the present application provide an image processing method, device, computer equipment, and storage medium. Randomly acquire a variety of training images, perform spatial transformation on the training image according to the spatial transformation network to obtain a registered image, superimpose the registered image according to the convolutional noise reduction model to obtain a noise-reduced image, based on the noise-reduced image, the registered image and The training image set performs iterative training on the convolution noise reduction model to obtain the trained convolution noise reduction model. The image to be processed is processed into an optimized image according to the spatial transformation network, the trained convolution noise reduction model and the superposition rule. Through the above method, the number of images that need to be processed can be greatly reduced, the time for image processing can be shortened, and the processing efficiency when image enhancement is performed on OCT images is improved.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the application;

FIG. 2 is a schematic diagram of a sub-flow of an image processing method provided by an embodiment of the application;

FIG. 3 is a schematic diagram of another sub-flow of the image processing method provided by an embodiment of the application;

FIG. 4 is a schematic diagram of another sub-flow of the image processing method provided by an embodiment of the application;

FIG. 5 is a schematic diagram of another sub-flow of the image processing method provided by an embodiment of the application;

FIG. 6 is a schematic block diagram of an image processing device provided by an embodiment of the application;

FIG. 7 is a schematic block diagram of a computer device provided by an embodiment of the application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be understood that when used in this specification and appended claims, the terms "including" and "including" indicate the existence of the described features, wholes, steps, operations, elements and/or components, but do not exclude one or The existence or addition of multiple other features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should be further understood that the term "and/or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the present application. The image processing method is applied to a user terminal, and the method is executed by application software installed in the user terminal. The user terminal is a terminal device used to execute image processing methods to complete image optimization processing, such as desktop computers and notebooks. Computer, tablet or mobile phone, etc.

As shown in Fig. 1, the method includes steps S110 to S150.

S110 randomly obtains a preset number of multiple training images from a preset training image set.

Randomly acquire a preset number of training images from the preset training image set. OCT scanning equipment is a relatively common ophthalmic disease image acquisition equipment. The training image set is the image set obtained by repeatedly scanning the suspected lesion area through the OCT scanning equipment. The training image set can contain 35-60 training images. And a target image obtained by image enhancement method based on all training images in the training image set, all training images have the same size, and the obtained target image is the target for training the convolutional noise reduction model. The preset number is the number of training images obtained randomly from the training image set.

Traditional image enhancement methods need to superimpose about 50 OCT images and reduce noise to obtain a clear fused image. The amount of calculation is very huge. With this solution, less OCT images can be used to superimpose and reduce noise to obtain the fused image. Images with the same quality greatly reduce the amount of calculation in the image processing process and shorten the processing time. In order to use less OCT image overlay noise reduction to obtain images with the same quality as the target image, it is necessary to reduce the number of training images during the training process of the convolutional noise reduction model. A preset number of images can be randomly obtained from the training image set. Two training images, that is, only part of the training images in the training image set are used to train the convolutional noise reduction model. The preset number can be preset by the user. Specifically, the preset number can be set to 5-10.

S120: Perform spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images.

The multiple training images are spatially transformed according to a preset spatial transformation network to obtain corresponding registered images, wherein the spatial transformation network includes a convolutional neural network and a two-dimensional affine transformation function. The Spatial Transformer Network (STN) is an image processing neural network that spatially transforms multiple training images. As the patient is not completely still during the repeated scanning of the suspected lesion area, the collected multiple images There are displacement phenomena such as rotation and translation between OCT images, and the images obtained by directly superimposing multiple training images have distortion problems. Therefore, in order to avoid the distortion of the images obtained by superimposing, the training images can be spaced before being superimposed. Transformation processing.

In an embodiment, as shown in FIG. 2, step S120 includes sub-steps S121, S122, S123, and S124.

S121. Determine any one of the multiple training images as a reference image, and determine other training images as images to be converted.

Determine any one of the multiple training images as a reference image, and determine other training images as images to be converted. Before performing spatial transformation on multiple training images, it is necessary to determine that any one of the training images is a reference image, and other training images are spatially transformed according to the reference image, and then the other training images are determined as the images to be converted.

S122. Using the reference image as a reference, obtain, according to the convolutional neural network, all parameter matrices of the image to be converted corresponding to the reference image.

Taking the reference image as a reference, and acquiring, according to the convolutional neural network, all the parameter matrices of the image to be converted corresponding to the reference image. Specifically, after performing convolution processing on the image to be converted and the reference image through the convolutional neural network, the reference image is used as a reference, and the parameter matrix corresponding to each image to be converted is obtained through regression of the fully connected layer in the convolutional neural network.

For example, the resolution of the image (or reference image) to be converted is 600×600. According to the calculation formula in the first convolution kernel in the convolutional neural network, the resolution is 16*16 as the window, and the step size is 1, and the volume is The product operation obtains a vector matrix with a size of 585×585, which is the shallow feature of the image to be converted; according to the pooling calculation formula, with a resolution of 13×13 as the window and a step size of 13, downsampling is performed to obtain the size It is a 45×45 vector matrix, which is the in-depth feature of the image to be converted; according to the calculation formula in the 5 second convolution kernels, the resolution is 5×5 as the window, and the step size is 5 for convolution operation , To get 5 vector matrices with a size of 9×9. Input the five 9×9 vector matrices of the image to be converted and the five 9×9 vector matrices of the reference image into the fully-connected calculation formula in the fully-connected layer for calculation. Since the image input to the convolutional neural network contains any For an image to be converted and a reference image, the full connection calculation formula includes 2×5×9×9 input nodes and 6 output nodes. The full connection calculation formula is used to reflect the difference between the input node and the output node. Association relationship, the output result of 6 output nodes is the parameter matrix.

The parameter matrix A _θ can be expressed by the following formula:

其中，参数矩阵A _θ中包含六个参数，其中四个为旋转参数，另外两个为平移参数。

Among them, the parameter matrix A _θ contains six parameters, four of which are rotation parameters, and the other two are translation parameters.

S123: Map the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image.

The image to be converted is mapped according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image. Specifically, according to the two-dimensional affine transformation function and the parameter matrix, the affine transformation calculation is performed on the coordinate values of the pixels included in the image to be transformed to obtain the affine transformation coordinate value, and the image to be transformed is mapped and filled based on the affine transformation coordinate value to obtain the corresponding Map image.

Specifically, the specific process of affine transformation calculation can be expressed as:

其中，T _θ即为二维仿射变换函数，对待转换图像中对应像素进行仿射变换计算所得的坐标值为(x ^s _i，y ^s _i)， 待转换图像中像素的坐标值为(x ^t _i，y ^t _i)。

Among them, T _{θ is} the two-dimensional affine transformation function, the coordinate value calculated by performing affine transformation on the corresponding pixel in the image to be converted is (x ^s _i , y ^s _i ), and the coordinate value of the pixel in the image to be converted is (x ^t _i , y ^t _i ).

其中，U _nm即为待转换图像中第n行第m列坐标对应的像素值，待转换图像的分辨率为(H×W)，(x ^s _i，y ^s _i)即为映射图像与待转换图像对应的像素在待转换图像中的坐标值，V _i即为映射图像中第i个像素点进行填充得到的像素值。 The specific process of mapping and filling can be expressed as:

Among them, U _{nm is} the pixel value corresponding to the coordinates of the nth row and mth column in the image to be converted, the resolution of the image to be converted is (H×W), (x ^s _i , y ^s _i ) is the mapped image and the conversion image corresponding to the pixel to be converted in the image coordinate values, V _i is the i-th mapping image pixels to fill pixel values obtained.

S124: Obtain the mapped images and reference images corresponding to all the images to be converted to obtain registered images.

Obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registered images. Based on the above process, all the images to be converted are spatially transformed based on the registered image, and multiple mapped images with the same angle and orientation of the suspected lesion area in the reference image are obtained. The resolution of the mapped image is consistent with the reference image, and the resulting mapped image The reference image is used as the registration image.

For example, randomly obtain the corresponding 5 training images, take 1 of them as the reference image, and 4 as the image to be converted, respectively perform 4 spatial transformations on the 4 images to be converted to obtain 4 mapped images, and map the 4 images The image and a reference image are used as the corresponding registration image.

S130: Perform superposition noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image.

Perform superposition and noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image. The convolutional denoising model is a model used to denoise the image. The convolutional denoising model includes an activation function and multiple convolution kernels. Each convolution kernel contains multiple parameters, and each parameter corresponds to For a parameter value, performing a convolution operation on an image is to perform a convolution operation on a two-dimensional array corresponding to the image through the parameter values included in the convolution kernel.

In an embodiment, as shown in FIG. 3, step S130 includes sub-steps S131 and S132.

S131. Perform pixel-by-pixel superposition on all the registered images to obtain a first superimposed image.

All the registered images are superimposed pixel by pixel to obtain a first superimposed image. The obtained registration images are multiple, and the angle and orientation of the suspected lesion area in all the registration images are the same. The multiple registration images can be superimposed pixel by pixel to obtain the first superimposed image, and the obtained first superimposed image The resolution is the same as the registered image. Specifically, the pixel values of all registered images in the same pixel are added and averaged to complete the superimposition of the pixel. Based on the above method, the average value of each pixel in the multiple registered images is obtained. Then the corresponding first superimposed image can be obtained.

S132: Perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image.

Perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image. Specifically, the array value corresponding to each pixel in the first superimposed image is first calculated by the activation function to obtain the two-dimensional array corresponding to the first superimposed image, based on the array value in the two-dimensional array and the parameters in each convolution kernel The value performs convolution operation on the two-dimensional array to obtain the corresponding two-dimensional convolution array, and the corresponding denoising image can be obtained by inversely activating the two-dimensional convolution array through the activation function.

Any one of the activation functions can be selected. For example, if the Sigmoid function is selected as the activation function, the expression of the activation function is: f(x)=(1+e ^-x/51 ) ^-1 ; one of the first superimposed images The pixel value of the pixel is 238 (the pixel value is an integer between [0,255]), the corresponding array value of the pixel is calculated to be 0.9907, and the array value corresponding to each pixel in the first superimposed image can be obtained to obtain a two-dimensional array .

S140. Perform iterative training on the convolutional noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained convolution noise reduction model.

Performing iterative training on the convolution noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained convolution noise reduction model. In order to make the convolution noise reduction model have a good use effect when performing convolution noise reduction processing on the image, the convolution noise reduction model needs to be iteratively trained, that is, the parameter value of the convolution kernel in the convolution noise reduction model After adjustment, the convolutional noise reduction model obtained after training can accurately reduce the noise of the image to obtain a clearer image. The gradient descent training model is a model for training the convolutional noise reduction model. The gradient descent training model contains a loss function and a gradient calculation formula. The loss function can be used to calculate the loss value between two images. The smaller the loss value, the smaller the loss value. The closer the contents of the two images are, the update value corresponding to each parameter can be calculated based on the calculated loss value and gradient calculation formula, and the parameter value corresponding to each parameter can be updated through the update value, that is, Train the convolutional noise reduction model.

In an embodiment, as shown in FIG. 4, step S140 includes sub-steps S141, S142, S143, and S144.

S141. Perform pixel-by-pixel superposition on the noise reduction image and the pixel averages of all the registered images according to the superposition rule to obtain a high-order superimposed image.

The pixel mean values of the noise-reduced image and all the registered images are superimposed pixel by pixel according to the superimposition rule to obtain a high-order superimposed image. Specifically, the pixel values of the registered images in the same pixel are obtained, and the pixel values of all registered images are averaged to obtain the pixel averages of all registered images. The denoising image and the pixel averages of all registered images are superimposed pixel by pixel to obtain a high-order superimposed image .

S142. Calculate the loss value between the high-order superimposed image and the target image in the training image set according to the loss function in the gradient descent training model.

其中，μ _x为图像x的像素平均值，μ _y为图像y的像素平均值，σ _xy为图像x与图像y之间的协方差，σ _x为图像x的标准差，σ _y为图像y的标准差，c ₁及c ₂均为公式中预设的参数值。 Calculate the loss value between the high-order superimposed image and the target image in the training image set according to the loss function in the gradient descent training model. Based on the loss function, the loss value between the high-order superimposed image and the target image can be calculated. In order to make the image obtained after image enhancement processing approach the target image, the loss value between the high-order superimposed image and the target image can be calculated based on the loss value. The difference is quantified. Specifically, the loss function can be expressed as: loss value S=w ₁ ×J ₁ +w ₂ ×J ₂ +w ₃ ×J ₃ +w ₄ ×J ₄ ;wherein, w ₁ , w ₂ , w ₃ and w ₄ Both are the weight values preset in the formula, J ₁ is the overall structural similarity between the _{high-order superimposed image and the target image, J 2} is the structural similarity between the high-order superimposed image and the target image in the retinal area, and J ₃ is the high-order superimposed image. The structural similarity between the superimposed image and the target image in the key areas (focus area, pigment epithelial area, nerve fiber layer, etc.), J ₄ is the gradient difference of pixels between the high-order superimposed image and the target image. Among them, the structural similarity between the calculated image x and the image y can be expressed as:

Among them, μ _x is the average pixel value of image x, μ _y is the average pixel value of image y, σ _xy is the covariance between image x and image y, σ _x is the standard deviation of image x, and σ _y is image y The standard deviation of c ₁ and c ₂ are the parameter values preset in the formula.

In addition, after the loss value is calculated, it can also be judged whether the loss value is less than the preset threshold. If the loss value is less than the preset threshold, it indicates that the current convolution noise reduction model has met the usage requirements, and the subsequent convolution reduction can be terminated. Training of the noise model; if the loss value is not less than the preset threshold, it indicates that the current convolution noise reduction model does not meet the requirements of use, and the convolution noise reduction model needs to be trained through the subsequent processing process.

S143. Calculate the updated value of each parameter in the convolution noise reduction model according to the gradient calculation formula in the gradient descent training model, the loss value, and the calculated value in the convolution noise reduction model.

The updated value of each parameter in the convolution noise reduction model is calculated according to the gradient calculation formula in the gradient descent training model, the loss value, and the calculated value corresponding to each parameter in the convolution noise reduction model. Specifically, the calculated value obtained by calculating a parameter in the convolutional noise reduction model on the array value in the two-dimensional array corresponding to the first superimposed image, inputting the gradient calculation formula and combining the above loss value, can be calculated and obtained The update value corresponding to the parameter, this calculation process is also the gradient descent calculation.

其中，

为计算得到的参数x的更新值，ω _x为参数x的原始值，η为梯度计算公式中预置的学习率，

为基于损失值及参数x对应的计算值对该参数的偏导值(这一计算过程中需使用该参数对应的计算值)。 Specifically, the gradient calculation formula can be expressed as:

among them,

Is the updated value of the calculated parameter x, ω _x is the original value of the parameter x, and η is the preset learning rate in the gradient calculation formula,

It is the partial derivative of the parameter based on the loss value and the calculated value corresponding to the parameter x (the calculated value corresponding to the parameter needs to be used in this calculation process).

S144. Update the parameter value of the corresponding parameter in the convolution noise reduction model according to the update value of each parameter, so as to train the convolution noise reduction model.

The parameter value of the corresponding parameter in the convolution noise reduction model is updated according to the update value of each parameter, so as to train the convolution noise reduction model. Based on the calculated update value, the parameter value of each parameter in the convolution noise reduction model is correspondingly updated, that is, a training process of the convolution noise reduction model is completed. Based on the convolution noise reduction model obtained after one training, perform the convolution operation again on the first superimposed image, and repeat the above training process, then the convolution noise reduction model can be iteratively trained; when the calculated loss value is less than the preset Threshold, that is, terminate the training process to obtain the trained convolutional noise reduction model.

S150. If the to-be-processed image input by the user is received, process the to-be-processed image according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image .

If the to-be-processed image input by the user is received, the to-be-processed image is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image. The convolutional noise reduction model obtained after training combined with the spatial transformation network and superposition rules can process the image to be processed input by the user to obtain a clear optimized image. The number of images to be processed input by the user is much smaller than the training image The number of training images is concentrated, so it is possible to use fewer OCT images for processing to obtain an image with the same quality as the target image, which greatly reduces the amount of calculation in the image processing process and shortens the image processing time.

In an embodiment, as shown in FIG. 5, step S150 includes sub-steps S151, S152, S153, and S154.

S151. Perform spatial transformation on the to-be-processed image according to the spatial transformation network to obtain a corresponding target registration image.

Performing spatial transformation on the to-be-processed image according to the spatial transformation network to obtain a corresponding target registration image. The specific process of performing spatial transformation on the image to be processed is the same as the above steps, and will not be repeated here.

S152: Perform pixel-by-pixel superposition on all the target registration images to obtain a second superimposed image.

All the target registration images are superimposed pixel by pixel to obtain a second superimposed image. The specific process of pixel-by-pixel superimposition on the obtained target registration image is the same as the above steps, and will not be repeated here.

S153. Perform convolution noise reduction on the second superimposed image according to the convolution noise reduction model to obtain a target noise reduction image.

Perform convolution noise reduction on the second superimposed image according to the convolution noise reduction model to obtain a target noise reduction image. The specific process of performing convolution noise reduction on the obtained second superimposed image through the convolution noise reduction model is the same as the above steps, and will not be repeated here.

S154. Perform pixel-by-pixel superposition on the target noise reduction image and the pixel averages of all the target registration images according to the superimposition rule to obtain an optimized image.

The pixel mean values of the target noise reduction image and all the target registration images are superimposed pixel by pixel according to the superposition rule to obtain an optimized image. The specific process of pixel-by-pixel superposition of the target noise reduction image and the pixel averages of all target registration images according to the superposition rule is the same as the above steps, and will not be repeated here.

In the image processing method provided by the embodiments of the present application, a variety of training images are randomly obtained, the training images are spatially transformed according to the spatial transformation network to obtain the registered images, and the registered images are superimposed and denoised according to the convolutional noise reduction model. Denoising image, iteratively train the convolution denoising model based on denoising image, registration image and training image set to obtain the trained convolution denoising model, according to the spatial transformation network, the trained convolution denoising model and superposition The rules process the to-be-processed image into an optimized image. Through the above method, the number of images that need to be processed can be greatly reduced, the time for image processing can be shortened, and the processing efficiency when image enhancement is performed on OCT images is improved.

An embodiment of the present application also provides an image processing device, which is used to execute any embodiment of the foregoing image processing method. Specifically, please refer to FIG. 6, which is a schematic block diagram of an image processing apparatus provided by an embodiment of the present application. The image processing device can be configured in a user terminal.

As shown in FIG. 6, the image processing apparatus 100 includes a training image acquisition unit 110, a registration image acquisition unit 120, a superposition noise reduction processing unit 130, a model training unit 140 and an optimized image acquisition unit 150.

The training image acquisition unit 110 is configured to randomly acquire a preset number of training images from a preset training image set.

The registration image acquisition unit 120 is configured to perform spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images.

In other application embodiments, the registration image acquisition unit 120 includes sub-units: a training image allocation unit, a parameter matrix acquisition unit, a mapped image acquisition unit, and a registration image determination unit.

The training image distribution unit is used to determine any one of the multiple training images as a reference image, and other training images are determined as the images to be converted; the parameter matrix acquisition unit is used to use the reference image as a reference, according to The convolutional neural network obtains all the parameter matrices of the image to be converted corresponding to the reference image; the mapping image obtaining unit is configured to perform the calculation of the image to be converted according to the two-dimensional affine transformation function and the parameter matrix. The mapping is performed to obtain the corresponding mapped image; the registration image determining unit is used to obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registered images.

The superposition noise reduction processing unit 130 is configured to perform superposition noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image.

In other application embodiments, the superimposition noise reduction processing unit 130 includes sub-units: a first superimposition image acquisition unit and a convolution noise reduction processing unit.

The first superimposed image acquisition unit is used to superimpose all the registered images pixel by pixel to obtain the first superimposed image; the convolution noise reduction processing unit is used to superimpose the first superimposed image according to the convolution noise reduction model The image undergoes convolution noise reduction to obtain a noise-reduced image.

The model training unit 140 is configured to iteratively train the convolutional noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained all Describe the convolution noise reduction model.

In other application embodiments, the model training unit 140 includes sub-units: a high-order superimposed image acquisition unit, a loss value calculation unit, an update value calculation unit, and a parameter update unit.

The high-order superimposed image acquisition unit is used to superimpose the pixel mean values of the noise-reduction image and all the registered images pixel by pixel according to the superimposition rule to obtain a high-order superimposed image; a loss value calculation unit is used to The loss function in the gradient descent training model calculates the loss value between the high-order superimposed image and the target image in the training image set; the update value calculation unit is used to calculate the gradient according to the gradient calculation formula in the gradient descent training model, The loss value and the calculated value in the convolutional noise reduction model are calculated to obtain the updated value of each parameter in the convolutional noise reduction model; The parameter values of the corresponding parameters in the convolution noise reduction model are updated to train the convolution noise reduction model.

The optimized image acquisition unit 150 is configured to process the to-be-processed image according to the spatial transformation network, the trained convolutional noise reduction model, and preset superimposition rules if the to-be-processed image input by the user is received To get the corresponding optimized image.

In other application embodiments, the optimized image acquisition unit 150 includes sub-units: a target registration image acquisition unit, a second superimposed image acquisition unit, a target noise reduction image acquisition unit, and an image superimposition unit.

The target registration image acquisition unit is configured to perform spatial transformation on the to-be-processed image according to the spatial transformation network to obtain a corresponding target registration image; the second superimposed image acquisition unit is configured to combine all the target registration images Perform pixel-by-pixel superposition to obtain a second superimposed image; a target noise reduction image acquisition unit, configured to perform convolution noise reduction on the second superimposed image according to the convolution noise reduction model to obtain a target noise reduction image; image superposition unit , For performing pixel-by-pixel superposition on the target noise reduction image and the pixel mean values of all the target registration images according to the superposition rule to obtain an optimized image.

The image processing device provided in the embodiment of the application is used to perform the above-mentioned image processing method, randomly obtain a variety of training images, perform spatial transformation on the training images according to the spatial transformation network to obtain a registered image, and register according to the convolutional noise reduction model The image is superimposed and denoised to obtain the denoised image, and the convolution denoising model is iteratively trained based on the denoised image, the registration image and the training image set to obtain the trained convolution denoising model. According to the spatial transformation network, the trained convolution The product noise reduction model and the superposition rule process the to-be-processed image into an optimized image. Through the above method, the number of images that need to be processed can be greatly reduced, the time for image processing can be shortened, and the processing efficiency when image enhancement is performed on OCT images is improved.

The above-mentioned image processing apparatus may be implemented in the form of a computer program, and the computer program may be run on a computer device as shown in FIG. 7.

Please refer to FIG. 7, which is a schematic block diagram of a computer device according to an embodiment of the present application.

Referring to FIG. 7, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504. The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the image processing method. The processor 502 is used to provide computing and control capabilities, and support the operation of the entire computer device 500. The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the image processing method. The network interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

Wherein, the processor 502 is configured to run a computer program 5032 stored in a memory to implement the image processing method of this embodiment.

Those skilled in the art can understand that the embodiment of the computer device shown in FIG. 7 does not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or less components than those shown in the figure. Or some parts are combined, or different parts are arranged. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are consistent with the embodiment shown in FIG. 7 and will not be repeated here.

It should be understood that in this embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, where the computer program is executed by a processor to implement the image processing method of the embodiment of the present application.

The storage medium may be an internal storage unit of the aforementioned device, such as a hard disk or memory of the device. The storage medium may also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a Smart Media Card (SMC), a Secure Digital (SD) card, and a flash memory card. (Flash Card) and so on. Further, the storage medium may also include both an internal storage unit of the device and an external storage device.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described equipment, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (20)

An image processing method applied to a user terminal, the method including: Randomly obtain a preset number of training images from a preset training image set; Performing spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images; Performing superposition noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image; Performing iterative training on the convolution noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained convolution noise reduction model; If the to-be-processed image input by the user is received, the to-be-processed image is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image. The image processing method according to claim 1, wherein the spatial transformation network includes a convolutional neural network and a two-dimensional affine transformation function, and the plurality of training images are spatially transformed according to a preset spatial transformation network To obtain the corresponding registration image, including: Determining any one of the plurality of training images as a reference image, and determining other training images as images to be converted; Using the reference image as a reference, acquiring, according to the convolutional neural network, all the parameter matrices of the image to be converted corresponding to the reference image; Mapping the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image; Obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registered images. The image processing method according to claim 1, wherein the superimposing noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image comprises: Superimpose all the registered images pixel by pixel to obtain a first superimposed image; Perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image. The image processing method according to claim 1, wherein the convolutional noise reduction model is performed on the convolutional noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set. Iterative training to obtain the trained convolutional noise reduction model includes: Performing pixel-by-pixel superposition on the noise reduction image and the pixel mean values of all the registered images according to the superimposition rule to obtain a high-order superimposed image; Calculating a loss value between the high-order superimposed image and the target image in the training image set according to the loss function in the gradient descent training model; Calculating the updated value of each parameter in the convolution noise reduction model according to the gradient calculation formula in the gradient descent training model, the loss value, and the calculation value in the convolution noise reduction model; The parameter value of the corresponding parameter in the convolution noise reduction model is updated according to the update value of each parameter, so as to train the convolution noise reduction model. The image processing method according to claim 1, wherein the image to be processed is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding Optimize the image, including: Performing spatial transformation on the to-be-processed image according to the spatial transformation network to obtain a corresponding target registration image; Superimpose all the target registration images pixel by pixel to obtain a second superimposed image; Performing convolution noise reduction on the second superimposed image according to the convolution noise reduction model to obtain a target noise reduction image; The pixel mean values of the target noise reduction image and all the target registration images are superimposed pixel by pixel according to the superposition rule to obtain an optimized image. 3. The image processing method according to claim 2, wherein the mapping the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain the corresponding mapped image comprises: Performing affine transformation calculation on the coordinate values of the pixels included in the image to be transformed according to the two-dimensional affine transformation function and the parameter matrix to obtain affine transformation coordinate values; Perform mapping and filling on the to-be-converted image according to the affine transformation coordinate value to obtain a corresponding mapping image. The image processing method according to claim 3, wherein the convolution noise reduction model comprises an activation function and a plurality of convolution kernels, and the first superimposed image is convolved according to the convolution noise reduction model Product noise reduction to obtain a noise reduction image, including: Calculating an array value corresponding to each pixel in the first superimposed image according to the activation function to obtain a two-dimensional array corresponding to the first superimposed image; Performing a convolution operation on the two-dimensional array according to the array values in the two-dimensional array and the parameter values in each of the convolution kernels to obtain a corresponding two-dimensional convolution array; According to the activation function, the two-dimensional convolution array is inversely activated to obtain a corresponding noise-reduced image. An image processing device, including: The training image acquisition unit randomly acquires a preset number of training images from a preset training image set; A registration image acquisition unit, performing spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images; A superposition noise reduction processing unit, configured to superpose and reduce noise on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image; The model training unit is used to iteratively train the convolutional noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained Convolutional noise reduction model; The optimized image acquisition unit is configured to, if the image to be processed input by the user is received, process the image to be processed according to the spatial transformation network, the trained convolutional noise reduction model, and preset superposition rules. Get the corresponding optimized image. 8. The image processing device according to claim 8, wherein the registered image acquisition unit comprises: A training image distribution unit, configured to determine any one of the multiple training images as a reference image, and determine other training images as images to be converted; A parameter matrix obtaining unit, configured to use the reference image as a reference and obtain, according to the convolutional neural network, all the parameter matrices of the image to be converted corresponding to the reference image; A mapping image acquisition unit, configured to map the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image; The registration image determination unit is used to obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registration images. 8. The image processing device according to claim 8, wherein the superposition noise reduction processing unit comprises: A first superimposed image acquisition unit, configured to superimpose all the registered images pixel by pixel to obtain a first superimposed image; The convolution noise reduction processing unit is configured to perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image. A computer device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer program: Randomly obtain a preset number of training images from a preset training image set; Performing spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images; Performing superposition noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image; Performing iterative training on the convolution noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained convolution noise reduction model; If the to-be-processed image input by the user is received, the to-be-processed image is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image. The computer device according to claim 11, wherein the spatial transformation network includes a convolutional neural network and a two-dimensional affine transformation function, and the plurality of training images are spatially transformed according to a preset spatial transformation network to Obtain the corresponding registration image, including: Determining any one of the plurality of training images as a reference image, and determining other training images as images to be converted; Using the reference image as a reference, acquiring, according to the convolutional neural network, all the parameter matrices of the image to be converted corresponding to the reference image; Mapping the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image; Obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registered images. 11. The computer device according to claim 11, wherein the superimposing noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image comprises: Superimpose all the registered images pixel by pixel to obtain a first superimposed image; Perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image. The computer device according to claim 11, wherein the convolutional noise reduction model is iterated according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set The training to obtain the trained convolutional noise reduction model includes: Performing pixel-by-pixel superposition on the noise reduction image and the pixel mean values of all the registered images according to the superimposition rule to obtain a high-order superimposed image; Calculating a loss value between the high-order superimposed image and the target image in the training image set according to the loss function in the gradient descent training model; Calculating the updated value of each parameter in the convolution noise reduction model according to the gradient calculation formula in the gradient descent training model, the loss value, and the calculation value in the convolution noise reduction model; The parameter value of the corresponding parameter in the convolution noise reduction model is updated according to the update value of each parameter, so as to train the convolution noise reduction model. The computer device according to claim 11, wherein the image to be processed is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimization Images, including: Performing spatial transformation on the to-be-processed image according to the spatial transformation network to obtain a corresponding target registration image; Superimpose all the target registration images pixel by pixel to obtain a second superimposed image; Performing convolution noise reduction on the second superimposed image according to the convolution noise reduction model to obtain a target noise reduction image; The pixel mean values of the target noise reduction image and all the target registration images are superimposed pixel by pixel according to the superposition rule to obtain an optimized image. The computer device according to claim 12, wherein the mapping the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image comprises: Performing affine transformation calculation on the coordinate values of the pixels included in the image to be transformed according to the two-dimensional affine transformation function and the parameter matrix to obtain affine transformation coordinate values; Perform mapping and filling on the to-be-converted image according to the affine transformation coordinate value to obtain a corresponding mapping image. The computer device according to claim 13, wherein the convolution noise reduction model comprises an activation function and a plurality of convolution kernels, and the first superimposed image is convolved according to the convolution noise reduction model Noise reduction to obtain a noise-reduced image, including: Calculating an array value corresponding to each pixel in the first superimposed image according to the activation function to obtain a two-dimensional array corresponding to the first superimposed image; Performing a convolution operation on the two-dimensional array according to the array values in the two-dimensional array and the parameter values in each of the convolution kernels to obtain a corresponding two-dimensional convolution array; According to the activation function, the two-dimensional convolution array is inversely activated to obtain a corresponding noise-reduced image. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the following operations: Randomly obtain a preset number of training images from a preset training image set; Performing spatial transformation on a plurality of the training images according to a preset spatial transformation network to obtain corresponding registration images; Performing superposition noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image; Performing iterative training on the convolution noise reduction model according to a preset gradient descent training model, the registration image, the noise reduction image, and the training image set to obtain the trained convolution noise reduction model; If the to-be-processed image input by the user is received, the to-be-processed image is processed according to the spatial transformation network, the trained convolutional noise reduction model, and a preset superposition rule to obtain a corresponding optimized image. The computer-readable storage medium according to claim 18, wherein the spatial transformation network includes a convolutional neural network and a two-dimensional affine transformation function, and the plurality of training images are performed on a plurality of the training images according to a preset spatial transformation network. Spatial transformation to obtain the corresponding registered image, including: Determining any one of the plurality of training images as a reference image, and determining other training images as images to be converted; Using the reference image as a reference, acquiring, according to the convolutional neural network, all the parameter matrices of the image to be converted corresponding to the reference image; Mapping the image to be converted according to the two-dimensional affine transformation function and the parameter matrix to obtain a corresponding mapped image; Obtain the mapped images and reference images corresponding to all the images to be converted to obtain the registered images. 18. The computer-readable storage medium according to claim 18, wherein the superimposing noise reduction on the registered image according to a preset convolution noise reduction model to obtain a corresponding noise reduction image comprises: Superimpose all the registered images pixel by pixel to obtain a first superimposed image; Perform convolution noise reduction on the first superimposed image according to the convolution noise reduction model to obtain a noise reduction image.

Images