CN116665112A - Tunnel inspection method and device, electronic equipment and storage medium - Google Patents

Abstract

An embodiment of the present invention provides a tunnel inspection method, which obtains the optical flow component and brightness component corresponding to the captured image in the target tunnel; combines the optical flow component and the brightness component with the color component of the captured image to obtain A target image; processing the target image through a trained disease recognition model to obtain a disease recognition result of the target tunnel; determining a patrol inspection result of the target tunnel based on the disease recognition result of the target tunnel. By combining the optical flow component and brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and uses the dynamic information of the optical flow component and the static information of the brightness component To assist the disease identification model to identify road diseases, so as to improve the accuracy of disease identification results, and then improve the accuracy of tunnel inspection results.

Description

A tunnel inspection method, device, electronic equipment and storage medium

technical field

The invention relates to the technical field of intelligent transportation, in particular to a tunnel inspection method, device, electronic equipment and storage medium.

Background technique

With the implementation of artificial intelligence in the transportation field, visual technology can already be used to replace manual road disease inspections. The road disease inspection vehicle is a vehicle equipped with an image sensor and an image processor. The image sensor is used to collect road images or point cloud data for image recognition processing, thereby identifying road diseases existing in the road. However, when conducting inspections in tunnels, due to the weak light in the tunnel, if image data is used for disease identification, the acquired images will be weak due to tunnel lighting problems. If the collected images are directly used for disease identification , it will lead to a lower accuracy of the disease identification results, thereby reducing the accuracy of the tunnel inspection results.

Contents of the invention

The embodiment of the present invention provides a tunnel inspection method, which aims to solve the problem that when the existing disease inspection is carried out in the tunnel, due to the weak light in the tunnel, if the collected images are directly used for disease identification, the result of disease identification will be caused. low accuracy problem. By combining the optical flow component and brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and uses the dynamic information of the optical flow component and the static information of the brightness component To assist the disease identification model to identify road diseases, so as to improve the accuracy of disease identification results, and then improve the accuracy of tunnel inspection results.

In a first aspect, an embodiment of the present invention provides a tunnel inspection method, the method comprising:

Obtain the optical flow component and brightness component corresponding to the captured image in the target tunnel;

combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

Processing the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel;

An inspection result of the target tunnel is determined based on a disease identification result of the target tunnel.

Optionally, before the acquisition of the optical flow component and brightness component of the captured image in the target tunnel, the method further includes:

Collect road sign information in front of the target tunnel;

determining shooting parameters of the target tunnel according to the road sign information;

After entering the target tunnel, the target tunnel is photographed with the photographing parameters.

Optionally, the acquiring the optical flow component and the brightness component of the captured image in the target tunnel includes:

Obtain the images taken in the target tunnel and the images taken before;

extracting an optical flow component between the previously captured image and the captured image, to obtain an optical flow component corresponding to the captured image;

converting the captured image into an HSV color mode, and determining a brightness component corresponding to the captured image based on the HSV color mode.

Optionally, the determining the brightness component corresponding to the captured image based on the HSV color mode includes:

determining a first luminance component of the captured image based on the HSV color mode;

A second brightness component of the captured image is predicted according to the first brightness component, and the second brightness component is determined as a corresponding brightness component of the captured image.

Optionally, the captured image is in RGB color mode, and combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image includes:

After splicing the optical flow component and the brightness component to the color component of the captured image, a spliced image is obtained;

A target image is obtained by performing convolution fusion processing on the spliced image through a preset convolution kernel.

Optionally, before the trained disease recognition model is used to process the target image to obtain the result of the target tunnel's disease recognition, the method further includes:

Obtain a first data set and a disease recognition model to be trained, the first data set includes a sample image and a disease label corresponding to the sample image, the sample image is obtained in the same manner as the target image;

performing supervised training on the disease identification model to be trained through the first data set;

After the training is completed, the trained disease recognition model is obtained.

Optionally, the determining the inspection result of the target tunnel based on the disease identification result of the target tunnel includes:

determining the comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

The inspection result of the target tunnel is determined according to the comprehensive disease score.

In the second aspect, the embodiment of the present invention also provides a tunnel inspection device, and the tunnel inspection device includes:

A first acquisition module, configured to acquire an optical flow component and a brightness component corresponding to the captured image in the target tunnel;

A combination module, configured to combine the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

The first processing module is used to process the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel;

The first determination module is configured to determine the inspection result of the target tunnel based on the disease identification result of the target tunnel.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, The steps in the tunnel inspection method provided by the embodiment of the present invention are implemented.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the tunnel inspection method provided by the embodiment of the present invention is implemented. A step of.

In the embodiment of the present invention, the optical flow component and brightness component corresponding to the captured image in the target tunnel are obtained; the optical flow component and the brightness component are combined with the color component of the captured image to obtain the target image; The disease identification model of the target image is processed to obtain a disease identification result of the target tunnel; and an inspection result of the target tunnel is determined based on the disease identification result of the target tunnel. By combining the optical flow component and brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and uses the dynamic information of the optical flow component and the static information of the brightness component To assist the disease identification model to identify road diseases, so as to improve the accuracy of disease identification results, and then improve the accuracy of tunnel inspection results.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flowchart of a tunnel inspection method provided by an embodiment of the present invention;

Fig. 2 is a schematic structural diagram of a tunnel inspection device provided in an embodiment of the present invention;

Fig. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

As shown in FIG. 1 , FIG. 1 is a method flowchart of a tunnel inspection method provided by an embodiment of the present invention. The tunnel inspection method includes steps:

101. Acquire an optical flow component and a brightness component corresponding to a captured image in a target tunnel.

In the embodiment of the present invention, the above-mentioned tunnel may be a tunnel for vehicles to pass through, the above-mentioned tunnel inspection method may be carried on a server or an edge end, and the above-mentioned edge end may be an electronic device for image processing and analysis on an inspection vehicle , The inspection vehicle is equipped with an image capture device, which can be used to capture the road surface and arch surface of the tunnel to obtain corresponding captured images. It should be noted that when the inspection vehicle is driving, the image capture device will capture images at a certain frame rate to obtain continuous captured images.

The above-mentioned target tunnel can be the tunnel where the inspection vehicle is currently located. At this time, the image captured in the target tunnel can be the image captured by the inspection vehicle in real time; or the above-mentioned target tunnel can be a tunnel selected by the user through the interactive interface. , the captured image in the target tunnel may be a historical captured image of the tunnel selected by the user, and the historical captured image is a historical captured image captured by the inspection vehicle in the selected tunnel.

After the image capturing device collects the captured image, it can transmit the captured image to the server or the edge terminal through the data transmission protocol for image processing and analysis.

After the server or the edge end obtains the captured image in the target tunnel, it can extract the optical flow component and brightness component corresponding to the captured image through image processing technology, and use the optical flow component and brightness component corresponding to the captured image to assist in the tunnel inspection. identify.

In a possible embodiment, after receiving the image captured by the image capture device, the above-mentioned edge terminal extracts the optical flow component and brightness component corresponding to the captured image through image processing technology, and converts the captured image and the corresponding optical flow component and The brightness component is uploaded to the server, and the server assists the identification of diseases in the tunnel inspection through the corresponding optical flow component and brightness component of the captured image.

The optical flow component corresponding to the above captured image can represent the change of each pixel point from the previous captured image to the captured image, and the previous captured image can be extracted to the captured image through the Lucas-Kanade algorithm, Farneback algorithm, and PyrLK algorithm The optical flow information between them is used as the optical flow component corresponding to the captured image.

The brightness component corresponding to the above captured image may be determined according to the grayscale image of the captured image, or may be obtained by converting the captured image into an HSV color mode.

It should be noted that in the grayscale image, the grayscale value can be represented by an integer between 0-255. Different grayscale values represent different brightness. The larger the grayscale value, the higher the brightness, and 0 means black. , the lowest brightness, 255 means white, the highest brightness. Specifically, the captured image may be converted into a grayscale image, and the grayscale image may be used as a brightness component corresponding to the captured image.

In the HSV color mode, the color of each pixel can be expressed as a triplet (H, S, V), where H represents the hue, S represents the saturation, and V represents the brightness. In the HSV color mode, the brightness of the image The component can be represented by a V component, and the above V component can also be called a V channel. Specifically, the captured image is converted into the HSV color mode, and the V component is extracted in the HSV color mode as the brightness component corresponding to the captured image.

102. Combine the optical flow component and brightness component with the color component of the captured image to obtain a target image.

In the embodiment of the present invention, the above-mentioned captured image includes multiple color components, and different color modes correspond to different color components, and one color component may also be called a channel. The color mode of the image includes RGB color mode, HSV color mode, CMYK color mode, LAB color mode, etc. Wherein, the color components corresponding to the RGB color mode are R component, G component and B component, the R component represents the red component, the G component represents the green component, and the B component represents the blue component. The color components corresponding to the HSV color mode are H component, S component and V component, wherein the H component represents the hue component, the S component represents the saturation component, and the V component represents the brightness component. The color components corresponding to the CMYK color mode are C component, M component, Y component and K component, where the C component represents the cyan component, the M component represents the magenta component, the Y component represents the yellow component, and the K component represents the black component. The color components corresponding to the LAB color mode are the L component, the A component, and the B component, where the L component represents the lightness component, the A component represents the green/red component, and the B represents the blue/yellow component. In the embodiment of the present invention, the color mode of the captured image is preferably the RGB color mode, because the RGB color mode can provide rich color expression capabilities, so that the captured image has more image details.

The color mode of the captured image is RGB color mode for illustration. After the server or the edge end obtains the optical flow component and brightness component corresponding to the captured image, it can splice the optical flow component and brightness component with the color component of the captured image to obtain the target Image, at this time, the target image includes RGB color components, optical flow components, and brightness components. In a possible embodiment, the optical flow component and the brightness component may be superimposed on the color component of the captured image to obtain the target image, and at this time the target image includes the superimposed RGB color components.

103. Process the target image through the trained defect recognition model to obtain a defect recognition result of the target tunnel.

In the embodiment of the present invention, the above-mentioned disease identification model is used to identify the disease type, disease location and disease degree in the target image. The above-mentioned disease identification model can be a deep convolutional neural network. Specifically, the above-mentioned disease identification model can be based on ResNet , YOLOV, Faster R-CNN, SSD, RetinaNet and other deep convolutional neural networks to build models.

The above-mentioned disease identification model can be trained through a pre-prepared data set, and a trained disease identification model can be obtained after the training is completed. The above-mentioned trained disease recognition model is deployed on the server or the edge. After the target image is obtained on the server or the edge, the target image is input into the disease recognition model for processing, and the disease recognition model outputs the type and location of the disease in the target image. The disease information such as disease degree and disease degree is used as the disease identification result.

The above-mentioned trained disease recognition model outputs a disease detection frame (x, y, w, h, r, u, v), where (x, y) represents the center position of the disease detection frame, and w represents the width of the disease detection frame, h represents the height of the disease detection frame, r represents the confidence of the disease detection frame, u represents the type of disease in the disease detection frame, and v represents the degree of disease in the disease detection frame.

The above-mentioned disease types can be determined by u in the disease detection frame. The disease types can include road surface diseases and arch surface diseases. The above-mentioned road surface diseases can be understood as diseases that appear on the road surface, and the above-mentioned arch surface diseases can be understood as appearing in arch holes Diseases of the inner surface. Specifically, the above-mentioned road surface diseases can be road surface cracks, potholes, wavy road surfaces, settlements and other diseases that affect the life of the road surface and traffic safety, and the above-mentioned arch surface diseases can be surface spalling, cracks, collapses, etc. safe disease.

The above-mentioned disease position may be the position of the disease in the target image, and the disease position may be represented by the center position of the disease detection frame. In a possible embodiment, each frame of captured image has a corresponding shooting time and shooting location. After determining the shooting location of the captured image, the coordinate system between the camera coordinate system of the image capturing device and the real world coordinate system can be used to The transformation relation converts the lesion location in the target image to the lesion location in the real world.

The above-mentioned disease degree can be determined through the v of the disease detection frame, and the disease degree can be expressed in the form of a score, or in the form of a grade. The higher the score of the disease degree, the more serious the disease, or the higher the disease grade, the more serious the disease.

Through the trained disease identification model, the disease identification results of all target images in the target tunnel can be obtained, and the disease identification results of the target tunnel can be obtained. The disease identification result of the target tunnel may include disease information such as disease quantity, disease type, disease location, and disease degree.

104. Determine the inspection result of the target tunnel based on the fault identification result of the target tunnel.

In the embodiment of the present invention, after the defect identification result of the target tunnel is obtained, data analysis may be performed on the basis of the defect identification result of the target tunnel to obtain the inspection result of the target tunnel.

The above inspection results include at least one of qualification and repair level. After obtaining the fault identification result of the target tunnel, the server or the edge end may determine whether the inspection result of the target tunnel is qualified according to the fault identification result of the target tunnel. Alternatively, the repair level of the target tunnel can be determined according to the identification result of the target tunnel. Alternatively, in a case where it is determined that the inspection result of the target tunnel is unqualified, the repair level of the target tunnel may be further determined.

It can be determined whether the inspection result of the target tunnel is qualified according to the number of diseases, disease type, disease location, disease degree, etc. of the target tunnel. For example, when the number of diseases is greater than or equal to the preset number of diseases, the inspection of the target tunnel can be determined. The result is unqualified. When the number of diseases is less than the preset number of diseases, it can be determined that the inspection result of the target tunnel is qualified; when the disease type has a preset type, it can be determined that the inspection result of the target tunnel is unqualified. When the number of diseases does not exist in the preset number of diseases, it can be determined that the inspection result of the target tunnel is qualified; When the position appears on the edge of the road surface or the edge of the arch surface, it can be determined that the inspection result of the target tunnel is qualified; when the disease degree is greater than or equal to the preset disease degree, it can be determined that the inspection result of the target tunnel is unqualified. When the degree of damage is less than the preset disease degree, it can be determined that the inspection result of the target tunnel is qualified. It is also possible to combine the number of diseases, disease types, disease locations, and disease degrees of the target tunnel to determine whether the inspection result of the target tunnel is qualified.

The repair level above can be determined by the damage degree of the target tunnel, the maximum damage degree in the target tunnel can be determined as the repair grade of the target tunnel, and the average damage degree in the target tunnel can also be determined as the repair grade of the target tunnel. In a possible embodiment, the repair grade of the target tunnel can also be determined according to the above-mentioned comprehensive damage score. The higher the comprehensive damage score of the target tunnel is, the higher the repair grade of the target tunnel is, and the lower the comprehensive damage score of the target tunnel is. , the lower the repair level of the target tunnel is.

After the server or the edge terminal obtains the inspection result, it can send the inspection result to the management system of the relevant road department or the user terminal, so that the relevant road department or user can view the inspection result of the target tunnel.

In the embodiment of the present invention, the optical flow component and brightness component corresponding to the captured image in the target tunnel are obtained; the optical flow component and the brightness component are combined with the color component of the captured image to obtain the target image; The image is processed to obtain the result of the target tunnel's disease identification; based on the result of the target tunnel's disease identification, the inspection result of the target tunnel is determined. By combining the optical flow component and brightness component of the captured image with the color component of the captured image, the target image has the dynamic information of the optical flow component and the static information of the brightness component, and uses the dynamic information of the optical flow component and the static information of the brightness component To assist the disease identification model to identify road diseases, so as to improve the accuracy of disease identification results, and then improve the accuracy of tunnel inspection results.

It can be understood that in the specific implementation of this application, related data such as photographed images inside and outside the tunnel are involved. When the embodiment in this application is applied to a specific product or technology, it is necessary to obtain the permission or consent of the relevant department. And the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Optionally, before the step of obtaining the optical flow component and brightness component of the captured image in the target tunnel, road sign information before the target tunnel can also be collected; according to the road sign information, the shooting parameters of the target tunnel are determined; After that, shoot the target tunnel with shooting parameters.

In the embodiment of the present invention, the photographed images collected before the inspection vehicle enters the target tunnel can be acquired, the photographed images collected before entering the target tunnel can be identified and identified, and the road identification information before the target tunnel can be obtained. The above road identification information may include tunnel name, tunnel speed limit, tunnel length, tunnel height and so on. The above shooting parameters include shooting light compensation, shooting frame rate, shooting resolution, focus position and so on.

Different road sign information corresponds to different shooting parameters. The relationship between road sign information and shooting parameters can be recorded through a mapping table. After obtaining the road sign information of the target tunnel, the corresponding shooting parameters can be found through the mapping table , so as to obtain the shooting parameters of the target tunnel.

In a possible embodiment, the shooting parameters may be output through a trained convolutional neural network. The road sign information can be input into the trained convolutional neural network for encoding, feature extraction and result output to obtain the corresponding shooting parameters. Specifically, the above-mentioned road sign information includes tunnel speed limit, tunnel length, tunnel height, etc., and the trained convolutional neural network includes an encoding layer, a convolution layer and an output layer, and the road sign information is encoded through the encoding layer, and the road sign The information is encoded from a string type to a numerical type, and the encoding features corresponding to the road sign information are obtained. The encoding features are convoluted through the convolution layer to obtain the feature vector of the road sign information, and the feature vector is linearly classified through the output layer. The corresponding shooting parameters are obtained for output. Further, the convolutional neural network collects a second data set for supervised training during training, and the second data set includes sample identification information and shooting parameter labels, wherein the sample identification information includes tunnel speed limit, tunnel length, tunnel height, etc. The shooting parameter label includes the shooting light compensation, shooting frame rate, shooting resolution, focus position, etc. actually corresponding to the sample identification information. The above shooting parameter label can be manually marked by relevant professionals. During the training process, the sample identification information is input into the convolutional neural network to be trained. After processing through the encoding layer, convolution layer and output layer, the shooting parameter results corresponding to the sample identification information are obtained, and the shooting parameters corresponding to the sample identification information are calculated. The error loss between the parameter result and the shooting parameter label corresponding to the sample identification information, with the optimization goal of minimizing the error loss, adjusts the parameters in the convolutional neural network to be trained through the error back propagation algorithm, and iterates the above error calculation and In the process of parameter adjustment, the training is completed until the convolutional neural network to be trained converges at the point where the error loss is minimum or the number of iterations reaches the preset number, and the trained convolutional neural network is obtained. After the trained convolutional neural network is obtained, the road identification information of the target tunnel is input to the trained convolutional neural network for processing, and the shooting parameters of the target tunnel are output through the trained convolutional neural network.

After obtaining the shooting parameters of the target tunnel, the shooting parameters are converted into corresponding control instructions. After the inspection vehicle enters the target tunnel, the corresponding image shooting equipment is controlled by the control instructions to shoot, so that the image shooting equipment can use the shooting parameters to control The target tunnel is photographed to obtain photographed images in the target tunnel.

Optionally, in the step of obtaining the optical flow component and the brightness component of the captured image in the target tunnel, the captured image in the target tunnel and the previously captured image can be acquired; the optical flow component between the previously captured image and the captured image is Extract to obtain the optical flow component corresponding to the captured image; convert the captured image to HSV color mode, and determine the brightness component corresponding to the captured image based on the HSV color mode.

In the embodiment of the present invention, when the server or the edge terminal obtains the captured image in the target tunnel, it will also acquire the previous captured image of the captured image. The above-mentioned previously captured image of the target tunnel image can be understood as For an image captured at a moment, the captured image and the previously captured image are two adjacent frame captured images. The optical flow information between the previously captured image and the captured image can be extracted as the optical flow component corresponding to the captured image through the optical flow extraction algorithm such as Lucas-Kanade algorithm, Farneback algorithm, and PyrLK algorithm deployed in the server or edge.

After the server or the edge end obtains the captured image in the target tunnel, the color mode of the captured image can be converted to the HSV color mode through the color mode conversion algorithm, and the corresponding brightness component is extracted in the HSV color mode as the corresponding brightness component of the captured image. It should be noted that, in the HSV color mode, the luminance component of an image may be represented by a V component, and the V component may also be called a V channel. Specifically, the captured image is converted into the HSV color mode, and the V component is extracted in the HSV color mode as the brightness component corresponding to the captured image. Since the resolution size of the image does not change during the color mode conversion process, the extracted brightness component also has the same resolution size as the captured image.

Optionally, in the step of determining the brightness component corresponding to the captured image based on the HSV color mode, the first brightness component of the captured image may also be determined based on the HSV color mode; the second brightness component of the captured image is predicted according to the first brightness component, The second brightness component is determined as the brightness component corresponding to the captured image.

In the embodiment of the present invention, after the server or the edge side converts the captured image into the HSV color mode, the V component in the HSV color mode is extracted as the first luminance component of the captured image, and the resolution size of the first luminance component is the same as that of the captured image. The resolution size is the same, each pixel in the first brightness component corresponds to the pixel with the same coordinates in the captured image, where the value of each pixel in the first brightness component represents the brightness value of the corresponding pixel in the captured image .

After the first luminance component is obtained, prediction processing can be performed on the first luminance component to obtain the second luminance component of the captured image, and the luminance value of each pixel in the second luminance component is greater than the luminance value of each pixel in the first luminance component .

In one embodiment, in the step of predicting the second brightness component of the captured image according to the first brightness component, the first brightness component can be subjected to singular value decomposition (SVD) or principal component analysis (PCA) to obtain the first brightness The singular value or principal component of the component, and then the first brightness component is adjusted by the singular value or principal component to obtain the second brightness component.

In one embodiment, a generator can be trained to generate the second luminance component from the first luminance component. Specifically, a Generative Adversarial Network (GAN) can be constructed, and the Generative Adversarial Network includes a generator and a discriminator. It should be noted that the generator tries to generate samples similar to real data, while the discriminator tries to distinguish real data from the data generated by the generator. During training, the generator and the discriminator compete against each other, and finally the generator can generate realistic samples. Construct the third data set. The third data set includes the sample brightness component and the target brightness component. The target brightness component can be understood as the brightness component after the brightness enhancement of the sample brightness component, and the target brightness component can also be understood as the V in the brighter HSV image. The sample brightness component can also be understood as the V component in the darker HSV image. The darker HSV image can be obtained by reducing the brightness of the brighter HSV image, and the brighter HSV image can also be obtained by increasing the brightness of the darker HSV image. The sample luminance component and the target luminance component have the same resolution size, and the luminance value of each pixel in the sample luminance component is smaller than the luminance value of each pixel in the target luminance component. In the training process, the sample brightness component is input to the generator for generation processing, and the generation result is obtained, and the generation result is input into the discriminator for discrimination processing with the target brightness component. If the discrimination result is false, it indicates that the discriminator judges the generator The generated result of is not similar to the target brightness component. At this time, calculate the error loss between the generated result and the target brightness component, and adjust the parameters of the generator by taking the minimum error loss as the optimization goal, so that the generator can generate The results of the more similar components; if the discriminant result is true, it means that the discriminator judges that the generated result of the generator is very similar to the target luminance component, and the error loss between the generated result and the target luminance component can be calculated, and the minimum error loss is The optimization goal is to adjust the parameters of the discriminator so that the generator can improve the discrimination performance, thereby improving the discrimination accuracy of the discriminator for the generated results, iterating the parameter adjustment process of the generator and the discriminator above until the number of iterations reaches the preset number, Stop iterating and get the trained generator. The first luminance component of the captured image is input into the trained generator, and the generator outputs a corresponding generation result, which is used as the second luminance component of the captured image.

After obtaining the second luminance component, the second luminance component can be determined as the luminance component of the captured image. Since the luminance value of the second luminance component is higher than that of the first luminance component, the second luminance component is determined as The brightness component of the captured image can further increase the brightness of the target image after combining the brightness component with the color component of the captured image.

Optionally, the captured image is in RGB color mode, and in the step of combining the optical flow component and the brightness component with the color component of the captured image to obtain the target image, the optical flow component and the brightness component can be spliced into the color component of the captured image Afterwards, the stitched image is obtained; the stitched image is subjected to convolution and fusion processing through a preset convolution kernel to obtain the target image.

In the embodiment of the present invention, the captured image is preferably in the RGB color mode. The RGB color mode can provide rich color expression capabilities, so that the captured image has more image details. After combining the optical flow component and the brightness component, the target image has light The dynamic information of the flow component and the static information of the luminance component, while retaining rich image details.

The color components of the captured image include R color component, G color component and B color component, and the optical flow component and brightness component can be spliced before or after the R color component, G color component and B color component, so that the spliced image has 5 Component (can be understood as 5 channels), the R color component, G color component and B color component in the stitched image record the image detail information, the optical flow component records the dynamic information of the brightness, and the brightness component records the static information of the brightness.

The aforementioned preset convolution kernel can be a 1*1 convolution kernel, and the function of the 1*1 convolution kernel is to linearly combine multiple channels to obtain a new channel. Through the 1*1 convolution kernel, the color component, optical flow component and brightness component in the spliced image can be convolved, and multiple components can be fused into a single-channel image, and the single-channel image can be determined as the target image. Image details, dynamic information of brightness and static information of brightness are integrated in the image.

Optionally, before the step of processing the target image through the trained disease recognition model to obtain the disease recognition result of the target tunnel, the first data set and the disease recognition model to be trained can also be obtained. The first data set includes sample image and the disease label corresponding to the sample image, the sample image is acquired in the same way as the target image; supervised training is performed on the disease recognition model to be trained through the first data set; the trained disease recognition model is obtained after the training is completed.

In the embodiment of the present invention, the above-mentioned first data set includes a certain number of sample images and disease labels corresponding to the sample images, that is, the sample images are obtained by combining the color components, optical flow components, and brightness components of the original image, specifically For the combination method, please refer to the combination method of the target image in the above-mentioned embodiment, which will not be repeated here. The disease label corresponding to the sample image can be marked by relevant professionals. A sample image can correspond to a set of disease labels, and a set of disease labels includes label information such as disease type, disease location, and disease degree.

In the training process, the sample image is input into the disease recognition model to be trained for processing, and the disease recognition result corresponding to the sample image is output, and the error loss between the disease recognition result corresponding to the sample image and the disease label corresponding to the sample image is calculated , with the optimization goal of minimizing the error loss, adjust the parameters of the disease recognition model to be trained through the error back propagation algorithm, and iterate the above process of error calculation and parameter adjustment until the disease recognition model to be trained converges at the minimum error loss or After the number of iterations reaches the preset number, the training is completed and a trained disease recognition model is obtained. After the trained disease recognition model is obtained, the target image can be input into the trained disease recognition model for processing, and the disease recognition result corresponding to the target image can be output, and the disease recognition result corresponding to the target image is the disease of the captured image recognition result. Further, the disease identification results of all the captured images in the tunnel can be recorded to obtain the disease identification results of the target tunnel.

Optionally, in the step of determining the inspection result of the target tunnel based on the disease identification result of the target tunnel, the comprehensive disease score of the target tunnel can be determined based on the disease identification result of the target tunnel; the inspection result of the target tunnel can be determined according to the comprehensive disease score .

In the embodiment of the present invention, the disease identification result of the target tunnel may include disease information such as the number of diseases, the type of disease, the location of the disease, and the degree of the disease. Quantity, disease type, disease location and disease degree are used to calculate the comprehensive disease score of the target tunnel to obtain the comprehensive disease score of the target tunnel, and determine whether the target tunnel is qualified according to the comprehensive disease score. If the comprehensive disease score of the target tunnel is greater than or equal to the preset If the score threshold of the target tunnel is less than the preset scoring threshold, it can be explained that the inspection result of the target tunnel is unqualified. Specifically, the calculation of the above-mentioned comprehensive disease score S can be performed by the following formula:

Among them, the above-mentioned a _u,i represents the empirical coefficient corresponding to the i-th disease when the disease type is u, the greater the impact of the disease type on traffic safety, the greater the corresponding experience coefficient of the disease type, v _i represents the i-th The disease degree of the disease, the above n is the number of diseases, ( _xi , y _i ) indicates the disease position of the i-th disease, (x ₀ , y ₀ ) indicates the center of the road surface or the center of the arch surface, and the center of the above road surface or the center of the arch surface It can be obtained by performing image recognition processing on the target image. Specifically, it can be obtained through the output of the trained disease recognition model. At this time, the output disease detection frame can be (x, y, x ₀ , y ₀ , w, h, r, u , v), if the disease type in the disease detection frame is road surface disease, then (x ₀ , y ₀ ) represents the center position of the road surface, if the disease type in the disease detection frame is arch surface disease, then (x ₀ , y ₀ ) Indicates the position of the center of the arch. It can be seen from the above formula that the calculation of the comprehensive disease score is positively correlated with the empirical coefficient of the disease type and the degree of disease, and is inversely correlated with the distance from the disease position to the center of the road surface or the center of the arch surface.

It should be noted that the tunnel inspection method provided by the embodiment of the present invention can be applied to devices such as photographing equipment, smart phones, computers, servers, etc. that can perform tunnel inspection.

As shown in Figure 2, an embodiment of the present invention provides a tunnel inspection device, which includes:

The first acquisition module 201 is configured to acquire the optical flow component and brightness component corresponding to the captured image in the target tunnel;

A combining module 202, configured to combine the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

The first processing module 203 is configured to process the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel;

The first determining module 204 is configured to determine an inspection result of the target tunnel based on a disease identification result of the target tunnel.

Optionally, the device also includes:

A collection module, configured to collect road sign information in front of the target tunnel;

A second determining module, configured to determine shooting parameters of the target tunnel according to the road sign information;

The photographing module is configured to photograph the target tunnel with the photographing parameters after entering the target tunnel.

Optionally, the first acquiring module 201 includes:

The acquisition sub-module is used to acquire the image taken in the target tunnel and the image taken before;

An extracting submodule, configured to extract an optical flow component between the previously captured image and the captured image, to obtain an optical flow component corresponding to the captured image;

The first determination sub-module is configured to convert the captured image into an HSV color mode, and determine a brightness component corresponding to the captured image based on the HSV color mode.

Optionally, the first determining submodule includes:

a first determining unit, configured to determine a first brightness component of the captured image based on the HSV color mode;

The second determining unit is configured to predict a second brightness component of the captured image according to the first brightness component, and determine the second brightness component as a corresponding brightness component of the captured image.

Optionally, the combining module 202 includes:

A splicing submodule, configured to splice the optical flow component and the brightness component to the color component of the captured image to obtain a spliced image;

The processing sub-module is used to perform convolution fusion processing on the mosaic image through a preset convolution kernel to obtain a target image.

Optionally, the device also includes:

The second acquisition module is used to acquire a first data set and a lesion recognition model to be trained, the first data set includes a sample image and a lesion label corresponding to the sample image, the sample image is the same as the target image The acquisition method is the same;

A training module, configured to perform supervised training on the disease identification model to be trained through the first data set;

The second processing module is used for training to obtain the trained disease identification model.

Optionally, the first determining module 204 includes:

The second determining submodule is used to determine the comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

The third determining submodule is configured to determine the inspection result of the target tunnel according to the comprehensive damage score.

It should be noted that the tunnel inspection device provided by the embodiment of the present invention can be applied to equipment such as photographing equipment, smart phones, computers, servers, etc. that can perform tunnel inspection.

The tunnel inspection device provided by the embodiment of the present invention can realize each process realized by the tunnel inspection method in the above method embodiment, and can achieve the same beneficial effect. To avoid repetition, details are not repeated here.

Referring to FIG. 3, FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As shown in FIG. 3, it includes: a memory 302, a processor 301, and an A computer program for the tunnel inspection method, wherein:

The processor 301 is used to call the computer program stored in the memory 302, and perform the following steps:

Obtain the optical flow component and brightness component corresponding to the captured image in the target tunnel;

combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image;

Processing the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel;

An inspection result of the target tunnel is determined based on a disease identification result of the target tunnel.

Optionally, before the acquisition of the optical flow component and brightness component of the captured image in the target tunnel, the method executed by the processor 301 further includes:

Collect road sign information in front of the target tunnel;

determining shooting parameters of the target tunnel according to the road sign information;

After entering the target tunnel, the target tunnel is photographed with the photographing parameters.

Optionally, the acquiring the optical flow component and the brightness component of the captured image in the target tunnel executed by the processor 301 includes:

Obtain the images taken in the target tunnel and the images taken before;

extracting an optical flow component between the previously captured image and the captured image, to obtain an optical flow component corresponding to the captured image;

converting the captured image into an HSV color mode, and determining a brightness component corresponding to the captured image based on the HSV color mode.

Optionally, the determining the luminance component corresponding to the captured image based on the HSV color mode performed by the processor 301 includes:

determining a first luminance component of the captured image based on the HSV color mode;

A second brightness component of the captured image is predicted according to the first brightness component, and the second brightness component is determined as a corresponding brightness component of the captured image.

Optionally, the captured image is in RGB color mode, and the combining the optical flow component and the brightness component with the color component of the captured image performed by the processor 301 to obtain the target image includes:

After splicing the optical flow component and the brightness component to the color component of the captured image, a spliced image is obtained;

A target image is obtained by performing convolution fusion processing on the spliced image through a preset convolution kernel.

Optionally, before the target image is processed by the trained disease recognition model to obtain the result of the target tunnel's disease recognition, the method executed by the processor 301 further includes:

Obtain a first data set and a disease recognition model to be trained, the first data set includes a sample image and a disease label corresponding to the sample image, and the sample image is obtained in the same manner as the target image;

performing supervised training on the disease identification model to be trained through the first data set;

After the training is completed, the trained disease recognition model is obtained.

Optionally, the determining the patrol inspection result of the target tunnel based on the disease identification result of the target tunnel performed by the processor 301 includes:

determining the comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel;

The inspection result of the target tunnel is determined according to the comprehensive disease score.

It should be noted that the electronic device provided by the embodiment of the present invention can be applied to smart phones, computers, servers and other devices that can implement the tunnel inspection method.

The electronic device provided by the embodiment of the present invention can realize each process realized by the tunnel inspection method in the above method embodiment, and can achieve the same beneficial effect. To avoid repetition, details are not repeated here.

The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the tunnel inspection method or the tunnel inspection at the application end provided by the embodiment of the present invention is implemented. Each process of the method, and can achieve the same technical effect, in order to avoid repetition, will not repeat them here.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the programs can be stored in a computer-readable storage medium. , may include the flow of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM for short).

The above disclosures are only preferred embodiments of the present invention, and certainly cannot limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims (10)

1. A tunnel inspection method, characterized in that said method comprises the following steps: Obtain the optical flow component and brightness component corresponding to the captured image in the target tunnel; combining the optical flow component and the brightness component with the color component of the captured image to obtain a target image; Processing the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel; An inspection result of the target tunnel is determined based on a disease identification result of the target tunnel. 2. The tunnel inspection method according to claim 1, wherein, before the optical flow component and the brightness component of the captured image in the acquisition target tunnel, the method further comprises: Collect road sign information in front of the target tunnel; determining shooting parameters of the target tunnel according to the road sign information; After entering the target tunnel, the target tunnel is photographed with the photographing parameters. 3. The tunnel inspection method according to claim 2, wherein said obtaining the optical flow component and the brightness component of the captured image in the target tunnel comprises: Obtain the images taken in the target tunnel and the images taken before; extracting an optical flow component between the previously captured image and the captured image, to obtain an optical flow component corresponding to the captured image; converting the captured image into an HSV color mode, and determining a brightness component corresponding to the captured image based on the HSV color mode. 4. The tunnel inspection method according to claim 2, wherein the determination of the brightness component corresponding to the captured image based on the HSV color mode comprises: determining a first luminance component of the captured image based on the HSV color mode; A second brightness component of the captured image is predicted according to the first brightness component, and the second brightness component is determined as a corresponding brightness component of the captured image. 5. The tunnel inspection method according to claim 4, wherein the captured image is in RGB color mode, and the optical flow component and the brightness component are combined with the color component of the captured image , to get the target image, including: After splicing the optical flow component and the brightness component to the color component of the captured image, a spliced image is obtained; A target image is obtained by performing convolution fusion processing on the spliced image through a preset convolution kernel. 6. The tunnel inspection method according to any one of claims 1 to 5, wherein the target image is processed by the trained defect recognition model to obtain a defect recognition result of the target tunnel Previously, the method further included: Obtain a first data set and a disease recognition model to be trained, the first data set includes a sample image and a disease label corresponding to the sample image, the sample image is obtained in the same manner as the target image; performing supervised training on the disease identification model to be trained through the first data set; After the training is completed, the trained disease identification model is obtained. 7. The tunnel inspection method according to claim 6, wherein the determining the inspection result of the target tunnel based on the disease identification result of the target tunnel comprises: determining the comprehensive disease score of the target tunnel based on the disease identification result of the target tunnel; The inspection result of the target tunnel is determined according to the comprehensive disease score. 8. A tunnel inspection device, characterized in that the tunnel inspection device comprises: A first acquisition module, configured to acquire an optical flow component and a brightness component corresponding to the captured image in the target tunnel; A combination module, configured to combine the optical flow component and the brightness component with the color component of the captured image to obtain a target image; The first processing module is used to process the target image through the trained disease recognition model to obtain a disease recognition result of the target tunnel; The first determination module is configured to determine the inspection result of the target tunnel based on the disease identification result of the target tunnel. 9. An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored on the memory and operable on the processor, when the processor executes the computer program, it realizes the The steps in the tunnel inspection method described in any one of requirements 1 to 7. 10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is realized. Steps in the tunnel inspection method.