WO2023065497A1 - Low-illumination obstacle detection method and system, and terminal and storage medium - Google Patents

Abstract

A low-illumination obstacle detection method and system, and a terminal and a storage medium. The method comprises: converting normal-illumination sample images into low-illumination sample images using a cyclic adversarial network; performing enhancement processing on the low-illumination sample images by means of an illumination enhancement network, so as to obtain low-illumination enhanced images; inputting the low-illumination enhanced images into a target detection model for training, so as to obtain a trained target detection model; and performing, by means of the trained target detection model, obstacle recognition on a low-illumination image to be subjected to detection. According to the method, a self-supervised low-illumination enhancement algorithm is introduced to perform enhancement processing on a low-illumination image, and the low-illumination enhancement algorithm and a target detection algorithm are effectively combined, thereby improving the robustness and precision of the target detection algorithm for low-illumination target detection, and solving the problem of the detection effect of performing obstacle detection using a common target detection algorithm in a low-illumination scenario being poor.

Description

A low-light obstacle detection method, system, terminal and storage medium technical field

The present application belongs to the technical field of natural image processing, and in particular relates to a low-light obstacle detection method, system, terminal and storage medium.

Background technique

According to the statistics of the World Health Organization, nearly 253 million people around the world suffer from visual impairment, and 36 million of them have completely lost their vision. Low vision guide equipment also has a huge demand that cannot be ignored.

Traditional blind-guiding means include guide sticks, guide dogs, Braille, blind trails, etc., but the safety of these blind-guiding means is not completely reliable, and they are expensive and have a narrow scope of application. In recent years, with deep learning and its breakthrough development in the field of computer vision, the development of blind guide equipment has ushered in new vitality. At present, the mainstream framework of target detection based on deep learning can be roughly divided into two major technical directions: the two-stage target detection algorithm represented by Fast R-CNN and the single-stage target detection algorithm represented by YOLO series and SSD. Compared with the traditional algorithm, the deep learning algorithm has a breakthrough improvement, which brings a new research boom to the target detection technology. For example, Lin et al. developed an indoor visual assistance system based on Faster-RCNN, which detects daily objects such as fans, chairs, tables, and dehumidifiers in an indoor environment, and notifies users of their current location and plans movement routes through smart devices. Tapu et al developed a wearable device that can detect and alert obstacles with the help of YOLOv3, and designed a smartphone-based object detection and classification scheme to detect objects (cars, bicycles, people, etc.) in an outdoor environment to help visually impaired people; SonayDuman et al. also invented a portable guide system based on YOLO to help visually impaired people perceive objects and people around them and accurately estimate the distance between them.

Among the above, although the existing blind-guiding equipment based on target detection can complete the tasks of identifying obstacles, recognizing faces of people, or guiding the route to the destination, etc., most of the existing blind-guiding equipment or technologies are only applicable to normal It is difficult to meet the needs of blind guides in low-light scenes.

technical problem

The present application provides a low-light obstacle detection method, system, terminal and storage medium, aiming to solve one of the above-mentioned technical problems in the prior art at least to a certain extent.

technical solution

In order to solve the above problems, the application provides the following technical solutions:

A low-light obstacle detection method, comprising:

Convert normal lighting sample images to low lighting sample images using a recurrent adversarial network;

performing enhancement processing on the low-light sample image through a light-enhancing network to obtain a low-light enhanced image;

Inputting the low-light enhanced image into a target detection model for training to obtain a trained target detection model;

Obstacle recognition is performed on the low-light image to be detected through the trained target detection model.

The technical solution adopted in the embodiment of the present application also includes: the conversion of the normal illumination sample image into the low illumination sample image by using the cyclic confrontation network includes:

The cyclic confrontation network includes a generator and a discriminator, the normal illumination image is converted by the generator, and whether the converted image meets the requirements of the low-light image is judged by the discriminator, and if not, the converted image is discarded; if If the low-light image requirements are met, put the transformed image into the low-light image dataset.

The technical solution adopted in the embodiment of the present application further includes: the enhancement processing of the low-light sample image through the illumination enhancement network includes:

The illumination enhancement network includes a CNN network, the input of the CNN network is a low-light image data set, and during network training, the CNN network utilizes the relationship between the low-light image histogram and the normal illumination image histogram curve and A loss function is designed for data distribution, a self-supervised learning network is formed, and a mapping matrix for characterizing illumination enhancement features is output.

The technical solution adopted in the embodiment of the present application further includes: the enhancement processing of the low-light sample image through the illumination enhancement network further includes:

The illumination enhancement network also includes a residual connection adjustment module, the residual connection adjustment module includes four convolutional layers with residual connections; the input of the residual connection adjustment module is a low-light image data set, low After the low-light images in the light image data set are processed by four convolutional layers, three grayscale parameter matrices are obtained, and the mapping matrix output by the CNN network is iteratively adjusted through the three grayscale parameter matrices to obtain each low-light image The image corresponds to the resized image.

The technical solution adopted in the embodiment of the present application further includes: the enhancement processing of the low-light sample image through the illumination enhancement network further includes:

Each low-light image is separately added to the corresponding adjusted image to generate a low-light enhanced image.

The technical solution adopted in the embodiment of the present application further includes: the inputting the low-light enhanced image into the target detection model for training includes:

The target detection algorithm of the target detection model includes YOLO algorithm or Fast R-CNN detection algorithm.

Another technical solution adopted in the embodiment of the present application is: a low-light obstacle detection system, including:

Image conversion module: used to convert the normal illumination sample image into a low illumination sample image by using the cyclic confrontation network;

Image enhancement module: used to enhance the low-light sample image through the light-enhancing network to obtain a low-light enhanced image;

Model training module: used to input the low-light enhanced image into the target detection model for training to obtain a trained target detection model;

Image recognition module: used for performing obstacle recognition on the low-light image to be detected through the trained target detection model.

Another technical solution adopted by the embodiment of the present application is: a terminal, the terminal includes a processor and a memory coupled to the processor, wherein,

The memory stores program instructions for implementing the low-light obstacle detection method;

The processor is configured to execute the program instructions stored in the memory to control low-light obstacle detection.

Yet another technical solution adopted in the embodiment of the present application is: a storage medium storing program instructions executable by a processor, the program instructions being used to execute the low-light obstacle detection method.

Beneficial effect

Compared with the prior art, the beneficial effects produced by the embodiments of the present application are that the low-light obstacle detection method, system, terminal and storage medium of the embodiments of the present application enhance low-light images by introducing a self-supervised low-light enhancement algorithm, Effectively combine the low-light enhancement with the target detection algorithm to improve the robustness and accuracy of the target detection algorithm for low-light target detection, and solve the problem of poor detection effect of using ordinary target detection algorithms for obstacle detection in low-light scenarios question. And adding a residual connection module can make better use of the original low-light image information, avoid the structural damage of the light enhancement network to the low-light image, and further improve the target detection accuracy of the target detection algorithm in low-light conditions.

Description of drawings

Fig. 1 is the flow chart of the low-light obstacle detection method of the embodiment of the present application;

Fig. 2 is a schematic diagram of the structure of the cyclic confrontation network of the embodiment of the present application;

FIG. 3 is a schematic structural diagram of a low-light obstacle detection system according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present application.

Embodiments of the present invention

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

Please refer to FIG. 1 , which is a flow chart of a low-light obstacle detection method according to an embodiment of the present application. The low-light obstacle detection method of the embodiment of the present application includes the following steps:

S10: Obtain a certain number of normal illumination sample images;

In this step, the normal illumination sample image may be obtained from a public dataset of normal illumination images.

S20: Using the recurrent confrontation network to convert the normal illumination sample image into a low illumination sample image, and generate a low illumination data set for training the target detection network;

In this step, most of the existing public datasets of illumination images are normal illumination images, and the amount of data provided by low-light image datasets is very limited (such as the Exdark dataset). In order to ensure that the target detection network can be fully trained, the embodiment of the present application uses a cycle confrontation network (CycleGAN) to convert the normal illumination sample image into a low illumination sample image. Specifically, as shown in FIG. 2 , it is a schematic diagram of the structure of the cyclic confrontation network in the embodiment of the present application. Among them, the cyclic confrontation network includes a generator and a discriminator. The A-style data set represents normal-light images, and the B-style data set represents low-light images. The normal-light images in the A-style data set are converted into low-light images through the generator G _A. Input the converted image into the discriminator D _A to judge whether it conforms to the B style, if not, discard the image, if it conforms to the B style, then put the image into the B style data set, so as to convert the normal light sample image into low light sample image. In the same way, the image in the B-style data set is converted into a normal illumination image through the generator G _B , and the converted image is input into the discriminator D _B to judge whether it conforms to the A style. If it does not conform, the image is discarded. If it conforms to the A style, Then put the image into the A-style dataset to convert the low-light sample image into a normal-light sample image. In such a cycle, after the network training is completed, a low-light data set for training the target detection network can be generated to provide sufficient training data support for the target detection network.

S30: Input the low-light image dataset into the illumination enhancement network for enhancement processing, and output the low-light enhancement image dataset through the illumination enhancement network;

In this step, the illumination enhancement network includes a CNN (convolutional neural network) network and a residual connection adjustment module. The input of the CNN network is a low-light image data set. During network training, the CNN network uses the low-light image histogram and The relationship between the histogram curves of the normal illumination image and the data distribution design a loss function to form a self-supervised learning network, and output a mapping matrix for representing illumination enhancement features. The input of the residual connection adjustment module is a low-light image dataset, including four convolutional layers with residual connections. After the low-light image is processed by four convolutional layers, three grayscale parameter matrices are obtained. Through three The grayscale parameter matrix performs three iterative adjustments on the mapping matrix output by the CNN network to obtain the adjusted image corresponding to each low-light image. Finally, each low-light image is added to the corresponding adjusted image to generate a low-light enhanced image.

In the above, the residual connection adjustment module in the embodiment of the present application can effectively fine-tune the weight of the illumination enhancement network during network training, and can make better use of the original image information, avoiding the structure of the illumination enhancement network on low-light images. on the destruction.

S40: Input the low-light enhanced image data set into the target detection model for training, and output the target category and position information in the low-light enhanced image through the target detection model;

In this step, the target detection model can use the YOLO algorithm or the Fast R-CNN detection algorithm.

S50: Input the low-light image to be detected into the trained target detection model, and perform obstacle recognition on the low-light image through the target detection model.

Based on the above, the low-light obstacle detection method of the embodiment of the present application introduces a self-supervised low-light enhancement algorithm to enhance the low-light image, effectively combines the low-light enhancement with the target detection algorithm, and improves the accuracy of the target detection algorithm for low-light targets. The robustness and precision of the detection solve the problem of poor detection effect of using ordinary target detection algorithms for obstacle detection in low-light scenarios. And adding a residual connection module can make better use of the original low-light image information, avoid the structural damage of the light enhancement network to the low-light image, and further improve the target detection accuracy of the target detection algorithm in low-light conditions.

Please refer to FIG. 3 , which is a schematic structural diagram of a low-light obstacle detection system according to an embodiment of the present application. The low-light obstacle detection system 40 of the embodiment of the present application includes:

Image conversion module 41: used to convert the normal illumination sample image into a low illumination sample image by using the cyclic confrontation network;

Image enhancement module 42: used to enhance the low-light sample image through the light-enhancing network to obtain a low-light enhanced image;

Model training module 43: used for inputting low-light enhanced images into the target detection model for training to obtain a trained target detection model;

Image recognition module 44: used for performing obstacle recognition on the low-light image to be detected through the trained target detection model.

Please refer to FIG. 4 , which is a schematic diagram of a terminal structure in an embodiment of the present application. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51 .

The memory 52 stores program instructions for realizing the above-mentioned low-light obstacle detection method.

The processor 51 is used to execute the program instructions stored in the memory 52 to control low-light obstacle detection.

Wherein, the processor 51 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 51 may be an integrated circuit chip with signal processing capability. The processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components . A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

Please refer to FIG. 5 , which is a schematic structural diagram of a storage medium according to an embodiment of the present application. The storage medium in the embodiment of the present application stores a program file 61 capable of implementing all the above-mentioned methods, wherein the program file 61 can be stored in the above-mentioned storage medium in the form of a software product, and includes several instructions to make a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes. , or terminal devices such as computers, servers, mobile phones, and tablets.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this application may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown in the present application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the present application.

Claims (9)

A low-light obstacle detection method, characterized in that, comprising: Convert normal lighting sample images to low lighting sample images using a recurrent adversarial network; performing enhancement processing on the low-light sample image through a light-enhancing network to obtain a low-light enhanced image; Inputting the low-light enhanced image into a target detection model for training to obtain a trained target detection model; Obstacle recognition is performed on the low-light image to be detected through the trained target detection model. The low-light obstacle detection method according to claim 1, wherein said converting a normal-light sample image into a low-light sample image using a cyclic confrontation network comprises: The cyclic confrontation network includes a generator and a discriminator, the normal illumination image is converted by the generator, and whether the converted image meets the requirements of the low-light image is judged by the discriminator, and if not, the converted image is discarded; if If the low-light image requirements are met, put the transformed image into the low-light image dataset. The low-light obstacle detection method according to claim 2, wherein said enhancing the low-light sample image through the light enhancement network comprises: The illumination enhancement network includes a CNN network, the input of the CNN network is a low-light image data set, and during network training, the CNN network utilizes the relationship between the low-light image histogram and the normal illumination image histogram curve and A loss function is designed for data distribution, a self-supervised learning network is formed, and a mapping matrix for characterizing illumination enhancement features is output. The low-light obstacle detection method according to claim 3, wherein said enhancing the low-light sample image through the light enhancement network further comprises: The illumination enhancement network also includes a residual connection adjustment module, the residual connection adjustment module includes four convolutional layers with residual connections; the input of the residual connection adjustment module is a low-light image data set, low After the low-light images in the light image data set are processed by four convolutional layers, three grayscale parameter matrices are obtained, and the mapping matrix output by the CNN network is iteratively adjusted through the three grayscale parameter matrices to obtain each low-light image The image corresponds to the resized image. The low-light obstacle detection method according to claim 4, wherein said enhancing the low-light sample image through the light enhancement network further comprises: Each low-light image is separately added to the corresponding adjusted image to generate a low-light enhanced image. The low-light obstacle detection method according to any one of claims 1 to 5, wherein said inputting said low-light enhanced image into a target detection model for training comprises: The target detection algorithm of the target detection model includes YOLO algorithm or Fast R-CNN detection algorithm. A low-light obstacle detection system, characterized in that it comprises: Image conversion module: used to convert the normal illumination sample image into a low illumination sample image by using the cyclic confrontation network; Image enhancement module: used to enhance the low-light sample image through the light-enhancing network to obtain a low-light enhanced image; Model training module: used to input the low-light enhanced image into the target detection model for training to obtain a trained target detection model; Image recognition module: used for performing obstacle recognition on the low-light image to be detected through the trained target detection model. A terminal, characterized in that the terminal includes a processor and a memory coupled to the processor, wherein, The memory stores program instructions for realizing the method for detecting obstacles in low light according to any one of claims 1-6; The processor is configured to execute the program instructions stored in the memory to control low-light obstacle detection. A storage medium, characterized in that it stores program instructions executable by a processor, and the program instructions are used to execute the low-light obstacle detection method according to any one of claims 1 to 6.