[go: up one dir, main page]

WO2022117165A1 - Apprentissage d'un réseau de neurones artificiels - Google Patents

Apprentissage d'un réseau de neurones artificiels Download PDF

Info

Publication number
WO2022117165A1
WO2022117165A1 PCT/DE2021/200208 DE2021200208W WO2022117165A1 WO 2022117165 A1 WO2022117165 A1 WO 2022117165A1 DE 2021200208 W DE2021200208 W DE 2021200208W WO 2022117165 A1 WO2022117165 A1 WO 2022117165A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
neural network
artificial neural
vehicle occupant
daylight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/DE2021/200208
Other languages
German (de)
English (en)
Inventor
Christian Scharfenberger
Michelle Karg
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Continental Automotive GmbH
Original Assignee
Continental Automotive GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive GmbH filed Critical Continental Automotive GmbH
Publication of WO2022117165A1 publication Critical patent/WO2022117165A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/617—Upgrading or updating of programs or applications for camera control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the invention relates to a method for training an artificial neural network for converting an input image into an output image, the input image being designed as a night photograph of at least one vehicle occupant. Furthermore, the invention relates to an image processing system with such a method.
  • Today's vehicles are equipped with interior cameras that are intended to monitor the driver in particular.
  • the recognition of the head pose or body posture and face recognition plays an important role, since characteristics such as alertness, tiredness, the direction of view and other properties of the driver's condition can be derived from them.
  • This information is fed to a system in the vehicle, which either generates a warning to the driver or takes a certain action itself if there is a need, such as a lack of attention.
  • DE 10 2005 023 697 A1 shows a device for controlling the interior lighting of a motor vehicle, with at least one sensor being arranged in the motor vehicle which detects the line of sight of vehicle occupants, with a control unit using output variables from the at least one sensor to generate control signals for lighting elements located in the motor vehicle generated. It is therefore an object of the invention to specify means which lead to improved and simplified vehicle occupant monitoring at night.
  • the object is achieved by a method with the features of claim 1.
  • the object is also achieved by such an image system with the features of claim 14 and a use with the features of claim 18.
  • the object is achieved by a method for training an artificial neural network for converting an input image into an output image, the input image being designed as a night photograph of at least one vehicle occupant, comprising the following steps:
  • Output of initial images by the artificial neural network which include at least one predefined area of the at least one vehicle occupant in full illumination or daylight, so that extraction of predetermined vehicle occupant features is made possible by the area brightened in full illumination or daylight.
  • an image processing system for converting an input image into an output image comprising an artificial neural network trained according to a method as described above, the image processing system being designed to convert the input image into an output image, which has at least one predefined area of the at least one vehicle occupant in full illumination or daylight, using the trained artificial neural network, so that the area brightened in full illumination or daylight allows extraction of predetermined vehicle occupant features.
  • a trained artificial neural network is available, which creates a day image from a night photograph of a driver or other vehicle occupants or at least brightens those areas or displays them in daylight that are necessary for an extraction of the desired vehicle occupant features.
  • such areas can be the face, for example, if, for example, a gaze detection is to be determined to determine the alertness/tiredness of the vehicle occupant.
  • An image in daylight can be understood to mean a recording which corresponds to a recording taken in daylight.
  • the areas to be brightened or displayed in daylight can, for example, be specified in advance or dynamically during system operation.
  • Images or recordings mean corresponding image data which are generated with at least one sensor.
  • the method according to the invention and the image processing system according to the invention make it possible to achieve good upgrading of weakly or insufficiently illuminated areas in a simplified manner without additional illumination.
  • image pairs with different exposure times are preferably recorded during the training. These pairs of images are used to train the artificial neural network in such a way that it can reconstruct output images with longer exposures based on input images with shorter exposures.
  • These baseline images are then similar to daylight shots, and further algorithms can be applied to detailed detection of vehicle occupant features on the faces or poses on these baseline images.
  • the invention makes it possible to convert areas of interest into a display that corresponds to a recording with full illumination or daylight, even without additional lighting, despite darkness and a lack of color information.
  • the method according to the invention and the image processing system according to the invention specify an efficient method for improving the image quality in the event of insufficient lighting.
  • the method according to the invention and the image processing system according to the invention achieve a significant improvement in the image quality when displaying night shots without increasing the interior lighting of a vehicle. Therefore, no additional lighting is required to brighten up the interior areas. This is particularly advantageous when using wide-angle cameras, which are usually used in the vehicle interior of a vehicle.
  • the output image thus brightened or generated in daylight can be forwarded to a processing unit for extraction of the desired vehicle occupant characteristics.
  • Various applications can be executed with the aid of the vehicle occupant characteristics obtained in this way, for example a warning tone can be output if, for example, increased tiredness/reduced alertness has been determined.
  • the initial image can be displayed to the vehicle occupant on a display unit in the vehicle, for example via a head-up display.
  • the invention makes it possible to convert a very dark input image with little contrast and color information into a representation that is, for example, daylight or at least sufficiently bright, or to convert at least areas of interest of the image to daylight or at least sufficiently bright.
  • the image processing system preferably precedes a detection or display unit for displaying the processed initial image or for Further processing of the original image.
  • the detection or display unit can also be integrated in the image processing system.
  • existing layers of the artificial neural network are shared with layers for extraction functions so that vehicle occupant features are automatically available. Furthermore, the training for this can preferably take place together.
  • the areas are extracted using semantic segmentation of the interior space.
  • different areas in the input image can be brightened to a different degree, e.g. with additional illumination of individual areas in the interior, for example by a reading lamp or light from outside, or in particularly dark areas in the interior, e.g. by shadows.
  • the at least one predefined area includes the face of the at least one vehicle occupant.
  • vehicle occupant features such as the direction of view/movement of the eyelids can be extracted particularly well from the facial image that has been brightened or converted to daylight. If tiredness is detected, for example, warning tones can be emitted or other measures can be taken.
  • the vehicle occupant features can be extracted and evaluated by a connected evaluation unit, for example, without illuminating the interior of the vehicle too much and causing the driver to record the input image, for example, in a disruptive manner. This also improves the trained artificial neural network image processing system.
  • the at least one predefined area preferably includes at least the head pose of the vehicle occupant. This is also particularly important for detecting tiredness/alertness during night driving. This also improves the trained artificial neural network image processing system.
  • the at least one predefined area preferably includes at least the posture of the vehicle occupant.
  • the driving attention level of the vehicle occupant can be estimated from a posture.
  • a warning tone can also be emitted in the event of an unbalanced posture or even a dangerous posture. This also improves the trained artificial neural network image processing system.
  • the artificial neural network is preferably designed as a CNN (Convolutional Neural Network).
  • This convolutional neural network is particularly suitable for image processing. This also improves the trained artificial neural network image processing system.
  • Such an artificial neural network can automatically learn the parameters for complex scenes by locally and adaptively applying the enhancements to different image areas (people in the interior). Furthermore, such an artificial neural network can reduce the computing time since the CNN can be easily combined with CNNs for a subsequent extraction. With this combination, the vehicle occupant features are enhanced in the artificial neural network so that the extraction functions operate on features that can compensate for the lower illumination at night.
  • the artificial neural network is preferably trained to use information from better illuminated image areas of the input image for the conversion in order to generate the output image. This means that when there is lighting in the passenger compartment, information from the better lit areas is used to further improve the conversion for the unlit areas. This improves the original image. This also improves the trained artificial neural network image processing system.
  • a plurality of input images are preferably provided for conversion into at least one output image, with the artificial neural network being trained in such a way that information from better illuminated image areas of a second input image is used to convert a first input image in order to convert the at least one predefined area of the at least one vehicle occupant into to generate full illumination or daylight as the initial image.
  • the network is trained less with individual images for each camera, but as an overall system consisting of several camera systems.
  • an artificial neural network can be adapted to the conditions of the individual interior spaces and an improved result can be achieved. This also improves the trained artificial neural network image processing system.
  • Information is preferably provided to compensate for missing color and/or contrast and/or brightness information, the artificial neural network being trained to generate the conversion using the color and/or contrast information provided. This means that brightness values or luminance values and/or color information and/or contrast information are provided, with which the artificial neural network achieves improved conversion.
  • the degree of lightening is preferably learned in stages.
  • the method can thus brighten the areas with people in the image by a factor d, with this factor d being dynamically adaptable to the prevailing lighting conditions.
  • the factor d can be adjusted separately for the individual image areas, e.g. driver or occupants in the rear area, so that different lighting conditions in the interior can be taken into account locally.
  • the artificial neural network is trained to simulate a gamma correction and/or a white balance and/or a histogram equalization.
  • the artificial neural network is given a data set consisting of "dark input images (night shots)" and the associated "bright as day” or “illuminated pictures” are made available.
  • the artificial neural network is configured to optimally emulate methods such as white balance, gamma correction and histogram equalization.
  • White balance is essentially the adjustment to the color temperature of the light.
  • Gamma correction is a correction function that is often used in image processing and changes the brightness information of pixels, for example.
  • Histogram equalization is a method for improving contrast in gray-scale images that goes beyond mere contrast enhancement.
  • Image quality information is preferably provided, and the artificial neural network is trained to generate the conversion using the image quality information provided.
  • the network can be trained to generate output images which calculate image data optimized for computer vision and human vision, for example.
  • Computer vision / human vision is understood as the attempt to process and analyze the images recorded by cameras in a wide variety of ways in order to understand their content or extract geometric information.
  • An improved output image can be generated in the image processing system by such an improved artificial neural network.
  • the artificial neural network is preferably trained to convert the input image into an output image which is fully illuminated or displayed in daylight.
  • One or more image sensors are preferably provided for recording the at least one vehicle occupant.
  • the image sensors can be designed as cameras. This achieves good coverage of the vehicle interior.
  • the one or more image sensors are preferably embodied as a wide-angle camera. This allows good coverage to be achieved with just a few cameras.
  • the object is achieved by using the image processing system as described above in a vehicle interior of a vehicle for monitoring at least one vehicle occupant.
  • FIG. 4 shows a further embodiment of a method according to the invention schematically.
  • FIG. 1 shows a training of a neural network according to the invention schematically.
  • this receives as input images 6 (FIG. 3) Night shots from the vehicle interior of a vehicle 1 (FIG 2), which shows at least one vehicle occupant, for example the driver.
  • the input image 6 (FIG. 3) is preferably generated by image sensors such as wide-angle cameras.
  • the artificial neural network is preferably in the form of a CNN convolutional neural network.
  • This convolutional neural network is particularly suitable for machine image processing.
  • Such a network has, for example, several levels.
  • the artificial neural network is then trained in a step S6 to convert the night shot into a brightened initial image or day image (night shot in daylight). For this purpose, several night recordings with different contrast levels/color information and associated desired initial images 7 (FIG. 3) are used during the training.
  • the entire input image 6 (FIG. 3) is preferably converted.
  • the artificial neural network can also be trained to merely brighten different areas from the input image 6 (FIG. 3) or to convert them into a day image. This can be especially the face, head pose and posture.
  • a physical condition (tiredness, lack of concentration, etc.) can be inferred from these vehicle occupant characteristics, for example by extracting the direction of view, the movement of the eyelids, etc. and, if necessary, suitable measures can be taken in the event of a poor physical condition. This can guarantee a safer ride.
  • These areas can be extracted, for example, using semantic segmentation of the interior.
  • different areas in the input image can be brightened to a different degree, e.g. with additional illumination of individual areas in the interior (e.g.
  • the artificial neural network can be trained to use information from better illuminated image areas of the input image 6 (FIG. 3) for conversion when there is lighting in the vehicle interior in order to generate the output image. This allows the conversion for the unlit areas to be further improved and a better output image can be achieved.
  • image pairs with different exposure times are preferably recorded during the training. These pairs of images are used to train the artificial neural network in such a way that it can reconstruct output images with longer exposures based on input images with shorter exposures. These baseline images are then similar to daylight shots, and further algorithms can be applied to detailed detection of vehicle occupant features on the faces or poses on these baseline images.
  • the artificial neural network can be trained to generate the conversion using provided color and/or contrast information.
  • Information stored in the network structure is used to automatically supplement missing color or contrast information in the original image.
  • methods such as gamma correction and/or white balance and/or histogram equalization could be simulated in an optimized manner. In this way, very dark images can be converted into a representation that is advantageous for feature-based recognition or viewing.
  • the artificial neural network is trained to simulate a gamma correction and/or white balance and/or histogram equalization.
  • the artificial neural network is trained using a data set consisting of "dark input images (night shots)" and the associated "bright as day” or “illuminated images”.
  • the artificial neural network is configured to emulate methods such as gamma correction and histogram equalization, etc. In this way, very dark input images 6 (FIG. 3) can be converted into output images 7 (FIG. 3), which are advantageous for feature-based recognition or viewing.
  • the artificial neural network can be trained to generate the conversion using information on the image quality.
  • information stored in the network structure regarding image quality is used in order to achieve a better initial image.
  • the output image is optimized, for example, in that it calculates image data optimized for computer vision and human vision.
  • Steps S2-S5 can each be included in the method individually or in any combination.
  • the 2 shows a vehicle 1 with the image processing system 2 according to the invention, which has an artificial neural network 3 trained with the method according to the invention.
  • the vehicle 1 has a vehicle interior 4 which has interior cameras 5 for recording the vehicle occupants.
  • the interior cameras 5 can in particular be wide-angle cameras.
  • the artificial neural network 3 is trained less with individual images for each interior camera 5, but as an overall system consisting of the multiple interior cameras 5.
  • the image processing system 2 can be integrated as a hardware-based image pre-processing stage in an ISP (Image Signal Processor) of the ISP.
  • the image processing system 2 can carry out the corresponding conversion in the ISP and, for example, make the processed information available with the original data for possible detection or display functions.
  • the image processing system 2 according to the invention specifies a system for improving the image quality in the event of insufficient lighting. Furthermore, the image processing system 2 according to the invention improves the image quality when displaying or processing night shots without additional lighting is required, which brightens the vehicle interior 4. This is a particular advantage when using wide-angle cameras.
  • Image data streams for applications in the vehicle interior 4 can thus be generated by means of the image processing system 2 according to the invention, which has the artificial neural network 3 trained according to the invention. Based on the at least clearly brightened areas of interest, such as the face of the vehicle occupant, features can be extracted and fed to a further processing unit. This can then, for example, analyze these characteristics and carry out measures if there are deviations from the target values
  • the image processing system 2 enables the nighttime recordings of the underlying interior cameras 5 to be converted into a display that corresponds to a recording with full illumination or daylight without additional lighting, despite darkness and a lack of color information, quickly, inexpensively and without disruptive additional interior lighting.
  • the image processing system 2 enables poorly or insufficiently illuminated areas to be well illuminated by means of the trained neural network 3 without additional illumination.
  • the 3 shows an input image 6 which was converted by means of the image processing system 2 according to the invention and the artificial neural network 3 trained according to the invention.
  • the trained artificial neural network 3 is designed here as a CNN.
  • a significantly improved output image 7 can be generated from a dark input image 6, for example for recognizing the head pose or body posture.
  • FIG. 4 shows a further embodiment of a method according to the invention.
  • Steps S11 to S15 here correspond to steps S1 to S5 of FIG
  • the artificial neural network is trained, as in step S6 in FIG. 1, to convert night shots or input images into brightened output images or day images (night shot in daylight).
  • step S6 the artificial neural network is trained, as in step S6 in FIG. 1, to convert night shots or input images into brightened output images or day images (night shot in daylight).
  • several night recordings with different contrast levels/color information and associated desired initial images 7 are used during the training.
  • the artificial neural network is trained in step S16 to identify and determine predefined areas in captured nighttime images that are to be brightened.
  • the predefined area can also include the entire night shot.
  • Predefined areas in the recorded night shots can differ in the desired brightening.
  • different areas can be determined in the image, e.g., using statistical calculations, using semantic segmentation and/or using information about different areas from recordings made at previous times.
  • the types of range determination mentioned above are only examples and should not be regarded as conclusive.
  • a further variant can be, for example, that the network uses segmentation into lighter and darker areas.
  • This segmentation can be obtained, for example, from a previous journal t-1 , from a separate network or from a multitasking network which first outputs a map for image regions and then performs a brightening enhancement based on this map.
  • the latter is a two-step approach, where network calculations from the first step can be reused for the second step, e.g., the calculations of the first network layers.
  • the artificial neural network is trained to individually determine a degree of brightening for each specific area.
  • the artificial neural network is also trained for this purpose, following the determination of the predefined areas and the individual determination of the degree of brightening for the specific areas to use the determined lightening level to lighten the specific areas around the lightening level.
  • the artificial neural network is given a factor d, which corresponds to the parameterization of the illumination of individual image areas and which also determines the degree of brightening.
  • the factor d represents the ratio between the exposure ratio of the input image and the exposure ratio of the brightened output image of the neural network.
  • the specific areas are dynamically adapted to the prevailing light conditions in a vehicle interior by the factor d.
  • the artificial neural network can be trained such that the factor d is adjusted separately for individual image areas, eg, driver or occupants in the rear area, so that locally different lighting conditions in the interior can be addressed. If several areas have been determined by the neural network that are to be brightened, a different factor d can be included in the brightening for each area. For example, a first factor d can be used for a first specific area and a second factor d, which differs from the first factor d, can be used for a second specific area in order to be included in the brightening of the corresponding areas.
  • the factor d can optionally be learned as follows and/or have the following: a) Images with different exposure times are available during the training. As a result, the artificial neural network can gradually learn the degree of brightening. For the training, an image pair is selected from a short exposure and a longer exposure image and the ratio of the exposure times is calculated. This corresponds to the factor d during training. The shorter exposed image and thus the darker image is made available as input to the network. The image with the longer exposure time is used as ground truth for calculating the loss. When calculating the loss, the output of the network is compared with the ground truth. The aim here is that the network learns to reconstruct a brighter image for shorter exposed images and to keep the factor d variable in order to enable a reconstruction of different degrees of brightening.
  • the factor d thus represents an artificial exposure time and the network learns to reconstruct an image with a different exposure time. This is particularly relevant for dynamic environments in which the actual exposure time to be used is limited to short times in order to enable sufficient image sharpness.
  • the factor d can be set variably at runtime, i.e. during operation of an image processing system that uses a suitably trained neural network. For example, a night shot is taken at runtime with an exposure time that is appropriate to the environment in order to enable sufficient image sharpness in a dynamic environment. This usually leads to dark recordings.
  • the trained network is applied to these images and the factor d is set in such a way that the image is sufficiently brightened, for example, similar to a daytime image.
  • the factor d can be determined here in various ways, such as, for example, from a determination of the brightness of the recording, from the brightening of the previous image for magazine t-1, from statistical calculations of the quality of the brightness in the image and the necessary brightening, from a network learning the estimation of the factor d, etc. c)
  • the factor d can be applied locally.
  • the training is extended by the adjustment of the factor d to image regions. Image regions are determined which require different brightening, eg poorly and well-illuminated image areas.
  • the training image pairs can differ for the regions in the exposure time of the ground truth and the network input, so that a factor d can be learned for areas in the network and individual areas can be brightened more.
  • the loss is calculated on the selected areas in the image and recordings of the same scene with different exposure times can be used for the different areas.
  • An example of this is that for a well-lit driver, a medium exposure time ground truth is sufficient, while for example, the occupants in the back seats a ground truth with a longer exposure time is used and therefore the factor d for the driver is correspondingly lower than for the area of the occupants on the back seat.
  • images with different exposure times can also be used for the image regions of the image input of the neural network and these can be combined region-wise with different ground truths with different exposure times.
  • the factor d is calculated during the training for individual image regions from the ratio of the ground truth to the image recording at the network input.
  • the network input is thus composed region by region from different images and the loss is also calculated region by region based on the same image regions by assembling the reference image for the loss calculation region by region from the ground truth recordings corresponding to the image regions in the input image.
  • the factor d is an input parameter of the neural network, based on which the network learns and can reconstruct different degrees of illumination improvement.
  • the factor d can be added to one or more layers of the network as an input parameter.
  • the factor d can also be combined with an additional network, which calculates the mapping of the factor d onto the network for illumination improvement, e.g.
  • the artificial neural network can be trained, for example, with a number of input images that have different exposure times but are otherwise identical, and an associated desired output image.
  • the factor d can be calculated from a downstream application, which is based, for example, on the quality of a possible detection provided and lightened images determines the best factor for image brightening. This can be implemented both when training a neural network and online in the application at runtime. This allows the artificial neural network to gradually learn the degree of whitening.
  • the input images can already be illuminated differently in order to thereby simulate dynamic illumination in the input images and thus train the artificial neural network.
  • the training of the artificial neural network also includes the output of initial images 7 by the artificial neural network, which include at least the one predefined area of the at least one vehicle occupant in full illumination or daylight, so that the areas brightened in full illumination or daylight allow an extraction predetermined vehicle occupant characteristics is made possible.
  • These output initial images can be used in training, for example, to compare them with desired initial images.
  • these images can also be made available to a downstream application which, for example, uses a detection quality to evaluate the quality of the brightened images.
  • training can be continued until an output image meets certain requirements and matches a desired output image as precisely as possible.
  • the output of the network can be used here for a loss calculation of the learning process of the neural network.
  • the output of the neural network can be compared to ground truth and a loss can be calculated. Based on this, the weights of the neural network are updated.
  • the loss can optionally be calculated globally for a constant factor d. Alternatively, the loss can be calculated locally or by region. As a result, a different factor d can be used locally for individual image regions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé d'apprentissage d'un réseau de neurones artificiels (3) destiné à convertir une image d'entrée en une image de sortie, l'image de sortie se présentant sous forme de photo de nuit d'au moins un occupant de véhicule, le procédé comprenant les étapes suivantes : disposer d'un réseau de neurones artificiels (3), effectuer l'apprentissage (S6,S16) du réseau de neurones artificiels (3) sur la base d'images d'entrée, qui se présentent sous forme de photo de nuit d'au moins un occupant de véhicule, au moyen d'images de sortie, qui comprennent au moins une zone prédéfinie dudit au moins un occupant de véhicule en pleine lumière ou en lumière du jour, de sorte qu'une extraction de caractéristiques prédéfinies d'occupants de véhicule est rendue possible par la zone éclairée en pleine lumière ou en lumière du jour, par les étapes suivantes : définir les zones prédéfinies et déterminer un degré d'éclaircissement pour les zones définies, utiliser le réseau de neurones artificiels (3) sur les images d'entrée par utilisation du degré d'éclaircissement déterminé pour les zones définies, sortir des images de sortie par le réseau de neurones artificiels (3). En outre, l'invention concerne un système de traitement d'images (2) avec un tel procédé.
PCT/DE2021/200208 2020-12-01 2021-11-26 Apprentissage d'un réseau de neurones artificiels Ceased WO2022117165A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102020215122.9A DE102020215122A1 (de) 2020-12-01 2020-12-01 Verfahren zum Trainieren eines künstlichen neuronalen Netzes, Bildverarbeitungssystem und Verwendung
DE102020215122.9 2020-12-01

Publications (1)

Publication Number Publication Date
WO2022117165A1 true WO2022117165A1 (fr) 2022-06-09

Family

ID=78822541

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/DE2021/200208 Ceased WO2022117165A1 (fr) 2020-12-01 2021-11-26 Apprentissage d'un réseau de neurones artificiels

Country Status (2)

Country Link
DE (1) DE102020215122A1 (fr)
WO (1) WO2022117165A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102005023697A1 (de) 2005-05-23 2006-11-30 Robert Bosch Gmbh Einrichtung zur Steuerung der Innenbeleuchtung eines Kraftfahrzeugs
US20190043178A1 (en) * 2018-07-10 2019-02-07 Intel Corporation Low-light imaging using trained convolutional neural networks
US20190087648A1 (en) * 2017-09-21 2019-03-21 Baidu Online Network Technology (Beijing) Co., Ltd Method and apparatus for facial recognition
DE112018006886T5 (de) * 2018-02-19 2020-09-24 Mitsubishi Electric Corporation Insassenzustand-Detektionsvorrichtung, Insassenzustand-Detektionssystem und Insassenzustand-Detektionsverfahren

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102005023697A1 (de) 2005-05-23 2006-11-30 Robert Bosch Gmbh Einrichtung zur Steuerung der Innenbeleuchtung eines Kraftfahrzeugs
US20190087648A1 (en) * 2017-09-21 2019-03-21 Baidu Online Network Technology (Beijing) Co., Ltd Method and apparatus for facial recognition
DE112018006886T5 (de) * 2018-02-19 2020-09-24 Mitsubishi Electric Corporation Insassenzustand-Detektionsvorrichtung, Insassenzustand-Detektionssystem und Insassenzustand-Detektionsverfahren
US20190043178A1 (en) * 2018-07-10 2019-02-07 Intel Corporation Low-light imaging using trained convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JINXIU LIANG ET AL: "Recurrent Exposure Generation for Low-Light Face Detection", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 21 July 2020 (2020-07-21), XP081724942 *
YEOREUM CHOI ET AL: "Two-step Learning of Deep Convolutional Neural Network for Discriminative Face Recognition under Varying Illumination", ELECTRONIC IMAGING, vol. 2016, no. 11, 14 February 2016 (2016-02-14), US, pages 1 - 5, XP055363231, ISSN: 2470-1173, DOI: 10.2352/ISSN.2470-1173.2016.11.IMAWM-470 *

Also Published As

Publication number Publication date
DE102020215122A1 (de) 2022-06-02

Similar Documents

Publication Publication Date Title
DE102008027194B4 (de) Gesichtsbildaufnahmevorrichtung
DE112018005191T5 (de) System und Verfahren zur Verbesserung des Signal-Rausch-Verhältnisses bei der Objektverfolgung unter schlechten Lichtbedingungen
DE10203421C1 (de) Automobiles Infrarot-Nachtsichtgerät und automobiles Display
EP3291172B1 (fr) Procédé de détermination de données d'image de résultat
WO2021121491A2 (fr) Conversion de données image d'entrée d'une pluralité de caméras de véhicule d'un système à visibilité périphérique en données image de sortie optimisées
DE102014211889A1 (de) Bildverarbeitungsvorrichtung und Bildverarbeitungsverfahren zum Entfernen von Regenschlieren aus Bilddaten
EP3504871A1 (fr) Procédé et dispositif de fonctionnement d'une caméra d'intérieur
DE102004034532A1 (de) Verfahren zur Kennzeichnung von Bildinformationen in der Darstellung eines mit einer fahrzeugseitigen Bildaufnahmeeinrichtung aufgenommenen Nachtsichtbildes und zugehöriges Nachtsichtsystem
DE102018217183B4 (de) Erlernen einer Lichtverteilung für einen Scheinwerfer
DE102018201909A1 (de) Verfahren und Vorrichtung zur Objekterkennung
DE102019220168A1 (de) Helligkeits-Umwandlung von Bildern einer Kamera
DE102013011844A1 (de) Verfahren zum Anpassen einer Gammakurve eines Kamerasystems eines Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug
WO2022117165A1 (fr) Apprentissage d'un réseau de neurones artificiels
DE102012214637A1 (de) Verfahren und Vorrichtung zur Einstellung einer Leuchtcharakteristik eines Scheinwerfers und Scheinwerfersystem
DE102019106258A1 (de) Insassenüberwachungsvorrichtung
EP1447985A1 (fr) Appareil et méthode pour améliorer la vision dans un vehicule
DE102018010099B4 (de) Vermeiden der Blendung von Personen durch eine Lichtquelle
EP4184456B1 (fr) Procédé et dispositif de détermination d'une image biométrique numérique pour un document de sécurité et procédé de personnalisation d'un document de sécurité
DE102022202827A1 (de) Verfahren zum Bestimmen einer Orientierung einer Kamera
DE10250705A1 (de) Verfahren und Vorrichtung zur Schattenkompensation in digitalen Bildern
WO2022083833A1 (fr) Système pour éviter les accidents provoqués par des animaux sauvages qui traversent au crépuscule et la nuit
DE102017215051A1 (de) Vorrichtung und Verfahren zum Reduzieren des Einflusses von Streulicht und Reflexionen auf optische Bilderkennung
DE102020214713A1 (de) Verfahren zum Unterscheiden einer realen Person von einem Surrogat
DE102023001618A1 (de) Adaptives Dynamikkompressionsverfahren und Fahrzeug
DE102019209061B3 (de) Bildverarbeitungseinrichtung zur Verbesserung einer Nachtsicht, Verfahren und Computerprogramm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21820110

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21820110

Country of ref document: EP

Kind code of ref document: A1