[go: up one dir, main page]

WO2023001110A1 - Procédé et appareil d'entraînement de réseau neuronal, et dispositif électronique - Google Patents

Procédé et appareil d'entraînement de réseau neuronal, et dispositif électronique Download PDF

Info

Publication number
WO2023001110A1
WO2023001110A1 PCT/CN2022/106274 CN2022106274W WO2023001110A1 WO 2023001110 A1 WO2023001110 A1 WO 2023001110A1 CN 2022106274 W CN2022106274 W CN 2022106274W WO 2023001110 A1 WO2023001110 A1 WO 2023001110A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
neural network
degradation
target
parameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/106274
Other languages
English (en)
Chinese (zh)
Inventor
刘行
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Hangzhou Co Ltd
Original Assignee
Vivo Mobile Communication Hangzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Hangzhou Co Ltd filed Critical Vivo Mobile Communication Hangzhou Co Ltd
Publication of WO2023001110A1 publication Critical patent/WO2023001110A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the application belongs to the field of image processing and deep learning, and specifically relates to a neural network training method, device and electronic equipment.
  • the solution to improve the shooting quality of electronic equipment is mainly realized by learning the deep network of face quality enhancement through convolutional neural network.
  • a deep network with a single input resolution (such as 256, 512) and a single image degradation is trained through a training set to perform image enhancement processing on captured images.
  • image processing method in some special scenes, such as low-illumination scenes, the amount of light entering is small when shooting, which is greatly affected by noise. Compared with scenes with normal brightness, the quality of images captured in low-light scenes is degraded. More severe (ie, lower image quality). For another example, when shooting group portraits, the degrees of degradation of faces with inconsistent sizes in the lens are also different.
  • the purpose of the embodiment of the present application is to provide a neural network training method, device and electronic equipment, which can solve the technical problem of poor image processing effect of the deep learning network in the related art.
  • the embodiment of the present application provides a neural network training method, the method includes: determining N degradation degrees according to the acquired M environmental parameters of the M first images, and one degradation degree corresponds to at least one environmental parameter, A degree of degradation corresponds to at least one first image, and M and N are both positive integers; based on each degree of degradation, the first image corresponding to each degree of degradation is degraded to obtain a second image corresponding to each degree of degradation , each second image corresponds to a first image; based on the first image and the second image corresponding to each degree of degradation, a sample set is generated respectively, and N sample sets are obtained; based on N sample sets, Q The neural network is trained, and a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the embodiment of the present application provides a neural network training device, which includes: the device includes: a determination module, a processing module, a generation module and a training module, wherein: the determination module is used to The M environmental parameters of the first image determine N degradation degrees, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to at least one first image, M and N are both positive integers; the above processing module is used to For each degradation degree determined by the determination module, perform degradation processing on the first image corresponding to each degradation degree to obtain a second image corresponding to each degradation degree, and each second image corresponds to a first image respectively; the above-mentioned generation module , used to generate a sample set based on the first image and the second image corresponding to each degree of degradation obtained by the processing module to obtain N sample sets; the above training module is used to obtain N sample sets based on the generation module, Q neural networks are trained respectively, a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the determination module is used to The
  • an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is The processor implements the steps of the method described in the first aspect when executed.
  • an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .
  • the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.
  • an embodiment of the present application provides a computer program product, the program product is stored in a non-volatile storage medium, and the program product is executed by at least one processor to implement the method described in the first aspect.
  • the neural network training device determines N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to at least one first image , M and N are both positive integers; and based on each degree of degradation, degrade the first image corresponding to each degree of degradation to obtain the second image corresponding to each degree of degradation, and each second image corresponds to a The first image, then, based on the first image and the second image corresponding to each degree of degradation above, generate a sample set respectively to obtain N sample sets, and finally, based on the above N sample sets, respectively perform Q neural networks For training, a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the neural network training device can degrade the first image under different environmental parameters to a corresponding degree of degradation (that is, there is a difference), so as to obtain the corresponding degraded image after the first image is degraded differently (that is, the second image). image), so as to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation), and then obtain a neural network suitable for targeted processing of captured images under different environmental parameters.
  • Image enhancement processing effect can be used to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation).
  • Fig. 1 is the flowchart of a kind of neural network training method provided by the embodiment of the present application
  • Fig. 2 is a schematic flow chart of processing by a multi-intensity degradation module provided in an embodiment of the present application
  • Fig. 3 is a schematic diagram of constructing a low-definition-high-definition training set provided by the embodiment of the present application;
  • Fig. 4 is a schematic diagram of ISO and face size information processing provided by the embodiment of the present application.
  • FIG. 5 is a schematic diagram of a multi-complexity GAN network flow provided by an embodiment of the present application.
  • FIG. 6 is a flow chart of an image processing method provided by an embodiment of the present application.
  • Fig. 7 is a schematic structural diagram of a neural network training device provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an image processing device provided in an embodiment of the present application.
  • FIG. 9 is one of the schematic diagrams of the hardware structure of an electronic device provided in the embodiment of the present application.
  • FIG. 10 is a second schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
  • FIG. 1 shows a flowchart of the neural network training method provided in the embodiment of the present application.
  • the neural network training method provided by the embodiment of the present application may include the following steps 101 to 104:
  • Step 101 Determine N degradation degrees according to the acquired M environmental parameters of the M first images.
  • one degree of degradation corresponds to at least one environmental parameter
  • one degree of degradation corresponds to at least one first image
  • both M and N are positive integers.
  • the above-mentioned neural network training method can be a faceEnhanceGAN network (faceEnhanceGAN) training method
  • faceEnhanceGAN faceEnhanceGAN
  • faceEnhanceGAN faceEnhanceGAN
  • Paired image pairs ie, low-resolution images-high-resolution images
  • low-definition pictures can be obtained by performing various degradation processes (such as blurring, adding noise, etc.) on the acquired high-definition pictures (such as high-definition pictures taken by SLR) to obtain low-definition-high-definition image pairs, namely LQ -HQ dataset.
  • various degradation processes such as blurring, adding noise, etc.
  • the above-mentioned first image may be a high-definition picture.
  • the above-mentioned first image may be a high-definition picture captured by a camera of a high-quality imaging device (such as a single-lens reflex camera), or the above-mentioned first image may be a high-definition picture obtained by performing image enhancement on a low-quality image.
  • the clarity and quality of a picture can be measured by the degree of blur and noise, and the above-mentioned high-definition picture refers to a picture with less blur and less noise.
  • each first image and the environmental parameters corresponding to the first images can be stored in the database, and the neural network training device can be used when necessary
  • the M first images can be called from the database.
  • the foregoing environmental parameters may include at least one of the following: ISO value of sensitivity and brightness.
  • the sensitivity refers to the sensitivity to light expressed by numbers. The higher the ISO value, the stronger the sensitivity to light, and vice versa.
  • the M environmental parameters of the M first images may be obtained by inputting the M first images to the ISO and face size information processing module for processing.
  • the neural network training apparatus may determine the degree of degradation of the first image according to the magnitude relationship between the environmental parameters of the first image and the first threshold.
  • the above-mentioned first threshold is an ISO threshold, which may specifically be the size of the ISO.
  • the ISO value of image 1 is greater than the first threshold, the image 1 corresponds to a first degree of degradation (ie, a higher degree of degradation).
  • the multi-stage ISO threshold (ISO_threshold) can be reasonably set by collecting the user's photo data and analyzing the darkness of the scene.
  • the aforementioned environmental parameters may be referred to as prior information
  • the aforementioned ISO and face size information processing module is a preprocessing module for obtaining the prior information.
  • the foregoing N degradation degrees may be one or more of the preset L degradation degrees.
  • the above L degradation degrees may include: a first degradation degree and a second degradation degree.
  • the first degree of degradation may be a high degree of degradation
  • the second degree of degradation may be a low degree of degradation.
  • the above degradation degree may be flexibly determined according to actual conditions, for example, three or more degradation degrees may be set, which is not limited in this embodiment of the present application.
  • the above N degradation degrees correspond to N degradation modules respectively, each degradation module is used to perform corresponding degradation processing on the first image, and the degradation algorithm of each degradation module is different, namely , each degradation module corresponds to a different degradation effect.
  • the above N degradation modules may include: a high ISO segment degradation module and a low ISO segment degradation module. Further, the high ISO segment degradation module may correspond to the above-mentioned first degradation degree, and the low ISO segment degradation module may correspond to the above-mentioned second degradation degree.
  • N degradation modules may also be refined into degradation modules of more ISO segments according to actual requirements, which is not limited in this embodiment of the present application.
  • the high ISO segment degradation module is used to process the first image with a large ISO value, and its degradation degree is relatively high
  • the low ISO segment degradation module is used to process the first image with a large ISO value, and its degradation degree is low .
  • the neural network training device may determine the degree of degradation corresponding to the environment parameter according to the environment parameter of each first image in the M first images, so as to determine the degree of degradation of each first image. For example, taking an ISO value lower than 50 as a low sensitivity and corresponding to the first degree of degradation (for example, a low degree of degradation) as an example, if the ISO value of image 1 is 45, then the degree of degradation corresponding to image 1 is the first degree of degradation , that is, the image 1 corresponds to a low degree of degradation.
  • Step 102 Based on each degree of degradation, perform degradation processing on the first image corresponding to each degree of degradation to obtain a second image corresponding to each degree of degradation.
  • each second image corresponds to a first image.
  • the neural network training device may use a degradation module corresponding to each degradation degree to perform quality degradation processing on the first image corresponding to each degradation degree to obtain a low-quality image or a low-resolution image corresponding to the first image (i.e., the second image above).
  • the neural network training device can use the multi-intensity data degradation module to determine corresponding data degradation modules of different intensities according to the degree of degradation corresponding to the first image, and perform corresponding quality degradation on the first image deal with.
  • the neural network training device can determine the ISO flag bit information through the above-mentioned ISO and face size information processing module, and then determine the corresponding degradation module based on the ISO flag bit information through the above-mentioned multi-intensity data degradation module.
  • the above ISO flag is used to indicate the level of ISO, for example, the ISO flag is a high flag (ie highISO_flag), or the ISO flag is a low flag (ie lowISO_flag).
  • the size relationship between the ISO value of the first image and the above-mentioned ISO threshold ie, ISO_threshold
  • ISO_threshold the size relationship between the ISO value of the first image and the above-mentioned ISO threshold
  • the corresponding degradation module is determined based on the ISO flag bit information through the above multi-intensity data degradation module
  • the ISO flag bit information in the ISO and face size information processing module can be identified, and then based on the identified flag bit information, Determine the corresponding degenerate modules.
  • Example 1 after inputting multiple high-definition images to the ISO and face size information processing module and the multi-intensity data degradation module, if the high ISO flag highISO_flag is recognized to be equal to 1 from the ISO and face size information processing module, then It indicates that the high ISO segment is entered, and at this time, the high ISO segment degradation module is entered, through which the image (that is, the image to be degraded) can be added with multiple types of noise with greater intensity during the degradation process, and multiple types of noise with higher intensity can be added.
  • Large model operations such as triangular models, Gaussian models, linear models, motion blur and other fuzzy functions, all increase the fuzziness.
  • the input high-definition image is randomly darkened to simulate the brightness of the image under dark light.
  • Example 2 after inputting multiple high-definition images to the ISO and face size information processing module and the multi-intensity data degradation module, if the low ISO flag bit lowISO_flag is recognized as 1 from the ISO and face size information processing module, then It means that the image is entered into the low ISO segment. At this time, it enters the low ISO segment degradation module. Through this degradation module, the image can be added with less intense noise, and less types of blur functions can be added for lesser degree of blurring.
  • FIG. 2 is a schematic flow chart of processing using a multi-intensity degradation module.
  • the low ISO flag bit lowISO_flag corresponding to the high-definition image is equal to 1
  • enter the low ISO degradation module to process the high-definition image to obtain the corresponding low-definition image 1
  • the high-ISO flag bit highISO_flag corresponding to the high-definition image If it is equal to 1, enter the high-ISO degradation module to process the high-definition image to obtain the corresponding low-definition image 2.
  • the above-mentioned degradation processing includes but is not limited to at least one of the following: noise addition processing, blur processing (defocus blur, motion blur, Gaussian blur, linear blur), reduce/improve brightness (3D lighting), increase/decrease Shadows (random shadows), area-aware degradation.
  • various blurring models can be added to blur the first image through various blurring functions, for example, defocus model, Gaussian model, linear model, Fuzzy functions such as motion models.
  • the above degradation degree may include at least one of the following: degradation intensity and degradation mode.
  • the above degradation intensity refers to the degree of processing, for example, the size of added noise, the size of the degree of blur
  • the above degradation mode refers to the processing method, for example, adding noise to image 1 and blurring image 2, that is There are two different processing modes.
  • the multi-intensity degradation module is designed to perform differential image quality degradation on the first images of different ISOs (ie, high ISO and low ISO) according to the degradation intensity and degradation mode, so as to construct a better data set Simulate the image quality of electronic equipment from different ISO ranges.
  • FIG. 3 shows a schematic diagram of low-definition-high-definition training set construction.
  • the image training device performs one or more of the above-mentioned degradation processes on several high-definition images to obtain several quality degraded images (i.e., low clear image), so as to obtain the high-definition-low-definition training set.
  • M first images as 100 high-definition images as an example.
  • the ISO values of 40 images in the 100 high-definition images are greater than the ISO threshold value, corresponding to the first degree of degradation
  • the ISO values of the 60 images other than the 40 images are less than the ISO threshold value, corresponding to the second degree of degradation
  • the The 40 images are input to the high ISO segment degradation module corresponding to the first degradation degree for processing
  • the 60 images are input to the low ISO segment degradation module corresponding to the second degradation degree for processing, so as to obtain low clear image.
  • Step 103 Based on the first image and the second image corresponding to each degree of degradation, respectively generate a sample set to obtain N sample sets.
  • one degree of degradation corresponds to one sample set
  • N degrees of degradation correspond to N sample sets, that is, the above N sample sets respectively correspond to N degrees of degradation
  • the N sample sets constitute the training data set.
  • a sample set includes at least one first image and at least one second image, that is, each sample set includes at least one low-definition-high-definition image pair, that is, LQ-HQ data set.
  • one degree of degradation corresponds to one training data set.
  • the neural network training device may store the first image and the second image corresponding to the first image as training sample images in a corresponding training data set.
  • the ISO value is the darker the scene is, and the quality of the picture taken in the darker scene is more severely degraded, such as more blurred, more noise, etc.; the ISO value The smaller the value, the brighter the shooting scene, and the quality degradation of the picture taken in a bright scene is obviously weaker than that in a high ISO scene.
  • the image signal processing i.e., ISP
  • the image quality during the shooting process can be simulated more closely
  • a high-quality training set can be constructed, and a neural network with better face enhancement effect can be trained.
  • Step 104 Based on the above N sample sets, train Q neural networks respectively.
  • one sample set corresponds to at least one neural network
  • Q is a positive integer
  • the aforementioned Q neural networks may include a generative adversarial network (GAN network).
  • GAN network generative adversarial network
  • the aforementioned Q neural networks may be neural networks among a plurality of preset neural networks of different complexity.
  • the neural network training device may train Q neural networks respectively based on the aforementioned N sample sets in the multi-complexity GAN network module.
  • the above multi-complexity GAN network module includes P neural networks, and the neural network training device can determine Q neural networks from the P neural networks for training. Further, the neural network training device determines Q neural networks from the P neural networks for training according to the degree of degradation corresponding to the N sample sets, where P is greater than Q, and P is a positive integer.
  • the neural network training device may respectively input the above N sample sets into the Q neural network to train the neural network, and obtain Q neural networks after training.
  • the above Q neural networks may be the same, different or partly the same.
  • the same neural network refers to a neural network with the same complexity and input resolution.
  • the neural network training device determines N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to At least one first image, M and N are both positive integers; and each degradation degree is used to perform degradation processing on the first image corresponding to each degradation degree to obtain a second image corresponding to each degradation degree, and each The second images respectively correspond to a first image, and then, based on the above-mentioned first image and second image corresponding to each degree of degradation, respectively generate a sample set to obtain N sample sets, and finally, based on the above-mentioned N sample sets, respectively Q neural networks are trained, a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the neural network training device can degrade the first image under different environmental parameters to a corresponding degree of degradation (that is, there is a difference), so as to obtain the corresponding degraded image after the first image is degraded differently (that is, the second image). image), so as to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation), and then obtain a neural network suitable for targeted processing of captured images under different environmental parameters.
  • Image enhancement processing effect can be used to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation).
  • the process of the above step 101 may include the following steps 101a and 101b:
  • Step 101a From the X parameter ranges, determine N target parameter ranges corresponding to the above M environmental parameters.
  • each parameter range corresponds to a degree of degradation
  • a target parameter range corresponds to at least one environmental parameter
  • Step 101b Determine the degradation degree corresponding to the i-th target parameter range among the N target parameter ranges as the i-th degradation degree.
  • the above i-th target parameter range is: according to the order of the corresponding degradation degree from small to large, the i-th target parameter range after sorting the above-mentioned N target parameter ranges, i is a positive integer; the above-mentioned i-th target The environment parameter corresponding to the parameter range is smaller than the environment parameter corresponding to the i+1th target parameter range.
  • the above-mentioned X parameter ranges may be preset parameter ranges, and the parameter ranges refer to the range of ISO values.
  • the X parameter ranges may include three parameter ranges with an ISO value less than 50, an ISO value greater than or equal to 50, less than 200, and an ISO value greater than or equal to 200.
  • the greater the environmental parameter corresponding to the image the darker the shooting scene corresponding to the image is, and the greater the degradation degree of the captured image is. Therefore, the image needs to be degraded to a greater extent to obtain a training data set that can better simulate the real shooting scene. That is, the greater the environmental parameter (ISO value) in the parameter range of the environmental parameter of the first image is, the greater the ISO value of the first image is, and the corresponding degradation degree of the first image is higher.
  • ISO value environmental parameter
  • each of the M first images includes at least one human face element.
  • the neural network training method provided in the embodiment of the present application further includes the following steps A1 and B1:
  • Step A1 Obtain R area size parameters corresponding to the human face elements in the above M first images.
  • each area size parameter is respectively: a size parameter of the area where the face element in the first image is located, R is a positive integer, and R is greater than or equal to M.
  • Step B1 Determine the aforementioned Q neural networks according to the aforementioned M environmental parameters and the aforementioned R area size parameters.
  • a neural network corresponds to: at least one environment parameter and at least one area size parameter.
  • one first image may include one or more human face elements.
  • the image 2 is a group photo of three people, the image 2 includes three face elements, and the image 2 corresponds to three area size parameters.
  • the aforementioned region size parameters may include at least one of the following: length, width, and area.
  • the neural network training device may obtain the region size parameters corresponding to at least one face element in each of the above M first images to obtain R region size parameters, wherein each first The image may correspond to one or more of the above R area size parameters.
  • the neural network training device may respectively determine corresponding environmental parameter levels and area size levels according to the aforementioned M environmental parameters and the aforementioned R area size parameters.
  • the aforementioned environmental parameter level may include a low ISO scene and a high ISO scene
  • the aforementioned area size level may include a large area and a small area.
  • the position information of the face in the first image can be acquired through a face detection algorithm.
  • the above-mentioned ISO and face size information processing module can be used to determine the face area flag information.
  • the face area flag is used to characterize the size of the area of the face element, for example, the face area flag is a large area flag (that is, bigFace_flag), or the face area flag is a small area flag ( That is, smallFace_flag).
  • the size relationship between the area of the face element of the first image and the area threshold (that is, area_threshold) can be judged, if face_area is greater than area_threshold, set the large area flag (or large area sign) to 1, and if face_area is smaller than area_threshold, set the small area flag to 1.
  • the above-mentioned area threshold is a multi-stage area threshold set in advance, and the specific area threshold can be determined by collecting user photo data.
  • the neural network training device can identify the flag information of the face area in the ISO and face size information processing module, and then determine the level of the size of the face area based on the identified flag information. For example, if it is recognized that bigFace_flag is 1, it is determined that the face element is a face element with a large area.
  • the neural network training apparatus may determine the ISO flag bit by comparing the environmental parameter (ie, the ISO value) of the first image with the ISO threshold. For example, if the ISO value is greater than ISO_threshold, set the high ISO flag bit (or high ISO sign bit) to 1, and if the ISO value is smaller than ISO_threshold, set the low ISO flag bit to 1.
  • the environmental parameter ie, the ISO value
  • the neural network training device can identify the ISO flag information in the ISO and face size information processing module, and then determine the ISO level based on the identified flag information. For example, if it is recognized that highISO_flag is equal to 1, it is determined that the first image corresponds to a low ISO value, that is, a low ISO scene.
  • Figure 4 is a schematic diagram of ISO and face size information processing.
  • the neural network training device may determine the neural network corresponding to each sample set according to the identified flag information of the face area corresponding to each sample set and the ISO flag information.
  • the neural network device can identify the N sample sets one by one.
  • the flag bit information of the face area of the first image in a certain sample set is recognized to indicate a large-area face element (that is, bigFace_flag is equal to 1)
  • the neural network training device inputs the sample set into the neural network of the first complexity (ie high complexity), and trains the neural network of the first complexity to obtain the trained neural network;
  • the flag bit information of the face area of the first image in this set indicates a small-area face element (that is, smallFace_flag is equal to 1)
  • the sample set is input into a low-complexity neural network for training.
  • the target parameter corresponding to the neural network with high complexity is greater than the target parameter corresponding to the neural network with small complexity
  • the above target parameters include: environment parameters and area size parameters.
  • the aforementioned neural network with high complexity refers to a neural network with relatively high time complexity and/or space complexity. Further, the complexity of the neural network can be reflected in the depth and breadth of the neural network.
  • FIG. 5 is a schematic diagram of a multi-complexity GAN network process.
  • the ISO value of image 1 corresponds to a low ISO scene
  • it is input into a simple GAN network for training to obtain the low ISO value of the scene.
  • the image to be processed has a trained neural network with better processing effect; assuming that the ISO value of image 1 corresponds to a high ISO scene, it is input into a complex GAN network for training to obtain an image to be processed that corresponds to a high ISO scene.
  • each sample set corresponds to at least two neural networks.
  • step B1 may include the following step C1:
  • Step C1 Determine the first neural network according to the first environment parameter and the first area size parameter
  • the above-mentioned first environmental parameter is: among the above-mentioned N sample sets, among the environmental parameters of the first image in any one of the sample sets;
  • the above-mentioned first area size parameter is: the area size corresponding to the first image in any of the above-mentioned sample sets in the parameter.
  • the above-mentioned first neural network is: one of the neural networks corresponding to any of the above-mentioned sample sets.
  • the first image in the aforementioned one sample set corresponds to a degree of degradation
  • the degree of degradation may correspond to a neural network
  • each sample set is a high-definition-low-definition image pair
  • the low-definition image in each sample set is a degraded image of the high-definition image.
  • the area size parameters (that is, the face area) of the face elements in the first image in a sample set are not the same, for example, there are large-area face elements and small-area face elements in the sample set.
  • a neural network with a larger input resolution and higher complexity can be selected for the first image including a large-area face element, and the first image selection for a face element including a small area Feed a neural network with a smaller resolution and lower complexity for training. In this way, the training efficiency can be improved while achieving a better training effect.
  • FIG. 6 shows a flow chart of the image processing method provided by the embodiment of the present application.
  • the neural network training method provided by the embodiment of the present application may include the following steps 201 and 202:
  • Step 201 In the case of displaying the preview image collected by the camera of the above-mentioned electronic device, according to the target environment parameters of the above-mentioned preview image, and the target area size parameters corresponding to the face elements in the above-mentioned preview image, from the Q neural networks after training , determine the target neural network.
  • the above-mentioned target environment parameters include at least one of the following: ISO value and brightness value; the above-mentioned target area size parameters include at least one of the following: width, height, and area.
  • the current ISO value of the camera may be obtained, and the current ISO value of the camera may be determined as the ISO value of the preview image, and the face of the face element in the preview image may be obtained area information.
  • the image processing device may determine whether the current shooting scene is a high ISO scene or a low ISO scene based on the obtained ISO value, and determine whether the face element to be processed is a large area or a small area based on the obtained face area information. area.
  • the above-mentioned target neural network is a trained adversarial neural network that matches the above-mentioned target environment parameters and target area size parameters.
  • the above-mentioned target neural network is used to perform image enhancement processing on the images captured by the camera.
  • Step 202 Input the third image captured by the above-mentioned camera into the above-mentioned target neural network for image processing to obtain a processed image.
  • the above-mentioned third image is: an image captured by the camera within a predetermined time period
  • the above-mentioned predetermined time period is a time period between the time when the camera collects the preview image and the time when the camera stops collecting the preview image.
  • the above-mentioned third image may be an image including human face elements.
  • the above image processing may be image enhancement processing, for example, performing enhancement processing on the human face area in the image by means of denoising, removing blur, increasing brightness, increasing contrast, and the like.
  • the image processing device can display the preview image captured by the camera of the electronic device, according to the target environment parameters of the preview image, and the size of the target area corresponding to the face elements in the preview image parameters, and determine the target neural network from the Q neural networks after training.
  • the image processing device can intelligently select a neural network that conforms to the current environmental parameters and face size for image enhancement processing based on the environmental parameters when the image is captured and the size of the human face in the captured image, thereby greatly Improves the effect and performance of face enhancement on captured images.
  • the image processing method provided in the embodiment of the present application also includes the following step D1:
  • Step D1 In the case that the above-mentioned preview image contains human face elements, obtain target environment parameters and target area size parameters corresponding to the human face elements.
  • the process of obtaining the target neural network may include the following steps 201a:
  • Step 201a Based on the pre-stored Q correspondences, determine the above-mentioned target neural network corresponding to the above-mentioned target environment parameters and the above-mentioned target area size parameters.
  • each corresponding relationship is respectively: a corresponding relationship between at least one environment parameter, at least one region size parameter and a trained neural network.
  • each of the above correspondences is respectively: a correspondence between an environment parameter level, an area size level and a trained neural network.
  • the above corresponding relationship may be established when a trained neural network is obtained after the training of a neural network is completed.
  • the trained neural network A corresponds to a low-ISO scene
  • the trained neural network B corresponds to a high-ISO scene.
  • the above-mentioned target neural network can be neural network A.
  • the trained neural network C corresponds to a large-area face element
  • the trained neural network D corresponds to a small-area face element.
  • the above-mentioned target neural network can be neural network network C.
  • the neural network training method provided in the embodiment of the present application may be executed by a neural network training device, or a control module in the neural network training device for executing the neural network training method.
  • the neural network training device provided by the embodiment of the present application is described by taking the neural network training method executed by the neural network training device as an example.
  • the embodiment of the present application provides a neural network training device 600, as shown in Figure 7, the device includes: a determination module 601, a processing module 602, a generation module 603 and a training module 604, wherein:
  • the above determination module 601 is configured to determine N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to at least one first image, M, N are positive integers; the processing module 602 is configured to perform degradation processing on the first image corresponding to each degradation degree based on each degradation degree determined by the determination module 601 to obtain the second image corresponding to each degradation degree.
  • each second image corresponds to a first image
  • the generation module 603 is used to generate a sample set based on the first image and the second image corresponding to each degree of degradation obtained by the processing module 602, and obtain N sample sets
  • the training module 604 is used to train Q neural networks based on the above N sample sets obtained by the generating module 603, one sample set corresponds to at least one neural network, and Q is a positive integer.
  • the determination module 601 is specifically configured to determine N target parameter ranges corresponding to the above M environmental parameters from the X parameter ranges, and each parameter range corresponds to a degradation degree , one target parameter range corresponds to at least one environmental parameter; the determination module 601 is specifically configured to determine the degradation degree corresponding to the i-th target parameter range in the above-mentioned N target parameter ranges as the i-th degradation degree; wherein, the above-mentioned The i-th target parameter range is: the i-th target parameter range after sorting the above N target parameter ranges according to the order of the corresponding degradation degree from small to large, where i is a positive integer; the above-mentioned i-th target parameter range corresponds to The environmental parameter of is smaller than the environmental parameter corresponding to the i+1th target parameter range.
  • each of the M first images includes at least one human face element
  • the above device also includes: an acquisition module 605;
  • the acquisition module 605 is configured to acquire R area size parameters corresponding to the face elements in the M first images, and each area size parameter is: the area where the face elements in the first image are located. Size parameter, R is a positive integer, and R is greater than or equal to M; the above determination module 601 is also used to determine the above Q neural networks according to the M environmental parameters obtained by the above acquisition module 605 and the above R area size parameters; wherein, one The neural network corresponds to: at least one environment parameter and at least one region size parameter.
  • the target parameters corresponding to the neural networks with high complexity are greater than the target parameters corresponding to the neural networks with small complexity; wherein, the above-mentioned target parameters include: environmental parameters and area size parameters.
  • each sample set corresponds to at least one neural network
  • the above determination module 601 is specifically configured to determine the first neural network according to the first environment parameter and the first area size parameter; wherein, the above first environment parameter is: the first image in any one of the N sample sets mentioned above Among the environmental parameters; the above-mentioned first area size parameter is: among the area size parameters corresponding to the first image in any of the above-mentioned sample sets;
  • the above-mentioned first neural network is: one of the neural networks corresponding to any of the above-mentioned sample sets.
  • the neural network training device determines N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to At least one first image, M and N are both positive integers; and each degradation degree is used to perform degradation processing on the first image corresponding to each degradation degree to obtain a second image corresponding to each degradation degree, and each The second images respectively correspond to a first image, and then, based on the above-mentioned first image and second image corresponding to each degree of degradation, respectively generate a sample set to obtain N sample sets, and finally, based on the above-mentioned N sample sets, respectively Q neural networks are trained, a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the neural network training device can degrade the first image under different environmental parameters to a corresponding degree of degradation (that is, there is a difference), so as to obtain the corresponding degraded image after the first image is degraded differently (that is, the second image). image), so as to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation), and then obtain a neural network suitable for targeted processing of captured images under different environmental parameters.
  • Image enhancement processing effect can be used to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation).
  • the image processing method provided in the embodiment of the present application may be executed by an image processing device, or a control module in the image processing device for executing the image processing method.
  • the image processing device executed by the image processing device is taken as an example to describe the image processing device provided in the embodiment of the present application.
  • the embodiment of the present application provides an image processing device 700.
  • the device includes Q neural networks trained by the above neural network training method, and Q is a positive integer;
  • the device includes: a determination module 701 and a processing module 702, of which:
  • the above-mentioned determination module 701 is used for displaying the preview image collected by the camera of the electronic device, according to the target environment parameters of the preview image, and the target area size parameters corresponding to the face elements in the preview image, from the Q neurons after training In the network, determine the target neural network;
  • the processing module 702 is configured to input the third image captured by the camera into the target neural network determined by the determination module 701 for image processing to obtain a processed image.
  • the third image is: an image captured by the camera within a predetermined period of time.
  • the predetermined time period is: the time period between the moment when the camera captures the preview image and the moment when the camera stops capturing the preview image.
  • the device 700 further includes an acquisition module 703, the above-mentioned acquisition module 703 is used to acquire the target environment parameters corresponding to the face element when the above-mentioned preview image contains a face element Target area size parameter.
  • the above-mentioned determination module 701 is specifically configured to determine the target neural network corresponding to the above-mentioned target environment parameters and the above-mentioned target area size parameters based on the pre-stored Q correspondences; wherein, each corresponding The relationships are respectively: a corresponding relationship between at least one environment parameter, at least one area size parameter and a trained neural network.
  • the image processing device can display the preview image captured by the camera of the electronic device, according to the target environment parameters of the preview image, and the size of the target area corresponding to the human face element in the preview image parameters, and determine the target neural network from the Q neural networks after training.
  • the image processing device can intelligently select a neural network that conforms to the current environmental parameters and face size for image enhancement processing based on the environmental parameters when the image is captured and the size of the human face in the captured image, thereby greatly Improves the effect and performance of face enhancement on captured images.
  • the neural network training device and the image processing device in the embodiments of the present application may be devices, or components, integrated circuits, or chips in a terminal.
  • the device may be a mobile electronic device or a non-mobile electronic device.
  • the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant).
  • non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.
  • Network Attached Storage NAS
  • personal computer personal computer, PC
  • television television
  • teller machine or self-service machine etc.
  • the neural network training device and the image processing device in the embodiment of the present application may be devices with an operating system.
  • the operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.
  • the neural network training device provided by the embodiment of the present application can realize various processes realized by the method embodiments in FIG. 1 to FIG. 5 , and details are not repeated here to avoid repetition.
  • the image processing apparatus provided in the embodiment of the present application can implement various processes implemented by the method embodiments in FIG. 6 and FIG. 7 , and details are not repeated here to avoid repetition.
  • the embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and programs or instructions stored in the memory 802 and operable on the processor 801.
  • the programs or instructions are executed by the processor 801
  • the various processes of the above-mentioned neural network training method or the above-mentioned image processing method embodiments can be achieved, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
  • the electronic device 100 includes but is not limited to: a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110, etc. part.
  • the electronic device 100 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 110 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions.
  • a power supply such as a battery
  • the structure of the electronic device shown in FIG. 10 does not constitute a limitation to the electronic device.
  • the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .
  • the above-mentioned processor 110 is configured to determine N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, one degradation degree corresponds to at least one first image, and M , N are both positive integers; the processor 110 is configured to perform degradation processing on the first image corresponding to each degradation degree based on each degradation degree, to obtain the second image corresponding to each degradation degree above, each The second images respectively correspond to a first image; the processor 110 is configured to generate a sample set based on the first image and the second image corresponding to each degree of degradation, and obtain N sample sets; the processor 110, It is used to train Q neural networks respectively based on the above N sample sets, one sample set corresponds to at least one neural network, and Q is a positive integer.
  • the above-mentioned processor 110 is specifically configured to determine N target parameter ranges corresponding to the above-mentioned M environmental parameters from the X parameter ranges, and each parameter range corresponds to a degradation degree , one target parameter range corresponds to at least one environmental parameter; the above-mentioned processor 110 is specifically configured to determine the degradation degree corresponding to the i-th target parameter range in the N target parameter ranges as the i-th degradation degree; wherein, the above-mentioned The range of the i target parameter is: according to the order of the corresponding degradation degree from small to large, the ith target parameter range after sorting the above N target parameter ranges, i is a positive integer; the above i-th target parameter range corresponds to The environment parameter is smaller than the environment parameter corresponding to the i+1th target parameter range.
  • each of the M first images includes at least one human face element; the processor 110 is configured to obtain the human face element in the M first images Corresponding R area size parameters, each area size parameter is: a size parameter of the area where the face element in the first image is located, R is a positive integer, and R is greater than or equal to M; the processor 110 also It is used to determine the aforementioned Q neural networks according to the aforementioned M environmental parameters and the aforementioned R area size parameters; wherein, one neural network corresponds to: at least one environmental parameter and at least one area size parameter.
  • the target parameters corresponding to the neural networks with high complexity are greater than the target parameters corresponding to the neural networks with small complexity; wherein, the above-mentioned target parameters include: environmental parameters and area size parameters.
  • each sample set corresponds to at least one neural network
  • the above-mentioned processor 110 is specifically configured to determine the first neural network according to the first environment parameter and the first area size parameter; wherein, the above-mentioned first environment parameter is: the first image in any one of the above-mentioned N sample sets Among the environmental parameters; the above-mentioned first area size parameter is: among the area size parameters corresponding to the first image in any of the above-mentioned sample sets;
  • the above-mentioned first neural network is: one of the neural networks corresponding to any of the above-mentioned sample sets.
  • the electronic device determines N degradation degrees according to the acquired M environmental parameters of the M first images, one degradation degree corresponds to at least one environmental parameter, and one degradation degree corresponds to at least one of the first images.
  • An image where M and N are both positive integers; and using each degree of degradation, degrade the first image corresponding to each degree of degradation to obtain a second image corresponding to each degree of degradation, and each second image is respectively Corresponding to a first image, then, based on the above-mentioned first image and second image corresponding to each degree of degradation, generate a sample set respectively, and obtain N sample sets, and finally, based on the above-mentioned N sample sets, Q nerves
  • the network is trained, a sample set corresponds to at least one neural network, and Q is a positive integer.
  • the electronic device can degrade the first image under different environmental parameters corresponding to the degree of degradation (that is, there is a difference), so as to obtain the corresponding degraded image (that is, the second image) after the first image is degraded differently. , so as to construct a training sample set that conforms to the shooting quality under various environmental parameters (that is, different degrees of degradation), and then obtain a neural network suitable for targeted processing of captured images under different environmental parameters, and improve the accuracy of the image. Enhanced processing.
  • the above-mentioned processor 110 is configured to, in the case of displaying a preview image collected by a camera of an electronic device, according to the target environment parameters of the preview image and the target area size parameters corresponding to the face elements in the preview image, from the trained Q In the first neural network, the target neural network is determined; the third image taken by the camera is input to the target neural network determined by the determination module 701 for image processing, and the processed image is obtained.
  • the above-mentioned third image is: the camera shoots within a predetermined time period
  • the predetermined time period is the time period between the moment when the camera captures the preview image and the moment when the camera stops capturing the preview image.
  • the processor 110 is configured to acquire target environment parameters and target area size parameters corresponding to the face elements when the preview image contains face elements.
  • the above-mentioned processor 110 is specifically configured to determine the target neural network corresponding to the target environment parameter and the target area size parameter based on the pre-stored Q correspondences; wherein, each correspondence is respectively is: a corresponding relationship between at least one environment parameter, at least one region size parameter and a trained neural network.
  • the electronic device may display the preview image captured by the camera of the electronic device, according to the target environment parameters of the preview image and the target area size parameters corresponding to the face elements in the preview image, A target neural network is determined from the Q neural networks after training.
  • the electronic device can intelligently select a neural network that conforms to the current environmental parameters and face size for image enhancement processing based on the environmental parameters and the size of the human face in the captured image, thereby greatly improving Effects and performance of face enhancements on captured images.
  • the input unit 104 may include a graphics processing unit (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processing unit 1041 is used by the image capturing device (such as the image data of the still picture or video obtained by the camera) for processing.
  • the display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the user input unit 107 includes a touch panel 1071 and other input devices 1072 .
  • the touch panel 1071 is also called a touch screen.
  • the touch panel 1071 may include two parts, a touch detection device and a touch controller.
  • Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.
  • Memory 109 may be used to store software programs as well as various data, including but not limited to application programs and operating systems.
  • the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communications. It can be understood that the foregoing modem processor may not be integrated into the processor 110 .
  • the embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, each process of the above-mentioned neural network training method embodiment is realized, or the above-mentioned image
  • a readable storage medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, each process of the above-mentioned neural network training method embodiment is realized, or the above-mentioned image
  • the processor is the processor in the electronic device described in the above embodiments.
  • the readable storage medium includes computer readable storage medium, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
  • the embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above-mentioned embodiment of the neural network training method
  • the chip includes a processor and a communication interface
  • the communication interface is coupled to the processor
  • the processor is used to run programs or instructions to implement the above-mentioned embodiment of the neural network training method
  • chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.
  • the embodiment of the present application provides a computer program product, the program product is stored in a non-volatile storage medium, and the program product is executed by at least one processor to realize the various processes of the above-mentioned neural network training method embodiment, or the above-mentioned image
  • Each process of the embodiment of the method is processed, and the same technical effect can be achieved.
  • the term “comprising”, “comprising” or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase “comprising a " does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.
  • the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
  • the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation.
  • the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to enable a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present application.
  • a terminal which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un procédé et un appareil d'entraînement de réseau neuronal, et un dispositif électronique. Le procédé consiste à : déterminer N degrés de dégradation en fonction de M paramètres environnementaux de M premières images obtenues (101), un degré de dégradation correspondant à au moins un paramètre d'environnement, un degré de dégradation correspondant à au moins une première image et M et N étant des entiers positifs; effectuer un traitement de dégradation sur la première image correspondant à chaque degré de dégradation sur la base de chaque degré de dégradation de façon à obtenir une deuxième image correspondant à chaque degré de dégradation (102), chaque deuxième image correspondant à une première image; générer un ensemble d'échantillons sur la base de la première image et de la deuxième image correspondant à chaque degré de dégradation pour obtenir N ensembles d'échantillons (103); entraîner respectivement Q réseaux neuronaux sur la base des N ensembles d'échantillons (104), un ensemble d'échantillons correspondant à au moins un réseau neuronal, et Q étant un nombre entier positif.
PCT/CN2022/106274 2021-07-23 2022-07-18 Procédé et appareil d'entraînement de réseau neuronal, et dispositif électronique Ceased WO2023001110A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110838140.6A CN113706402B (zh) 2021-07-23 2021-07-23 神经网络训练方法、装置及电子设备
CN202110838140.6 2021-07-23

Publications (1)

Publication Number Publication Date
WO2023001110A1 true WO2023001110A1 (fr) 2023-01-26

Family

ID=78650358

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/106274 Ceased WO2023001110A1 (fr) 2021-07-23 2022-07-18 Procédé et appareil d'entraînement de réseau neuronal, et dispositif électronique

Country Status (2)

Country Link
CN (1) CN113706402B (fr)
WO (1) WO2023001110A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113706402B (zh) * 2021-07-23 2024-12-06 维沃移动通信（杭州）有限公司 神经网络训练方法、装置及电子设备
CN114090453A (zh) * 2021-11-26 2022-02-25 上汽通用汽车有限公司 一种机器视觉系统测试方法、系统及存储介质
WO2023149649A1 (fr) * 2022-02-07 2023-08-10 삼성전자 주식회사 Dispositif électronique et procédé destinés à améliorer la qualité d'image

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109285119A (zh) * 2018-10-23 2019-01-29 百度在线网络技术（北京）有限公司 超分辨图像生成方法及装置
CN111105375A (zh) * 2019-12-17 2020-05-05 北京金山云网络技术有限公司 图像生成方法及其模型训练方法、装置及电子设备
CN111950693A (zh) * 2019-05-14 2020-11-17 辉达公司 使用衰减参数进行神经网络推理
US20210073945A1 (en) * 2019-09-11 2021-03-11 Lg Electronics Inc. Method and apparatus for enhancing image resolution
CN113706402A (zh) * 2021-07-23 2021-11-26 维沃移动通信（杭州）有限公司 神经网络训练方法、装置及电子设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110378235B (zh) * 2019-06-20 2024-05-28 平安科技（深圳）有限公司 一种模糊人脸图像识别方法、装置及终端设备
CN112241668B (zh) * 2019-07-18 2024-06-28 杭州海康威视数字技术股份有限公司 图像处理方法、装置及设备
CN111968052B (zh) * 2020-08-11 2024-04-30 北京小米松果电子有限公司 图像处理方法、图像处理装置及存储介质
CN112330574B (zh) * 2020-11-30 2022-07-12 深圳市慧鲤科技有限公司 人像修复方法、装置、电子设备及计算机存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109285119A (zh) * 2018-10-23 2019-01-29 百度在线网络技术（北京）有限公司 超分辨图像生成方法及装置
CN111950693A (zh) * 2019-05-14 2020-11-17 辉达公司 使用衰减参数进行神经网络推理
US20210073945A1 (en) * 2019-09-11 2021-03-11 Lg Electronics Inc. Method and apparatus for enhancing image resolution
CN111105375A (zh) * 2019-12-17 2020-05-05 北京金山云网络技术有限公司 图像生成方法及其模型训练方法、装置及电子设备
CN113706402A (zh) * 2021-07-23 2021-11-26 维沃移动通信（杭州）有限公司 神经网络训练方法、装置及电子设备

Also Published As

Publication number Publication date
CN113706402A (zh) 2021-11-26
CN113706402B (zh) 2024-12-06

Similar Documents

Publication Publication Date Title
Peng et al. LVE-S2D: Low-light video enhancement from static to dynamic
CN111667420B (zh) 图像处理方法及装置
WO2023001110A1 (fr) Procédé et appareil d'entraînement de réseau neuronal, et dispositif électronique
CN110839129A (zh) 图像处理方法、装置以及移动终端
CN113012081A (zh) 图像处理方法、装置和电子系统
CN111835982A (zh) 图像获取方法、图像获取装置、电子设备及存储介质
CN112330546B (zh) 图像增强方法及相关产品
CN108566516A (zh) 图像处理方法、装置、存储介质及移动终端
CN111047543A (zh) 图像增强方法、装置和存储介质
CN112308797A (zh) 角点检测方法、装置、电子设备及可读存储介质
CN113989387A (zh) 相机拍摄参数调整方法、装置及电子设备
CN112837251A (zh) 图像处理方法及装置
WO2023011280A1 (fr) Procédé et appareil d'estimation de degré de bruit d'image, ainsi que dispositif électronique et support d'enregistrement
CN117391975B (zh) 一种高效的实时水下图像增强方法及其模型搭建方法
CN109961403B (zh) 照片的调整方法、装置、存储介质及电子设备
CN105338221B (zh) 一种图像处理方法及电子设备
CN113709370A (zh) 图像生成方法、装置、电子设备及可读存储介质
CN106651918B (zh) 抖动背景下的前景提取方法
CN108495038A (zh) 图像处理方法、装置、存储介质及电子设备
CN112528770A (zh) 人体质量评价方法、装置、机器可读介质及设备
CN115439386B (zh) 图像融合方法、装置、电子设备和存储介质
CN112714246A (zh) 连拍照片获取方法、智能终端及存储介质
CN117710273A (zh) 图像增强模型的构建方法、图像增强方法、设备及介质
CN116916164A (zh) 图像频闪处理方法、装置、电子设备及存储介质
CN115908150A (zh) 图像增强方法、装置、计算机设备及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22845275

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22845275

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.06.2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22845275

Country of ref document: EP

Kind code of ref document: A1