WO2022067761A1 - Image processing method and apparatus, capturing device, movable platform, and computer readable storage medium - Google Patents

Abstract

An image processing method and apparatus, a capturing device (400), a movable platform (500), and a computer readable storage medium. A reference database comprises multiple reference parameters, and reference color temperature lines and white balance statistics areas corresponding to the reference parameters; different reference parameters are set for simulating different environments having different degrees of brightness and darkness that may appear in actual capturing scenes, so as to configure reference data which is applicable to actual environments having different degrees of brightness and darkness, thus, during image processing, the reference data having matched brightness of the capturing scene of the image can be selected for white balance; especially in a low-brightness environment, because corresponding reference data can be selected, the image captured in the low-brightness environment can also achieve a good white balance effect.

Description

Image processing method, apparatus, photographing device, removable platform, and computer-readable storage medium technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, an apparatus, a photographing device, a movable platform, and a computer-readable storage medium.

Background technique

In natural scenes, different light sources exhibit different color temperature characteristics. Color temperature refers to the color radiated by a black body (absolute black body) as the temperature increases after being heated. As the temperature increases, the black body first emits red light, and as the temperature continues to increase, it becomes brighter and brighter to yellow, white, and blue light. For the light source, it is considered that the color of the light emitted by the light source is the same as the color of the light radiated by the black body at a certain temperature, and the black body temperature at this time is called the color temperature of the light source, thus introducing the concept of correlated color temperature. Correlated color temperature mainly refers to the temperature of the most similar black body radiator with the same luminance stimulus.

In the field of human biology, color constancy mainly refers to the perceptual characteristics that people's perception of the color of the surface of the object remains unchanged when the color light irradiated on the surface of the object changes, that is, the visual perception of the color change of the object. invariance. Human beings have a psychological tendency to not change the color judgment of a specific object due to light source or external environmental factors. This tendency is color constancy. However, there is no way for ordinary imaging systems to directly achieve color constancy. In order to achieve the color constancy of the device, the concept of Auto White Balance (AWB) needs to be introduced into the imaging system.

SUMMARY OF THE INVENTION

In view of this, the present application provides an image processing method, an apparatus, a photographing device, a movable platform and a computer-readable storage medium to solve the problem of poor white balance effect in the related art.

In a first aspect, an image processing method is provided, including:

acquiring an image, and determining a target parameter corresponding to the image, where the target parameter is related to the brightness of the shooting scene of the image;

Select a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;

White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In a second aspect, an image processing apparatus is provided, the apparatus includes a processor and a memory, the memory stores instructions, and the processor implements the following operations when executing the instructions:

acquiring an image, and determining a target parameter corresponding to the image, where the target parameter is related to the brightness of the shooting scene of the image;

Select a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;

White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In a third aspect, a shooting device is provided, including:

shell;

a lens assembly, arranged inside the casing;

a sensor assembly disposed inside the housing for sensing light passing through the lens assembly and generating electrical signals; and,

The image processing apparatus according to the second aspect.

In a fourth aspect, a movable platform is provided, including:

body;

a power system mounted within the body for powering the movable platform; and,

The image processing apparatus according to the second aspect.

In a fifth aspect, a computer-readable storage medium is provided, where several computer instructions are stored thereon, and when the computer instructions are executed, the operations of the method of the first aspect are implemented.

Applying the solution provided by this application, the reference database in this solution contains multiple reference parameters, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter; different reference parameters are set to simulate the possible occurrences in the actual shooting scene. Therefore, the reference data suitable for the actual environment with different brightness and darkness can be configured; thus, the reference data that matches the brightness of the shooting scene of the image can be selected for white balance during image processing; especially In a low-brightness environment, since the corresponding reference data can be selected, an image captured in a low-brightness environment can also obtain a good white balance effect.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 is a schematic diagram of images captured by an imaging system under different lighting conditions according to an embodiment of the present application.

FIG. 2A is a flowchart of an image processing method according to an embodiment of the present application.

FIG. 2B is a schematic diagram of a white balance reference data.

FIG. 2C , FIG. 2D and FIG. 2E are a plurality of reference parameters according to an embodiment of the present application, and schematic diagrams of reference color temperature lines and white balance statistical regions corresponding to each reference parameter.

FIG. 2F is a schematic diagram illustrating an offset between a reference color temperature line relative to a preset standard color temperature line based on FIG. 2E in an embodiment of the present application.

FIG. 2G is a schematic diagram of a fusion ratio according to an embodiment of the present application.

FIG. 2H is a schematic flowchart of another image processing method shown in this embodiment.

FIG. 2I is a schematic flowchart of another image processing method shown in this embodiment.

FIG. 3 is a schematic structural diagram of a device for implementing the image processing method of this embodiment.

FIG. 4 is a block diagram of a mobile platform according to an embodiment of the present application.

FIG. 5 is a block diagram of a camera according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

If the light illuminating the object changes, the color temperature of the light also changes, and the color reflected by it also changes. The human visual system has the characteristics of color constancy. In the field of human biology, color constancy mainly refers to the perception that the color of the object surface remains unchanged when the color light irradiated on the surface of the object changes. Characteristics, that is, the invariance of visual perception to the color change of objects. When a specific object has a very different reflection spectrum due to the environment (especially the lighting environment is within a certain range of change), the reflection spectrum of the object will be very different, but the visual recognition system of the human eye can recognize this change, and can judge It is found that the change is caused by the change of the lighting environment. When the light changes within a certain range, the human recognition mechanism will consider the surface color of the object to be constant within this range of change. Example: A red rose looks red on sunny days and red on cloudy days. Another example is a piece of white A4 paper, over the course of the day you would think that the color of the sunlight changes over time while the A4 paper is still white.

For humans, because the constancy of color maintains the human eye's perception of the color of the surface of an object, how can an imaging system achieve color constancy similar to that of humans? In fact, there is no way for ordinary imaging systems to directly achieve color constancy. In different light environments, the image acquisition device will appear color reproduction distortion, that is, the color of the image is either reddish or bluish. For example, in a light environment with a low color temperature, the color of the image output by the camera is reddish; in a light environment with a high color temperature, the color of the image output by the camera is blue. As shown in Figure 1, it is a schematic diagram of the image captured by the imaging system under different lighting conditions, where the inconstancy of color is reflected.

In order to achieve the color constancy of the device, the concept of Auto White Balance (AWB) needs to be introduced into the imaging system. White balance is the process of restoring the "white" in the scene to "white" in the imaging system, that is, the neutral color blocks such as white blocks and gray blocks in the image reach the state of R=G=B. The white balance in this state is equivalent to the CCT process in the perception environment, and the effect of the color matrix under the CCT is equivalent to the restoration of various surface colors in the environment and the color constancy of human perception. At this time, the resulting image obtained by the imaging system is close to the perceptual constancy of the human eye.

There are many kinds of white balance algorithms. The current white balance algorithms mainly include: maximum brightness method (Bright Surface First), gray world method (Gray World), improved gray world method, color gamut limit method, light source prediction method, etc. The sensor data of the scene is used to calculate the gain values of the R, G, and B channels of AWB (Automatic white balance) and the CCT (Correlated color temperature) value of the current scene. In ordinary scenarios, these algorithms can play a relatively good effect.

However, in a low-illumination scene, the sensor needs to use a larger gain to perceive the image. In this state, there will be more bright and dark dead pixels, and the output signal-to-noise ratio of the sensor will also become worse. The sensor may still be unable to use a larger gain. The signal of the image exhibits normal luminance values. The image is underexposed, and the bright and dark dead pixels and low signal-to-noise ratio of the image cause the white point in the scene to drift on the image, so that the ordinary white balance algorithm cannot identify the white point, and thus cannot. The calculation result of the white balance is obtained, which results in that the image is prone to appear greenish, purpleish, etc., resulting in a poor experience for the user.

Based on this, the present application provides an image processing solution, in which the reference database contains multiple reference parameters, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter; different reference parameters are set for the purpose of simulating Various environments with different brightness and darkness levels that may appear in the actual shooting scene, so that the reference data suitable for the actual environment with different brightness and darkness levels can be configured; thus, the reference data that matches the brightness of the shooting scene of the image can be selected during image processing. Perform white balance; especially in a low-brightness environment, since the corresponding reference data can be selected, the images captured in a low-brightness environment can also obtain a good white balance effect. In this application, the color temperature line can represent the corresponding Planck line of the device under different brightness environments.

The solution of this embodiment can be applied to photographing devices such as cameras or video cameras; it can also be applied to electronic devices equipped with cameras, and the electronic devices here may include devices such as movable platforms or smart phones.

Among them, the camera has a built-in ISP (Image Signal Processing) unit, which is mainly used to process the output signal of the front-end image sensor. The ISP completes the effect processing of the digital image through a series of digital image processing algorithms. Including 3A (auto exposure, auto focus, auto white balance), dead pixel correction, denoising, glare suppression, backlight compensation, color enhancement, lens shading correction, etc. The solution of this embodiment can be applied to an ISP unit in a camera to realize automatic white balance processing of an image. The image for white balance processing in this embodiment may be the original image raw collected by the built-in image sensor of the photographing device, or may be the image generated by the ISP unit in the process of processing the image, such as a YUV or RGB image.

In other examples, the solution of this embodiment can also be applied to image processing software, which can run on tablet computers, smart phones, personal digital assistants (PDAs), laptop computers, desktop computers, or media content players, etc. In any electronic device capable of processing image data, the image processing software may apply the image processing method provided in this embodiment to perform white balance processing on a specified image.

Next, the image processing scheme will be described in detail. Please refer to FIG. 2A. FIG. 2A is a flowchart of an image processing method provided by an embodiment of the present application. The method includes the following operations:

In step 202, an image is acquired, and a target parameter corresponding to the image is determined, and the target parameter is related to the brightness of the shooting scene of the image;

In step 204, a reference parameter associated with the target parameter is selected from a preset reference database, and a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter are obtained; wherein the reference parameter is the same as the selected reference parameter. The brightness of the shooting scene of the image is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;

In step 206, white balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In this embodiment, the white point in the image means that the color component of the R channel, the color component of the G channel, and the color component of the B channel of the pixel are equal.

Next, the white balance processing procedure of the image will be described. For the consideration of computational efficiency, this embodiment divides the image into blocks, and each block contains multiple pixels. For the pixels of each block, the brightness values of the R channel of each block are accumulated and averaged, the brightness values of the G channel are accumulated and averaged, and the brightness values of the B channel are accumulated and averaged; then, the G channel is accumulated and averaged. The average value of the R channel is divided by the average value of the R channel to obtain the grayscale gain value Rgain of the R channel (that is, Rgain=Gavg/Ravg), and the average value of the G channel is divided by the average value of the B channel to obtain the grayscale gain value of the B channel Bgain ( That is, Bgain=Gavg/Bavg), the grayscale gain value of the G channel is 1. Alternatively, Rgain=Ravg/Gavg, and Bgain=Bavg/Gavg.

In this embodiment, the image is divided into blocks, and the Rgain and Bgain of the image block are known. Next, it is necessary to determine whether the block is a white point; and specifically, to determine whether each block is a white point, it is determined by the white balance reference data; As shown in FIG. 2B, a schematic diagram of a white balance reference data is shown, the white balance reference data characterizes the color temperature line and the white balance statistical area, and the Rgain and Bgain of each block are used to match the white balance in the white balance reference data. The balance statistics area is compared, if the Rgain and Bgain of the block fall within the white balance statistics area, that is, the block is a white point, and if it is not within the white balance statistics area, the block is not a white point. So far, all the image blocks belonging to the white point are determined. For these image blocks belonging to the white point, an Rgain value is obtained according to the Rgain weighted average of each image block belonging to the white point. According to the Bgain weighted average of each image block belonging to the white point Get a Bgain value; blocks that do not fall within the white balance statistical area are not white points and do not need to be added to the calculation.

Finally, the weighted average Rgain value and Bgain value are applied to the whole image, and the corrected three-channel component value is obtained by multiplying the three-channel component of each pixel with the corresponding gain value. If the above formula for determining gain is G/R=Rgain, G/B=Bgain, then during calibration, the corrected R channel component value is equal to: R times the weighted average Rgain, and the corrected B channel component value is equal to: B times Take the weighted average Bgain; if the above formula for determining the gain is R/G=Rgain, B/G=Bgain, then the corrected R channel component value is equal to: R divided by the weighted average Rgain, the corrected B The channel component value is equal to: B divided by the weighted average of Bgain.

In practical applications, the calculation processes of different white balance algorithms are slightly different. The above embodiments are described by taking the gray-scale world method as an example. In practical applications, other algorithms can also be used, such as the maximum brightness method, the color gamut limit method, the light source method. A prediction method, a perfect reflection method, a dynamic threshold method, a fuzzy logic method, etc., are not limited in this embodiment.

The above-described embodiments are explained by dividing the image into blocks. In practical applications, there can also be other implementation methods, such as the non-blocking method, that is, the processing method of determining whether each pixel in the image is a white point; or the blocking method is also used, but in each White balance processing is performed separately within the block.

In the above embodiment, the white balance reference data is described by taking Rgain and Bgain as an example for the gain of the pixel. In practical applications, other methods can also be used. For example, the two gains of Rgain and Ggain can be used to determine the white balance statistical area. Or use the two gains of Bgain and Ggain to determine the white balance statistical area, or use four-channel gain values, such as R, B, G _R gain, and G _B gain, to determine the white balance statistical area, or It may also be white balance based on other color spaces, which is not limited in this embodiment.

It can be seen from the foregoing analysis that in the process of white balance processing, it is necessary to accurately find out the white point in the image, so the setting of white balance reference data is one of the key factors affecting the effect of white balance processing. In the prior art, fixed white balance reference data is usually set, and the setting method is usually an empirical value. The white balance reference data is usually set in an ideal state (such as ideal brightness) and is suitable for normal scenes. In a scene with low brightness, the white balance of the low-brightness image is performed by the white balance reference data suitable for normal brightness, which obviously cannot obtain a better processing effect.

It can be understood that for shooting equipment, a variety of environments are often faced in actual shooting, and environmental changes may also be faced during a continuous shooting process; based on this, in order to be able to adapt to a variety of shooting scenarios, this embodiment The reference database contains multiple reference parameters, as well as the reference color temperature line and white balance statistical area corresponding to each reference parameter; different reference parameters are set to simulate various environments with different degrees of brightness and darkness that may appear in the actual shooting scene. , so that reference data suitable for actual environments with different brightness and darkness levels can be configured. These reference data may be generated by collecting images of multiple different standard light sources under different reference parameters in advance, and using the collected images. Next, an embodiment will be used to describe the process of how to obtain the data in the reference database.

First, M kinds of standard light sources and N kinds of different reference parameters can be used to collect images with the device; wherein, the standard light sources may refer to the light sources specified by the International Commission on Illumination for lighting in uniform color measurement. According to the standard light sources, the Determine the corresponding color temperature. In the process of collecting images, a standard light source can be placed on a standard gray card, and the device collects images containing the standard gray card, so that images with different color temperatures and different reference parameters can be collected. After that, process the collected images, extract the light source white points of these images, and map them to the Rgain/Bgain space (it can also be Rgain and Ggain, Bgain and Ggain, or four-channel gain value, etc. ), after removing a small number of very special light sources, the distribution of the light sources is used to form an area where the white point is located, that is, the white balance statistics area.

In this embodiment, the reference parameter may be a parameter related to the brightness of the shooting scene, which represents the brightness of the shooting scene of the image. As an example, it may include any one or more of the following: scene brightness parameter, image signal-to-noise ratio, image The exposure parameter of the sensor or the dead pixel parameter of the image sensor. Taking the reference parameter as the scene brightness parameter LV (light value) as an example, the data in the reference database can be generated from images collected under a variety of different standard light sources and under a variety of different brightnesses. In practical applications, other reference parameters can be used to generate data in the reference database.

Of course, other methods can also be used in practical applications, such as empirical values, or can also be set by users; as long as multiple reference parameters can be obtained, as well as the reference color temperature line and white balance statistics area corresponding to each reference parameter That is, this embodiment does not limit it.

So far, the generated reference database contains multiple reference parameters, as well as the reference color temperature line and white balance statistical area corresponding to each reference parameter. Different reference parameters correspond to the reference color temperature line and white balance statistical area suitable for the brightness of different shooting scenes. .

As shown in FIG. 2C , FIG. 2D and FIG. 2E , in FIG. 2C , the scene luminance parameter LV is taken as an example for the reference parameters, and 10 reference parameters are used as an example to show the corresponding reference color temperature line and white balance under the 10 reference parameters. Statistics area, wherein the white balance statistics area refers to the area surrounded by irregular graphics, and the curve in this area is the color temperature line; it can be seen that this embodiment implements a three-dimensional white balance calculation method. The number of reference parameters may be flexibly configured as required during specific implementation, which is not limited in this embodiment. As an example, if the hardware computing power of the device is good, multiple reference parameters can be set to match various different brightness environments faced during actual shooting.

Fig. 2D shows 4 reference data by taking a two-dimensional space as an example, wherein the reference data PreLV located on the top layer, PreLV refers to the reference data in an ideal state, and the specific reference parameters are not shown in Fig. 2D; in some examples , PreLV can also be included in the reference database as a kind of reference data.

Figure 2E shows 4 color temperature lines in the Rgain/Bgain space, which are the color temperature lines corresponding to LV of -2, 3 and 0 respectively, and also includes the color temperature line PreLV under ideal brightness, and the corresponding color temperature lines are not shown in Figure 2E white balance statistics area.

Based on this, when processing an image, white balance can be performed using reference data that is consistent with the brightness of the shooting scene of the image to be processed; specifically, for the image to be processed, the target parameter corresponding to the image is determined; The target parameter selects the reference parameter associated with the target parameter from the reference database, selects the associated reference parameter, and correspondingly selects the reference color temperature line and the white balance statistical area, according to the target parameter, the selected reference parameter and its corresponding The reference color temperature line and the white balance statistical area of are used to perform white balance processing on the image.

It can be seen from the foregoing analysis that there are multiple reference parameters in the reference database, and when performing white balance processing on an image, the target parameters of the image may be the same as or different from the reference parameters in the reference database.

In some examples, the value of the selected reference parameter is the same as the value of the target parameter; in other examples, there are two selected reference parameters; among the plurality of reference parameters, the two reference parameters The value of the parameter has the smallest difference from the value of the target parameter.

Taking the embodiment shown in FIG. 2C as an example, the reference parameters in the reference database include 10 integers between the interval [-2, 7], assuming that the target parameter LV of the image is 1.0, and the value of the target parameter 1.0 is the same as the reference value in the reference database. If the parameter value 1 is the same, the reference parameter 1.0 is selected, so as to obtain the reference color temperature line and the white balance statistical area corresponding to the reference parameter 1.0, and use the reference color temperature line and the white balance statistical area to white balance the image.

Assume that the target parameter LV of the image is 1.2, and the target parameter 1.2 is different from each reference parameter; at this time, the method of selecting the reference parameter can be flexibly configured as needed, for example, one reference parameter with the smallest difference can be selected; this embodiment can also be selected The two with the smallest difference, that is, among the multiple reference parameters, the values of the selected two reference parameters have the smallest difference with the value of the target parameter, for example, the reference parameters 1 and 1 with the smallest value difference from the target parameter 1.2 2. Since two reference parameters are selected, the reference color temperature line and the white balance statistics area corresponding to the two reference references are obtained respectively. When the image is white balanced, the reference color temperature line and white balance statistics corresponding to the two reference references are obtained respectively. area is processed.

In this embodiment, white balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In one embodiment, performing white balance processing on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical region includes:

The first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area. As an example, using the white balance statistical area corresponding to the selected reference parameter, based on the white balance statistical area, white pixels can be found from the image, and then the white balance gain can be calculated by using the found white pixels, and the calculation can be used as needed. The obtained white balance gain, as well as the target parameters, the selected reference parameters and their corresponding reference color temperature lines, further perform white balance.

In some examples, two pieces of reference data are obtained for the aforementioned case of selecting two reference parameters. In order to improve the processing effect, this embodiment uses the two pieces of reference data to perform white balance, and in the processing process, the two pieces of reference data are fused by fusing the two pieces of reference data. the way the data is processed. As an example, the reference color temperature line includes a first reference color temperature line and a second reference color temperature line, and the two reference parameters include a first reference parameter and a second reference parameter. Parameters and their corresponding reference color temperature lines and white balance statistical areas to determine the first target white balance result, including:

Determine the first reference white balance result and the second reference white balance corresponding to the first reference parameter and the second reference parameter based on the white balance statistical regions corresponding to the first reference color temperature line and the second reference color temperature line respectively Balance result; wherein, the first reference white balance result may be determined from the white point found in the image based on the white balance statistical area corresponding to the first reference color temperature line; the second reference white balance result may be based on the second reference color temperature The white balance statistics area corresponding to the line is determined from the white point found in the image.

Using the respective weights of the first reference parameter and the second reference parameter relative to the target parameter, the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result.

Still taking the value of the aforementioned target parameter as 1.2 as an example, the values of the two selected reference parameters are 1 and 2, which are referred to as the first reference parameter and the second reference parameter in this embodiment; The first reference white balance result is determined with reference to the color temperature line and the white balance statistics area; the second reference white balance result is determined by using the reference color temperature line and the white balance statistics area corresponding to the second reference parameter. Finally, the first target white balance result of the image is obtained by fusing the first reference white balance result and the second reference white balance result.

The fusion ratio of the first reference white balance result and the second reference white balance result is determined based on the respective weights of the first reference parameter and the second reference parameter relative to the target parameter. In some examples, the weight is determined based on differences between the first reference parameter and the second reference parameter and the target parameter, respectively.

As an example, the above-mentioned fusion process is shown in the following formula:

AWBGain_tmp=AWBgain0*lvratio+AWBgain1*(1-lvratio);

Among them, AWBGain_tmp represents the first target white balance result, AWBgain0 represents the first reference white balance result, AWBgain1 represents the second reference white balance result, lvratio represents the weight between the first reference parameter and the target parameter, and (1-lvratio) represents the second The weights of the reference parameters and the target parameters actually represent the fusion ratio of AWBgain0 and AWBgain1.

In some examples, lvratio can be calculated by normalizing the difference between the target parameter and the first reference parameter and the second reference parameter, or it can be calculated by using a weighting method; in practical applications, other parameters can also be configured as required. The implementation manner is not limited in this embodiment.

As an example, taking the value of the target parameter as 1.2 as an example, the values of the two selected reference parameters are 1 and 2, the difference between the target parameter 1.2 and the first reference parameter 1 is 0.2, and the target parameter 1.2 and the second reference parameter The difference of 1 is 0.8 of the difference of 2. The smaller the difference, the greater the corresponding weight; the weight of the first reference white balance result corresponding to the first reference parameter can be set to 0.8, and the second reference corresponding to the second reference parameter. The weight of the white balance result can be set to 0.2.

Through the above process, the first target white balance result is obtained, and in some examples, the first target white balance result may be output as the final result.

In other cases, the reference data can be used to find the exact white point, that is, the light source part in the image, and the light source belongs to the brighter area; but there are many other non-light source parts in the image, which are relatively dark, because White point drift may occur. If the first target white balance result is directly used, it may not be possible to perform better color restoration on the non-light source part in the image. Based on this, this embodiment may further process the first target white balance result to obtain a second target white balance result with a better processing effect. As an example, in response to determining the first target white balance result, a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line; wherein the preset standard color temperature line is at The color temperature line obtained under the ideal brightness of the shooting scene, such as the aforementioned ideal brightness PreLV; in this embodiment, the offset between the reference color temperature line and the preset standard color temperature line is used to determine the second target white balance result.

In some examples, the second target white balance result may be determined according to the operational relationship between the first target white balance result and the offset.

2F is used as an example for illustration. On the basis of FIG. 2E, FIG. 2F shows the offset of the reference color temperature line relative to the preset standard color temperature line. If the selected reference parameter LV is -2, that is, FIG. 2F In the reference color temperature line corresponding to lv-2, it can be seen from FIG. 2F that there is an offset between the reference color temperature line corresponding to lv-2 and the preset standard color temperature line.

In some examples, the offset includes a color temperature mapping coefficient. In this embodiment, the offset between the reference color temperature line and the preset standard color temperature line may be realized by using a color temperature mapping coefficient. As an example, it can be seen from the foregoing that the first target white balance result obtained by processing represents the grayscale gain of the image pixel, such as Rgain and Bgain; the color temperature can be determined by Rgain and Bgain, for example, the ratio of Rgain and Bgain falls on the reference color temperature line , the point on the reference color temperature line represents the color temperature; if it does not fall on the reference color temperature line, draw a line with the ratio of Rgain and Bgain as the starting point and perpendicular to the reference color temperature line, the intersection of the line and the reference color temperature line is the color temperature. As shown in FIG. 2F , the points on each color temperature line are the color temperature points. FIG. 2F shows the mapping between the color temperature points on the reference color temperature line corresponding to lv-2 and the corresponding color temperature points on the preset standard color temperature line. Mapping, the color temperature mapping coefficient can be determined according to the mapping relationship between the two.

The foregoing embodiment involves the case of selecting two reference parameters. In this case, it is necessary to fuse the color temperature mapping coefficients corresponding to the two reference parameters respectively. As an example, the first reference white balance result includes a first reference correlated color temperature, the second reference white balance result includes a second reference correlated color temperature, and the second target white balance result includes a target correlated color temperature, according to the reference The offset of the color temperature line relative to the preset standard color temperature line determines the second target white balance result, including:

determining, according to the target correlated color temperature, a first reference color temperature mapping coefficient and a second reference color temperature mapping coefficient corresponding to the first reference parameter and the second reference parameter, respectively;

Using the weights, the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient;

The second target white balance result is determined according to the target color temperature mapping coefficient.

As an example, it is explained by the following formula:

Remap=Remap0*lvratio+Remap1(1-lvratio)

Wherein, Remap represents the target color temperature mapping coefficient obtained by fusion, Remap0 represents the first reference color temperature mapping coefficient, and Remap1 represents the second reference color temperature mapping coefficient; The above formula is only for illustration, and other methods may also be used for fusion in practical applications, which is not limited in this embodiment.

The second target white balance result is determined based on the target color temperature mapping coefficient, for example, the second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient.

The second target white balance result is obtained based on the above processing. In some examples, the second target white balance result may be output as the final result; in other examples, further processing may be performed according to actual shooting requirements.

As an example, in a continuous shooting scene, such as shooting a video, the brightness may change greatly during the video shooting. For example, the user shoots a brighter scene at night, and then shoots a darker scene; The difference between the scenes is large. If only the current scene is considered to white balance the video image, it may cause the color jump of the front and back video pictures to be large and have a sudden feeling. Based on this, in some examples, for one frame of images in the continuous shooting scene, the image processing method may further include:

In response to determining the second target white balance result, the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result.

In this embodiment, for the second target white balance result of one frame of images in the continuous shooting scene, it is also possible to combine the pixel correction results of the historical images for fusion to determine the third target white balance result; since the correction of the historical images is referred to The results are fused, so that the color transition of the video picture in the continuous shooting scene can be smoothed and the sense of abruptness can be reduced.

In some examples, the pixel correction result of the historical image includes: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image.

In other examples, the shooting scene may be a night scene, usually based on the night scene, a night scene light source parameter is preset, and the night scene light source parameter is used to correct the image captured in the night scene, in order to make the image match the night scene and prevent the image The color is more obtrusive. In this embodiment, the preset night scene light source parameters can also be superimposed when the image is white balanced. As an example, if the scene brightness parameter of the image is lower than the set threshold, the image processing method further includes: in response to determining the For the second target white balance result, the second target white balance result is fused with the preset night scene light source parameters to determine the fourth target white balance result. In this embodiment, whether the current scene is a night scene is determined by the scene brightness parameter, wherein the threshold can be set as required. When the scene brightness of the image is lower than the set threshold, it can be determined that the shooting scene is a night scene, which needs to be superimposed The light source parameters of the night scene are fused to avoid the abrupt feeling of the image color.

In some examples, the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset. In some examples, the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset. In some examples, the fusion ratio corresponds to the target parameter, and in practical applications, the fusion ratio can be preset as required, for example, the fusion ratio is set based on different target parameters.

Take the following formula as an example:

AWBGain_t*(1-Blend_Ratio)+AWBGain-Final*Blend_ratio;

Among them, AWBGain_t is the pixel correction result of the historical image or the night light source parameter, AWBGain-Final is the second target white balance result, (1-Blend_Ratio) is the fusion ratio of AWBGain_t, and Blend_ratio is the fusion ratio of AWBGain-Final. As shown in FIG. 2G , a schematic diagram of a blending ratio (Blend_ratio) is shown. In this embodiment, corresponding blending ratios are preset for different reference parameters.

Next, the image processing scheme will be described through an embodiment. As shown in FIG. 2H , it is a schematic flowchart of another image processing method shown in this embodiment, including the following operations:

Step 211, the image sensor outputs RAW data;

In other examples, it can also be an image generated by the ISP unit in the process of processing the image.

Step 212, calculating the target parameters of the image through the RAW data;

The target parameter includes any one or more of the following: scene brightness parameter, image signal-to-noise ratio, image sensor exposure parameter or image sensor dead pixel parameter; this embodiment takes the scene brightness parameter LV value as an example for description.

The scene brightness parameter of the image may be determined by the exposure parameter of the image sensor and the average brightness of the image.

As an example, the following calculations can be used:

Among them, luma is the weighted average brightness of the raw image, again, dgain and shutter are the exposure parameters of the AE module, fnum is the aperture size of the raw data, k and b are constants, for example, k is 0.3 and b is 5.1.

Step 213, AWB calculation of the three-dimensional reference data.

In step 214, the calculated AWB result is fused with the night light source parameter or the correction result of the historical image.

Step 215: Transfer the final AWB result obtained by fusion to the ISP system.

Wherein, for step 213, as shown in Figure 2I, the following operations are included:

Step 2131, obtain the image and the target parameter LV of the image;

The first reference parameter LV Index0 and the second reference parameter LV Index1 that are associated with the target parameter LV of the image are selected.

Step 2132, obtain LV Index0 and LV Index1;

The corresponding color temperature curve and white balance statistical area are extracted through LV Index0 and LV Index1.

Step 2133, input the color temperature curve and the white balance statistical area corresponding to LV Index0 and LV Index1 respectively as the condition for the AWB algorithm to detect the white point, and perform the AWB calculation respectively;

Calculate the first reference white balance result AWBgain0, the first reference correlated color temperature CCT0, the second reference white balance result AWBgain1 and the second reference correlated color temperature CCT1;

The AWB algorithm can be the gray-scale world method, the maximum brightness method, the improved gray-scale world method, the color gamut boundary method, the frame area segmentation method, the light source prediction method, the perfect reflection method, the dynamic threshold method, and the fuzzy logic method.

Step 2134: Calculate the fusion ratio lvratio;

The calculation process of the fusion ratio lvratio may be: calculating the difference between the target parameter lv and the first reference parameter LV Index0, and calculating the difference between the target reference parameter lv and the second reference parameter LV Index1, and normalizing the calculated two differences The normalization can also be calculated by adopting a weighted method. That is, lvratio represents the weight of the target parameter lv from the first reference parameter LV Index0 (upper boundary), and (1-lvratio) represents the weight of the target parameter lv from the first reference parameter LV Index1 (lower boundary).

Step 2135: Calculate the fused target correlated color temperature CCT-final and the first target white balance result AWBGain_tmp according to lvratio;

Wherein CCT-final=CCT0*lvratio+CCT1*(1-lvratio), AWBGain_tmp=AWBGain0*lvratio+AWBgain1*(1-lvratio).

Step 2136: Calculate the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient;

As an example, according to the position of CCT-final on the reference color temperature line corresponding to LVIndex0, determine the first reference color temperature mapping coefficient Remap0 (rratio, bratio);

According to the position of CCT-final on the reference color temperature line corresponding to LVIndex1, the second reference color temperature mapping coefficient Remap1 (rratio, bratio) is determined.

Step 2137, calculate the target color temperature mapping coefficient Remap;

Calculate the final color temperature mapping coefficient Remap(rratio,bratio) according to lvratio:

Remap=Remap0*lvratio+Remap1(1-lvratio).

Step 2138, calculate and obtain the second target white balance result AWBGain-final;

AWBGain_tmp is mapped by the color temperature mapping coefficient, and the mapping method can be AWBGain-final=AWBGain_tmp*Remap.

The fusion process for step 215AWB and night light source parameters or pixel correction results of historical images may be:

According to the target parameter lv, the corresponding Blend_Ratio is obtained in Figure 2G;

To calculate AWBGain, AWBGain is equal to AWBGain_t in time domain (night light source parameter or pixel correction result of historical image) times (1-Blend_Ratio) plus AWBGain-Final*Blend_ratio.

In a low illumination scene, using the image processing method of this embodiment to process an image, compared with an image processed by a common white balance algorithm, the color can be better restored, and the white balance processing effect is better.

Taking video shooting in a low-brightness environment as an example, the image processing method of this embodiment can better demonstrate the white balance restoration capability of low-light scenes compared with ordinary white balance algorithms, and provide a more favorable guarantee for the image effect experience of super night scenes .

The foregoing method embodiments may be implemented by software, and may also be implemented by hardware or a combination of software and hardware. Taking software implementation as an example, a device in a logical sense is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation by the image processing processor where it is located. From a hardware perspective, as shown in FIG. 3 , it is a hardware structure diagram of an image processing apparatus 300 for implementing the image processing method of this embodiment. Except for the processor 301 and the memory 302 shown in FIG. 3 , in the embodiment The image processing device used to implement the image processing method generally may also include other hardware according to the actual function of the image processing device, which will not be repeated here.

In this embodiment, the processor 301 implements the following operations when executing the computer program:

acquiring an image, and determining a target parameter corresponding to the image, where the target parameter is related to the brightness of the shooting scene of the image;

Select a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;

White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In some examples, the multiple reference parameters included in the reference database, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter, are obtained by pre-collecting multiple images of different standard light sources under different reference parameters, using generated from the acquired images.

In some examples, the value of the selected reference parameter is the same as the value of the target parameter.

In some examples, two reference parameters are selected; among the plurality of reference parameters, the values of the two reference parameters and the value of the target parameter have the smallest difference.

In some examples, the processor performs white balance processing operations on the image according to the target parameters, the selected reference parameters and their corresponding reference color temperature lines and white balance statistical regions, including:

The first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.

In some examples, the reference color temperature line includes a first reference color temperature line and a second reference color temperature line, the two reference parameters include a first reference parameter and a second reference parameter, and the processor processes the target parameters according to the target parameters. , the selected reference parameter and its corresponding reference color temperature line and white balance statistical area, determine the operation of the first target white balance result, including:

Determine the first reference white balance result and the second reference white balance corresponding to the first reference parameter and the second reference parameter based on the white balance statistical regions corresponding to the first reference color temperature line and the second reference color temperature line respectively balance result;

Using the respective weights of the first reference parameter and the second reference parameter relative to the target parameter, the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result.

In some examples, the weight is determined based on differences between the first reference parameter and the second reference parameter and the target parameter, respectively.

In some examples, the processor, when executing the instructions, further implements the following operations:

In response to determining the first target white balance result, a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line.

In some examples, the processor performs the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including:

A second target white balance result is determined according to the operational relationship between the first target white balance result and the offset.

In some examples, the offset includes a color temperature mapping coefficient.

In some examples, the first reference white balance result includes a first reference correlated color temperature, the second reference white balance result includes a second reference correlated color temperature, the second target white balance result includes a target correlated color temperature, and the The processor performs the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including:

determining, according to the target correlated color temperature, a first reference color temperature mapping coefficient and a second reference color temperature mapping coefficient corresponding to the first reference parameter and the second reference parameter, respectively;

Using the weights, the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient;

The second target white balance result is determined according to the target color temperature mapping coefficient.

In some examples, the processor performing the operation of determining the second target white balance result according to the target color temperature mapping coefficient includes:

The second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient.

In some examples, the image is one frame of images in a continuous shooting scene, and the processor further implements the following operations when executing the instruction:

In response to determining the second target white balance result, the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result.

In some examples, the pixel correction result of the historical image includes: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image.

In some examples, if the scene brightness parameter of the image is lower than the set threshold, the processor further performs the following operations:

In response to determining the second target white balance result, the second target white balance result is fused with preset night scene light source parameters to determine a fourth target white balance result.

In some examples, the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset.

In some examples, the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset.

In some examples, the fusion ratio corresponds to a target parameter.

In some examples, the target parameters include any one or more of the following: a scene brightness parameter, a signal-to-noise ratio of an image, an exposure parameter of the image sensor, or a dead pixel parameter of the image sensor.

In some examples, the scene brightness parameter of the image is determined by the exposure parameter of the image sensor and the average brightness of the image.

As shown in FIG. 4 , an embodiment of the application further provides a photographing device 400 , including: a casing 401 ; a lens assembly 402 , which is arranged inside the casing 401 ; and a sensor assembly 403 , which is arranged inside the casing 401 for sensing Pass the light of the lens assembly 402 and generate an electrical signal; and the image processing apparatus 300 according to any one of the embodiments.

As shown in FIG. 5 , an embodiment of the present application further provides a movable platform 500, including: a body 501; a power system 502 installed in the body 501 and used to provide power for the movable platform; and any The image processing apparatus 300 described in the embodiment.

Optionally, the movable platform 500 is a vehicle, a drone or a mobile robot.

The embodiments of this specification further provide a computer-readable storage medium, where several computer instructions are stored on the readable storage medium, and when the computer instructions are executed, the operations of the image processing method described in any one of the embodiments are performed.

Embodiments of the present specification may take the form of a computer program product embodied on one or more storage media having program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like. Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

For the apparatus embodiments, since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. The terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also other not expressly listed elements, or also include elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The methods and devices provided by the embodiments of the present invention have been described in detail above. The principles and implementations of the present invention are described in this paper by using specific examples. At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. To sum up, the content of this description should not be construed as a limitation to the present invention. .

Claims (44)

An image processing method, comprising: acquiring an image, and determining a target parameter corresponding to the image, where the target parameter is related to the brightness of the shooting scene of the image; Select a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter; White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area. The method according to claim 1, wherein the multiple reference parameters contained in the reference database, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter, are obtained by pre-collecting multiple different standard light sources in the Images under different reference parameters are generated using the acquired images. The method according to claim 2, wherein the value of the selected reference parameter is the same as the value of the target parameter. The method according to claim 2, wherein there are two selected reference parameters; among the plurality of reference parameters, the difference between the values of the two reference parameters and the value of the target parameter is the smallest. The method according to claim 1, wherein the performing white balance processing on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area, comprising: The first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area. The method according to claim 5, wherein the reference color temperature line includes a first reference color temperature line and a second reference color temperature line, the two reference parameters include a first reference parameter and a second reference parameter, and the According to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area, determine the first target white balance result, including: Determine the first reference white balance result and the second reference white balance corresponding to the first reference parameter and the second reference parameter based on the white balance statistical regions corresponding to the first reference color temperature line and the second reference color temperature line respectively balance result; Using the respective weights of the first reference parameter and the second reference parameter relative to the target parameter, the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result. The method according to claim 6, wherein the weight is determined based on the difference between the first reference parameter and the second reference parameter and the target parameter, respectively. The method according to claim 5 or 6, wherein the method further comprises: In response to determining the first target white balance result, a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line. The method according to claim 8, wherein the determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line comprises: A second target white balance result is determined according to the operational relationship between the first target white balance result and the offset. The method of claim 8, wherein the offset comprises a color temperature mapping coefficient. The method according to claim 10, wherein the first reference white balance result comprises a first reference correlated color temperature, the second reference white balance result comprises a second reference correlated color temperature, and the second target white balance The result includes the target correlated color temperature; and determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including: According to the target correlated color temperature, determine the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient corresponding to the first reference parameter and the second reference parameter respectively; Using the weights, the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient; The second target white balance result is determined according to the target color temperature mapping coefficient. The method according to claim 11, wherein the determining the second target white balance result according to the target color temperature mapping coefficient comprises: The second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient. The method according to claim 6, wherein the image is one of the images in a continuous shooting scene, and the method further comprises: In response to determining the second target white balance result, the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result. The method according to claim 13, wherein the pixel correction result of the historical image comprises: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image. The method according to claim 6, wherein if the scene brightness parameter of the image is lower than a set threshold, the method further comprises: In response to determining the second target white balance result, the second target white balance result is fused with preset night scene light source parameters to determine a fourth target white balance result. The method according to claim 13, wherein the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset. The method according to claim 16, wherein the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset. The method according to claim 16 or 17, wherein the fusion ratio corresponds to the target parameter. The method according to claim 1, wherein the target parameter comprises any one or more of the following: scene brightness parameter, image signal-to-noise ratio, image sensor exposure parameter or image sensor dead pixel parameter. The method according to claim 19, wherein the scene brightness parameter of the image is determined by the exposure parameter of the image sensor and the average brightness of the image. An image processing device, characterized in that the device comprises a processor and a memory, the memory stores instructions, and the processor implements the following operations when executing the instructions: acquiring an image, and determining a target parameter corresponding to the image, where the target parameter is related to the brightness of the shooting scene of the image; Select a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter; White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area. The device according to claim 21, wherein the plurality of reference parameters contained in the reference database, and the reference color temperature line and the white balance statistical area corresponding to each reference parameter are obtained by pre-collecting a plurality of different standard light sources in the Images under different reference parameters are generated using the acquired images. The apparatus according to claim 22, wherein the value of the selected reference parameter is the same as the value of the target parameter. The device according to claim 22, wherein there are two selected reference parameters; among the plurality of reference parameters, the difference between the values of the two reference parameters and the value of the target parameter is the smallest. The device according to claim 21, wherein the processor performs white balance processing on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and a white balance statistical area. operations, including: The first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area. The device according to claim 25, wherein the reference color temperature line includes a first reference color temperature line and a second reference color temperature line, the two reference parameters include a first reference parameter and a second reference parameter, and the The processor processes the operation of determining the first target white balance result according to the target parameters, the selected reference parameters and their corresponding reference color temperature lines and white balance statistical regions, including: Determine the first reference white balance result and the second reference white balance corresponding to the first reference parameter and the second reference parameter based on the white balance statistical regions corresponding to the first reference color temperature line and the second reference color temperature line respectively balance result; Using the respective weights of the first reference parameter and the second reference parameter relative to the target parameter, the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result. The apparatus according to claim 26, wherein the weight is determined based on the difference between the first reference parameter and the second reference parameter and the target parameter, respectively. The apparatus according to claim 25 or 26, wherein the processor further implements the following operations when executing the instructions: In response to determining the first target white balance result, a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line. The device according to claim 28, wherein the processor performing the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line comprises: A second target white balance result is determined according to the operational relationship between the first target white balance result and the offset. The apparatus of claim 28, wherein the offset comprises a color temperature mapping coefficient. The apparatus of claim 30, wherein the first reference white balance result comprises a first reference correlated color temperature, the second reference white balance result comprises a second reference correlated color temperature, and the second target white balance The result includes the target correlated color temperature, and the processor performs the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including: determining, according to the target correlated color temperature, a first reference color temperature mapping coefficient and a second reference color temperature mapping coefficient corresponding to the first reference parameter and the second reference parameter, respectively; Using the weights, the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient; The second target white balance result is determined according to the target color temperature mapping coefficient. The apparatus according to claim 31, wherein the processor performing the operation of determining the second target white balance result according to the target color temperature mapping coefficient comprises: The second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient. The device according to claim 26, wherein the image is one of the frames of images in a continuous shooting scene, and the processor further implements the following operations when executing the instruction: In response to determining the second target white balance result, the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result. The device according to claim 33, wherein the pixel correction result of the historical image comprises: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image. The device according to claim 26, wherein, if the scene brightness parameter of the image is lower than a set threshold, the processor also performs the following operations: In response to determining the second target white balance result, the second target white balance result is fused with preset night scene light source parameters to determine a fourth target white balance result. The apparatus according to claim 33, wherein the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset. The device according to claim 36, wherein the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset. The apparatus according to claim 36 or 37, wherein the fusion ratio corresponds to a target parameter. The apparatus according to claim 21, wherein the target parameters include any one or more of the following: a scene brightness parameter, an image signal-to-noise ratio, an image sensor exposure parameter or an image sensor dead pixel parameter. The apparatus according to claim 39, wherein the scene brightness parameter of the image is determined by the exposure parameter of the image sensor and the average brightness of the image. A photographing device, comprising: shell; a lens assembly, arranged inside the casing; a sensor assembly disposed inside the housing for sensing light passing through the lens assembly and generating electrical signals; and, The image processing apparatus according to any one of claims 21 to 40. A movable platform, characterized in that, comprising: body; a power system mounted within the body for powering the movable platform; and, The image processing apparatus according to any one of claims 21 to 40. The movable platform according to claim 41, wherein the movable platform is a vehicle, a drone or a mobile robot. A computer-readable storage medium, characterized in that, several computer instructions are stored on the readable storage medium, and when the computer instructions are executed, the operation of the method according to any one of claims 1 to 20 is implemented.

Images