WO2021093528A1 - Focusing method and apparatus, and electronic device and computer readable storage medium - Google Patents

Abstract

A focusing method and apparatus, and an electronic device and a computer readable storage medium. The focusing method comprises: obtaining a phase difference during photographing, the phase difference comprising a phase difference in a first direction and a phase difference in a second direction, and the first direction and the second direction forming a preset included angle; obtaining gyroscope data by means of a gyroscope; determining a target phase difference from the phase difference in the first direction and the phase difference in the second direction according to the gyroscope data; and focusing based on the target phase difference.

Description

Focusing method and device, electronic equipment, and computer readable storage medium

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911101409.1, and the invention title is "focusing method and device, electronic equipment, computer-readable storage medium" on November 12, 2019, and the entire content of it is approved The reference is incorporated in this application.

Technical field

This application relates to the field of image processing technology, and in particular to a focusing method and device, electronic equipment, and computer-readable storage medium.

Background technique

With the development of imaging technology, people are becoming more and more accustomed to taking images or videos and recording various information through image acquisition devices such as cameras on electronic devices. The camera generally needs to focus on the object being photographed in the process of collecting images, so as to obtain a clear image of the object being photographed.

Traditional focusing methods include Phase Detection Auto Focus (PDAF), which acquires the phase difference value for focusing. However, the traditional focusing method has the problem of inaccurate focusing.

Summary of the invention

According to various embodiments of the present application, a focusing method, device, electronic device, and computer-readable storage medium are provided.

A focusing method, applied to an electronic device including a gyroscope, includes:

Acquiring a phase difference value during shooting, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset angle;

Obtain gyroscope data through the gyroscope;

Determining a target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data;

Focusing is performed based on the target phase difference value.

A focusing device applied to electronic equipment including a gyroscope, including:

The phase difference value acquisition module is used to acquire the phase difference value during shooting. The phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; Set angle

A gyroscope data acquisition module, configured to acquire gyroscope data through the gyroscope;

A target phase difference value determining module, configured to determine a target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data;

The focusing module is used for focusing based on the target phase difference value.

An electronic device includes a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the processor causes the processor to perform the operation of the above-mentioned focusing method.

A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the operation of the above-mentioned method is realized.

The above-mentioned focusing method and device, electronic equipment, and computer-readable storage medium acquire a phase difference value during shooting. The phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction are Preset angle; obtain gyroscope data through gyroscope; according to gyroscope data, the moving direction of the electronic device can be determined, so that a more accurate target phase can be determined from the phase difference value in the first direction and the phase difference value in the second direction Difference; based on the target phase difference value, it can focus more accurately.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Figure 1 is a schematic diagram of the principle of phase detection autofocus.

FIG. 2 is a schematic diagram of a part of the structure of an image sensor in an embodiment.

FIG. 3 is a schematic diagram of the structure of pixels in an embodiment.

Fig. 4 is a schematic structural diagram of an imaging device in an embodiment.

Fig. 5 is a schematic diagram of a filter set on a pixel point group in an embodiment.

Fig. 6 is a flowchart of a focusing method in an embodiment.

FIG. 7 is a flowchart of determining the target phase difference value in an embodiment.

Fig. 8 is a flowchart of focusing in an embodiment.

Fig. 9 is a flowchart of determining the phase difference value in an embodiment.

Fig. 10 is a schematic diagram of a pixel point group in an embodiment.

Fig. 11 is a flowchart of a focusing method in another embodiment.

Fig. 12 is a structural block diagram of a focusing device in an embodiment.

Fig. 13 is a schematic diagram of the internal structure of an electronic device in an embodiment.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

It can be understood that the terms "first", "second", etc. used in this application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first direction may be referred to as the second direction, and similarly, the second direction may be referred to as the first direction. Both the first direction and the second direction are directions, but they are not the same direction.

Figure 1 is a schematic diagram of the principle of phase detection auto focus (PDAF). As shown in FIG. 1, M1 is the position of the image sensor when the imaging device included in the electronic device is in the in-focus state, where the in-focus state refers to the state of successful focusing. When the image sensor is at the M1 position, the imaging light g reflected by the object W toward the lens Lens in different directions converges on the image sensor, that is, the imaging light g reflected by the object W toward the lens Lens in different directions is in the image The image is imaged at the same position on the sensor. At this time, the image of the image sensor is clear.

M2 and M3 are the possible positions of the image sensor when the imaging device is not in focus. As shown in Figure 1, when the image sensor is at the M2 position or the M3 position, the object W is reflected in different directions of the lens Lens. The imaging light g will be imaged at different positions. Please refer to Figure 1. When the image sensor is at the M2 position, the imaging light g reflected by the object W in different directions to the lens Lens is imaged at the position A and the position B respectively. When the image sensor is at the M3 position, the object W is reflected toward the lens. The imaging light g in different directions of the lens Lens is imaged at the position C and the position D respectively. At this time, the image of the image sensor is not clear.

In PDAF technology, the difference in position of the image formed by the imaging light entering the lens from different directions in the image sensor can be obtained. For example, as shown in Figure 1, the difference between position A and position B can be obtained, or, Obtain the difference between position C and position D; after obtaining the difference in position of the image formed by the imaging light entering the lens from different directions in the image sensor, the difference and the difference between the lens and the image sensor in the camera The geometric relationship is used to obtain the defocus distance. The so-called defocus distance refers to the distance between the current position of the image sensor and the position where the image sensor should be in the in-focus state; the imaging device can focus according to the obtained defocus distance.

It can be seen from this that when focusing, the calculated PD value (phase difference value) is 0, and conversely, the larger the calculated value, the farther the position of the clutch focus is, and the smaller the value, the closer the clutch focus. When using PDAF to focus, by calculating the PD value, and then obtaining the corresponding relationship between the PD value and the defocusing distance according to the calibration, the defocusing distance can be obtained, and then controlling the lens to move to the focal point according to the defocusing distance to achieve focusing.

In one embodiment, the present application provides an imaging assembly. The imaging component includes an image sensor. The image sensor may be a metal oxide semiconductor device (English: Complementary Metal Oxide Semiconductor; abbreviation: CMOS) image sensor, a charge-coupled device (English: Charge-coupled Device; abbreviation: CCD), a quantum thin film sensor, or an organic sensor.

Fig. 2 is a schematic diagram of a part of the image sensor in an embodiment. The image sensor includes a plurality of pixel point groups Z arranged in an array, each pixel point group Z includes a plurality of pixel points D arranged in an array, and each pixel point D corresponds to a photosensitive unit. The multiple pixels include M*N pixels, where both M and N are natural numbers greater than or equal to 2. Each pixel point D includes a plurality of sub-pixel points d arranged in an array. That is, each photosensitive unit can be composed of a plurality of photosensitive elements arranged in an array. Among them, the photosensitive element is an element that can convert light signals into electrical signals. Referring to FIG. 3, a plurality of sub-pixel points d arranged in an array in each pixel point D collectively cover a microlens W. In one embodiment, the photosensitive element may be a photodiode. In this embodiment, each pixel point group Z includes 4 pixel points D arranged in a 2*2 array, and each pixel point D may include 4 sub-pixel points d arranged in a 2*2 array. The four sub-pixel points d jointly cover a microlens W. Among them, each pixel point D includes 2*2 photodiodes, and the 2*2 photodiodes are arranged correspondingly to the 4 sub-pixel points d arranged in a 2*2 array. Each photodiode is used to receive optical signals and perform photoelectric conversion, thereby converting the optical signals into electrical signals for output. The 4 sub-pixels d included in each pixel D are set corresponding to the same color filter, so each pixel D corresponds to a color channel, such as the red R channel, or the green channel G, or the blue channel B .

As shown in Figure 3, taking each pixel D including sub-pixel 1, sub-pixel 2, sub-pixel 3, and sub-pixel 4 as an example, the signals of sub-pixel 1 and sub-pixel 2 can be combined and output, and sub-pixel The signals of point 3 and sub-pixel point 4 are combined and output, thereby constructing two PD pixel pairs along the second direction (ie the vertical direction). According to the phase value of the two PD pixel pairs, each sub-pixel point in pixel point D can be determined The PD value (phase difference value) in the second direction. The signals of sub-pixel point 1 and sub-pixel point 3 are combined and output, and the signals of sub-pixel point 2 and sub-pixel point 4 are combined and output, thereby constructing two PD pixel pairs along the first direction (ie, horizontal direction). According to the two PD pixel pairs The phase value of can determine the PD value (phase difference value) of each sub-pixel point in the pixel point D along the first direction.

Fig. 4 is a schematic structural diagram of an imaging device in an embodiment. As shown in FIG. 4, the imaging device includes a lens 40, a filter 42 and an imaging component 44. The lens 40, the filter 42 and the imaging component 44 are sequentially located on the incident light path, that is, the lens 40 is disposed on the filter 42, and the filter 42 is disposed on the imaging component 44.

The imaging component 44 includes the image sensor in FIG. 2. The image sensor includes a plurality of pixel point groups Z arranged in an array. Each pixel point group Z includes a plurality of pixel points D arranged in an array. Each pixel point D corresponds to a photosensitive unit, and each photosensitive unit can consist of multiple arrays. It is composed of arranged photosensitive elements. In this embodiment, each pixel point D includes 4 sub-pixel points d arranged in a 2*2 array, and each sub-pixel point d corresponds to a light spot diode 442, that is, 2*2 photodiodes 442 and a 2*2 array array. The 4 sub-pixel points d of the cloth are correspondingly arranged. The 4 sub-pixel points d share one lens.

The filter 42 may include three types of red, green, and blue, and can only transmit light of corresponding wavelengths of red, green, and blue, respectively. The four sub-pixel points d included in one pixel point D are arranged corresponding to the filters of the same color. In other embodiments, the filter may also be white, which facilitates the passage of light in a larger spectrum (wavelength) range and increases the luminous flux passing through the white filter.

The lens 40 is used to receive incident light and transmit the incident light to the filter 42. After the filter 42 performs filtering processing on the incident light, the filtered light signal is projected onto the imaging component 44.

The photosensitive unit in the image sensor included in the imaging component 44 converts the light incident from the filter 42 into a charge signal through the photoelectric effect, and generates a pixel signal consistent with the charge signal, and finally outputs an image after a series of processing.

It can be seen from the above description that the pixels included in the image sensor and the pixels included in the image are two different concepts. The pixels included in the image refer to the smallest component unit of the image, which is generally represented by a sequence of numbers. In the following, the sequence of numbers can be referred to as the pixel value of a pixel. The embodiments of the present application involve both concepts of "pixels included in an image sensor" and "pixels included in an image". To facilitate readers' understanding, a brief explanation is provided here.

Fig. 5 is a schematic diagram of a filter set on a pixel point group in an embodiment. The pixel group Z includes 4 pixels D arranged in an array arrangement of two rows and two columns, wherein the color channel of the pixels in the first row and the first column is green, that is, the first row and the first row The filter set on the pixels in one column is a green filter; the color channel of the pixels in the first row and second column is red, that is, the filter set on the pixels in the first row and second column The filter is a red filter; the color channel of the pixel in the second row and the first column is blue, that is, the filter set on the pixel in the second row and the first column is a blue filter; The color channel of the pixel points in the second row and second column is green, that is, the filter set on the pixel points in the second row and second column is a green filter.

Fig. 6 is a flowchart of a focusing method in an embodiment. As shown in FIG. 6, the focusing method includes operations 602 to 608.

Operation 602: Obtain a phase difference value during shooting, where the phase difference value includes the phase difference value in the first direction and the phase difference value in the second direction; the first direction and the second direction form a preset angle.

Specifically, when an image is taken by an imaging device of an electronic device, a phase difference value is acquired, and the phase difference value includes a phase difference value in the first direction and a phase difference value in the second direction. The first direction and the second direction may form a preset included angle, and the preset included angle may be any angle other than 0 degrees, 180 degrees, and 360 degrees.

In operation 604, the gyroscope data is obtained through the gyroscope.

The gyroscope is an angular motion detection device that uses the moment of momentum sensitive shell of a high-speed rotating body to rotate one or two axes orthogonal to the rotation axis relative to the inertial space. Gyroscopes include fiber optic gyroscopes, laser gyroscopes, and MEMS (Micro Electro Mechanical systems) gyroscopes.

The gyroscope data may include angular velocity data, movement direction, and so on. The gyroscope includes X-axis, Y-axis, and Z-axis, and can detect the gyroscope data of the X-axis, the gyroscope data of the Y-axis, and the gyroscope data of the Z-axis respectively.

The gyroscope data of the X-axis of the gyroscope may represent data moving left and right in the horizontal direction, and the gyroscope data of the Y-axis of the gyroscope may represent data moving back and forth in the horizontal direction. Therefore, the horizontal direction data of the electronic device can be determined based on the gyroscope data of the X axis and the gyroscope data of the Y axis of the gyroscope.

For example, when the gyroscope data of the X axis of the gyroscope moves to the left and the gyroscope data of the Y axis is zero, then the horizontal direction of the electronic device can be determined according to the gyroscope data of the X axis of the gyroscope and the gyroscope data of the Y axis. Move left. When the gyroscope data of the X axis of the gyroscope is zero and the gyroscope data of the Y axis is moving forward, the electronic device can be determined to move forward horizontally according to the gyroscope data of the X axis of the gyroscope and the gyroscope data of the Y axis . When the gyroscope data of the X axis of the gyroscope moves to the right and the gyroscope data of the Y axis moves forward, the horizontal direction of the electronic device can be determined according to the gyroscope data of the X axis of the gyroscope and the gyroscope data of the Y axis Move forward right.

In operation 606, the target phase difference value is determined from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data.

It is understandable that when the moving direction in the gyroscope data is the horizontal direction, especially when the object to be photographed contains vertical texture, the image blur of the pixel pairs arranged horizontally (left and right) is used to obtain the horizontal direction. The phase difference value is not accurate, which affects the accuracy of focusing. Similarly, when the moving direction in the gyroscope data is the vertical direction, especially when the object to be photographed contains a horizontal texture, the vertical alignment (arranged up and down) of the pixel pairs will be blurred by the program to obtain the vertical direction. The phase difference value is not accurate, which affects the accuracy of focusing.

In the embodiment of the present application, according to the gyroscope data, an accurate target phase difference value can be determined from the phase difference value in the first direction and the phase difference value in the second direction.

For example, the first direction is the horizontal direction, and the second direction is the vertical direction. When the gyroscope data is moving in the horizontal direction, the phase difference value in the second direction can be determined as the target phase difference value; when the gyroscope data is vertical Moving in the straight direction, the phase difference value in the first direction can be determined as the target phase difference value.

In operation 608, focusing is performed based on the target phase difference value.

Focusing refers to the process of driving the lens through a motor to change the position of the animal distance and distance, so that the image of the object is clear.

In the above focusing method, the phase difference value is acquired during shooting, and the phase difference value includes the phase difference value in the first direction and the phase difference value in the second direction; the first direction and the second direction form a preset angle; the gyroscope is obtained through a gyroscope Data; According to the gyroscope data, the moving direction of the electronic device can be determined, so that a more accurate target phase difference value can be determined from the phase difference value in the first direction and the phase difference value in the second direction; it can be more accurate based on the target phase difference value Focus.

In one embodiment, determining the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data includes: determining the moving direction of the electronic device according to the gyroscope data; In the moving direction, the target phase difference value is determined from the phase difference value in the first direction and the phase difference value in the second direction.

In one embodiment, the direction in the gyroscope data may be used as the moving direction of the electronic device. For example, if the direction in the gyroscope data is moving horizontally to the left, the moving direction of the electronic device is moving horizontally to the left.

In another embodiment, the direction in the gyroscope data can also be weighted to obtain the moving direction of the electronic device. Specifically, the X-axis weighting factor, the Y-axis weighting factor, and the Z-axis weighting factor in the gyroscope data are obtained; according to the gyroscope data of the X-axis, the gyroscope data of the Y-axis, the gyroscope data of the Z-axis, and the corresponding Each weighting factor of determines the direction of movement of the electronic device.

For example, the gyroscope data of the X-axis is 50cm, which means it moves 50cm horizontally to the right; the gyroscope data of the Y-axis is 40cm, which means it moves forward 40cm horizontally; the gyroscope data of the Z-axis is 0; the weighting factor of the X-axis is 1. , The weighting factor of the Y axis is 1.25, and the weighting factor of the Z axis is 1.5. After the weighting process, the gyroscope data of the X axis is 50*1=50cm, and the gyroscope data of the Y axis is 40*1.25=50cm. It is determined that the moving direction of the electronic device is 45 degrees to the right and forward.

In other embodiments, the moving direction of the electronic device may also be determined according to other methods, and the specific method may be set according to the needs of the user, and is not limited to this.

The electronic device contains an image sensor, and the image sensor contains each pixel pair. When the movement direction of the electronic device is determined more accurately, the movement direction of each pixel pair can be determined more accurately, so that the pixel pair that is blurred in the image can be determined , Remove the phase difference value corresponding to the imaged blurred pixel pair, and determine a more accurate target phase difference value.

In one embodiment, as shown in FIG. 7, determining the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the moving direction of the electronic device includes:

In operation 702, when the moving direction of the electronic device is the first direction, it is determined that the phase difference in the second direction is the target phase difference value.

When the moving direction of the electronic device is the first direction, especially when the subject has a texture in the second direction, the pixels arranged in the first direction in the image sensor included in the electronic device blur the image, and the image is blurred in the first direction. The phase difference value obtained by the aligned pixels is not accurate. However, the image of the pixel pair arranged in the second direction is clear, and the phase difference value obtained by the pixel pair arranged in the second direction is accurate, and the phase difference value in the second direction is determined as the target phase difference value.

For example, the first direction is the horizontal direction and the second direction is the vertical direction. When the electronic device moves horizontally, especially when the subject has a vertical texture, the image sensors included in the electronic device are arranged in the horizontal direction. Pixels (arranged left and right) are blurred for imaging, while pixels arranged in a vertical direction (arranged up and down) are sharp for imaging. That is to say, the phase difference value obtained by the pixel pair arranged in the horizontal direction is not accurate, while the phase difference value obtained by the pixel pair arranged in the vertical direction is accurate. Therefore, the phase difference value in the vertical direction is determined as the target phase difference value.

In operation 704, when the moving direction of the electronic device is the second direction, it is determined that the phase difference in the first direction is the target phase difference value.

When the moving direction of the electronic device is the second direction, especially when the subject has a texture in the first direction, the pixels arranged in the second direction in the image sensor included in the electronic device blur the image, and the image is blurred in the second direction. The phase difference value obtained by the aligned pixels is not accurate. However, the image of the pixel pair arranged in the first direction is clear, and the phase difference value obtained by the pixel pair arranged in the first direction is accurate, and the phase difference value in the first direction is determined as the target phase difference value.

For example, the first direction is the vertical direction, and the second direction is the horizontal direction. When the electronic device moves vertically, especially when the subject has a horizontal texture, the image sensor contained in the electronic device uses the vertical direction. Pixels arranged (arranged up and down) blur the image, while pixels arranged in the horizontal direction (arranged left and right) have a clear image. That is to say, the phase difference value obtained by the pixel pair arranged in the vertical direction is not accurate, while the phase difference value obtained by the pixel pair arranged in the horizontal direction is accurate. Therefore, the phase difference value in the horizontal direction is determined as the target phase difference value.

In the above focusing method, according to the moving direction of the electronic device, an accurate phase difference value is determined from the phase difference value in the first direction and the phase difference value in the second direction as the target phase difference value, which avoids performing based on inaccurate phase difference values. Focusing causes the problem of inaccurate focus, which improves the accuracy of focus.

In one embodiment, as shown in FIG. 8, performing focusing based on the target phase difference value includes:

In operation 802, a defocus distance value is determined according to the target phase difference value.

The corresponding relationship between the target phase difference value and the defocus distance value can be obtained through calibration.

The corresponding relationship between the defocus distance value and the target phase difference value is as follows:

defocus=PD*slope(DCC), where DCC (Defocus Conversion Coefficient) is obtained by calibration, and PD is the target phase difference value.

The calibration process of the corresponding relationship between the target phase difference value and the defocus distance value includes: dividing the effective focus stroke of the camera module into 10 equal parts, namely (near focus DAC-far focus DAC)/10, so as to cover the motor Focus range; focus at each focus DAC (DAC can be 0 to 1023) position, and record the phase difference of the current focus DAC position; after completing the motor focus stroke, take a group of 10 focus DACs and the obtained PD value to do Ratio; Generate 10 similar ratios K, and fit the two-dimensional data composed of DAC and PD to get a straight line with slope K.

In operation 804, the lens is controlled to move to focus according to the defocus distance value.

In the above focusing method, the defocus distance value and the moving direction are determined according to the target phase difference value; the lens movement can be controlled according to the defocus distance value and the moving direction to more accurately focus.

In one embodiment, determining the defocus distance value according to the target phase difference value includes: obtaining the confidence level of the target phase difference value; when the confidence level is greater than the confidence threshold, according to the target phase difference value from the phase difference value and the defocus distance The corresponding defocus distance value is determined in the value correspondence relationship.

The confidence level of the target phase difference value refers to the credibility of the target phase difference value. The higher the confidence of the target phase difference value, the more reliable the target phase difference value, that is, the more accurate the target phase difference value; the lower the confidence of the target phase difference value, the less reliable the target phase difference value, that is, the target phase difference. The more inaccurate the value.

When the confidence is greater than the confidence threshold, it means that the confidence of the target phase difference value is higher. It can be considered that the target phase difference value is more accurate. Then the target phase difference value is determined from the corresponding relationship between the phase difference value and the defocus distance value. The corresponding defocus distance value.

When the confidence is less than or equal to the confidence threshold, it means that the confidence of the target phase difference value is low, and it can be considered that the target phase difference value is inaccurate. You can return to the operation of obtaining the phase difference value during shooting and reacquire the phase in the first direction. The difference and the phase difference in the second direction.

In this embodiment, taking the phase difference value in the horizontal direction as an example, the phase difference value of a line coordinate x in the image is calculated, and the left figure is x-2, x-1, x, x+1, x+2, and a total of 5 The brightness value of each pixel is moved on the right picture, and the moving range can be -10 to +10. which is:

Compare the brightness values Rx-12, Rx-11, Rx-10, Rx-9, Rx-8 and x-2, x-1, x, x+1, x+2 in the picture on the right;

Compare the brightness values Rx-11, Rx-10, Rx-9, Rx-8, Rx-7 and x-2, x-1, x, x+1, x+2 in the picture on the right;

...

Compare the brightness values Rx-2, Rx-1, Rx, Rx+1, Rx+2 and x-2, x-1, x, x+1, x+2 in the picture on the right;

Compare the brightness values Rx-1, Rx, Rx+1, Rx+2, Rx+3 and x-2, x-1, x, x+1, x+2 in the picture on the right;

...

Compare the brightness values Rx+7, Rx+8, Rx+9, Rx+10, Rx+11 and x-2, x-1, x, x+1, x+2 in the right picture.

Compare the brightness values Rx+8, Rx+9, Rx+10, Rx+11, Rx+12 and x-2, x-1, x, x+1, x+2 in the picture on the right.

Take the five pixels on the right as Rx-2, Rx-1, Rx, Rx+1, Rx+2, and the five pixels on the left as x-2, x-1, x, x+1, x +2 as an example, the degree of similarity matching can be |Rx-2-x-2|+|Rx-1-x-1|+|Rx-x|+|Rx+1--x+1|+|Rx +2-x+2|. The smaller the value of the similarity matching degree, the higher the similarity. The higher the similarity, the higher the credibility. Similar pixel values can be used as matched pixels to get the phase difference. As for the upper and lower images, the brightness value of a column of pixels in the upper image can be compared with the brightness value of the same number of pixels in the lower image. The process of obtaining the credibility of the upper and lower images is similar to that of the left and right images, so I won't repeat them here.

In the above focusing method, when the confidence of the target phase difference value is greater than the confidence threshold, the target phase difference value is more accurate, and the corresponding relationship can be determined from the phase difference value and the defocus distance value according to the target phase difference value. Defocus distance value, the determined defocus distance value is also more accurate, so that the accuracy of focusing can be improved.

In an embodiment, the above method further includes: acquiring a first image; performing subject detection on the first image to obtain a region of interest. Obtaining the phase difference value during shooting includes: obtaining the phase difference value in the region of interest during shooting. Focusing based on the target phase difference value includes: focusing in the region of interest based on the target phase difference value to obtain the second image.

Among them, subject detection (salient object detection) refers to automatically processing regions of interest when facing a scene and selectively ignoring regions of interest. The area of interest is called the subject area. The visible light image refers to an RGB (Red, Green, Blue) image. A color image can be obtained by shooting any scene with a color camera, that is, an RGB image. The visible light image may be stored locally by the electronic device, may also be stored by other devices, may also be stored on the network, or may be captured by the electronic device in real time, but is not limited to this.

Specifically, the phase difference value of the region of interest in the first image is acquired. The phase difference value of the region of interest also includes the phase difference value in the first direction and the phase difference value in the second direction, the first direction and the second direction Set the preset angle; obtain gyroscope data through the gyroscope; determine the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data; based on the target phase difference value in the region of interest Focus in the middle to get the second image.

The above focusing method, combined with subject detection, acquires the region of interest, and performs focusing in the region of interest, which can avoid focusing in the background area and improve the accuracy of focusing.

In one embodiment, the electronic device includes an image sensor, the image sensor includes a plurality of pixel point groups arranged in an array, and each pixel point group includes M*N pixel points arranged in an array; each pixel point corresponds to a photosensitive unit , Where both M and N are natural numbers greater than or equal to 2.

As shown in Figure 9, obtaining the phase difference value includes:

In operation 902, a target brightness map is obtained according to the brightness values of the pixel points included in each pixel point group.

Generally, the brightness value of the pixel of the image sensor can be characterized by the brightness value of the sub-pixel included in the pixel. The imaging device may obtain the target brightness map according to the brightness values of the sub-pixel points in the pixel points included in each pixel point group. Wherein, the brightness value of a sub-pixel point refers to the brightness value of the light signal received by the photosensitive element corresponding to the sub-pixel point.

As mentioned above, the sub-pixel included in the image sensor is a photosensitive element that can convert light signals into electrical signals. Therefore, the light signal received by the sub-pixel can be obtained according to the electrical signal output by the sub-pixel. Intensity, the brightness value of the sub-pixel can be obtained according to the intensity of the light signal received by the sub-pixel.

The target brightness map in the embodiment of the present application is used to reflect the brightness value of the sub-pixels in the image sensor. The target brightness map may include multiple pixels, wherein the pixel value of each pixel in the target brightness map is based on the image sensor Obtained from the brightness value of the neutron pixel.

Operation 904: perform segmentation processing on the target brightness map to obtain a first segmented brightness map and a second segmented brightness map, and based on the position difference of pixels matching each other in the first segmented brightness map and the second segmented brightness map , Determine the phase difference value of the pixels that match each other.

In one embodiment, the imaging device may perform segmentation processing on the target luminance map along the column direction (the y-axis direction in the image coordinate system). In the process of segmenting the target luminance map along the column direction, Each dividing line of the segmentation process is perpendicular to the direction of the column.

In another embodiment, the imaging device may perform segmentation processing on the target brightness map along the row direction (the x-axis direction in the image coordinate system), and in the process of segmenting the target brightness map along the row direction , Each dividing line of the segmentation process is perpendicular to the direction of the row.

The first segmented brightness map and the second segmented brightness map obtained after the target brightness map is segmented along the column direction can be referred to as the upper image and the lower image, respectively. The first segmented brightness map and the second segmented brightness map obtained after the target brightness map is segmented along the row direction can be called the left image and the right image, respectively.

Among them, "mutually matched pixels" means that the pixel matrix composed of the pixel itself and the surrounding pixels are similar to each other. For example, the pixel a and its surrounding pixels in the first segmented brightness map form a pixel matrix with 3 rows and 3 columns, and the pixel value of the pixel matrix is:

The pixel b and the surrounding pixels in the second segmented brightness map also form a pixel matrix with 3 rows and 3 columns, and the pixel value of the pixel matrix is:

It can be seen from the above that the two matrices are similar, and it can be considered that the pixel a and the pixel b match each other. There are many ways to judge whether the pixel matrix is similar. Usually, the pixel value of each corresponding pixel in the two pixel matrices can be calculated, and then the absolute value of the difference obtained is added, and the result of the addition is used to determine Whether the pixel matrix is similar, that is, if the result of the addition is less than a preset threshold, the pixel matrix is considered to be similar; otherwise, the pixel matrix is considered to be dissimilar.

For example, for the above two pixel matrices with 3 rows and 3 columns, the difference of 1 and 2, the difference of 15 and 15, the difference of 70 and 70, ..., and then the absolute difference Values are added, and the result of the addition is 3. If the result of the addition of 3 is less than the preset threshold, it is considered that the two pixel matrices with 3 rows and 3 columns are similar.

Another way to judge whether the pixel matrix is similar is to use the Sobel convolution kernel calculation method or the high Laplacian calculation method to extract the edge characteristics, and judge whether the pixel matrix is similar by the edge characteristics.

In the embodiments of the present application, "the positional difference of pixels that match each other" refers to the positions of the pixels in the first split brightness map and the positions of the pixels in the second split brightness map among the matched pixels. The difference. As in the above example, the position difference between the pixel a and the pixel b that are matched with each other refers to the difference between the position of the pixel a in the first split brightness map and the position of the pixel b in the second split brightness map.

The pixels that match each other correspond to different images in the image sensor formed by the imaging light entering the lens from different directions. For example, the pixel a in the first split brightness map and the pixel b in the second split brightness map match each other, where the pixel a may correspond to the image formed at position A in FIG. 1, and the pixel b may correspond to The image formed at position B in Figure 1.

Since the matched pixels correspond to the different images in the image sensor formed by the imaging light entering the lens from different directions, the phase difference of the matched pixels can be determined according to the position difference of the matched pixels. .

In operation 906, the phase difference value in the first direction and the phase difference value in the second direction are determined according to the phase difference values of the pixels that match each other.

When the first split brightness map includes even-numbered rows of pixels, the second split brightness map includes odd-numbered rows, pixel a in the first split brightness map and pixel b in the second split brightness map Mutual matching, the phase difference value in the first direction can be determined according to the phase difference of the pixel a and the pixel b that are matched with each other.

When the first split brightness map includes even-numbered columns, the second split brightness map includes odd-numbered columns, pixel a in the first split brightness map and pixel b in the second split brightness map Mutual matching, based on the phase difference between the matched pixel a and the pixel b, the phase difference value in the second direction can be determined.

The brightness value of the pixel points in the above pixel point group obtains the target brightness map. After the target brightness map is divided into two segmented brightness maps, through pixel matching, the phase difference value of the matching pixels can be quickly determined, and the phase difference value of the matching pixels can be quickly determined. The rich phase difference value can improve the accuracy of the phase difference value and improve the accuracy and stability of the focus.

In one embodiment, each pixel point includes a plurality of sub-pixel points arranged in an array, and obtaining the target brightness map according to the brightness value of the pixel points included in each pixel point group includes: for each pixel point group, according to the pixel point The brightness value of the sub-pixel at the same position of each pixel in the group is obtained, and the sub-brightness map corresponding to the pixel group is obtained; the target brightness map is generated according to the sub-brightness map corresponding to each pixel group.

Wherein, the sub-pixel points at the same position of each pixel point refer to the sub-pixel points that are arranged in the same position in each pixel point.

FIG. 10 is a schematic diagram of a pixel point group in an embodiment. As shown in FIG. 10, the pixel point group includes 4 pixels arranged in an array arrangement of two rows and two columns, and the 4 pixels are respectively D1 pixel, D2 pixel, D3 pixel and D4 pixel, where each pixel includes 4 sub-pixels arranged in an array arrangement of two rows and two columns, where the sub-pixels are respectively d11, d12, d13, d14, d21, d22, d23, d24, d31, d32, d33, d34, d41, d42, d43, and d44.

As shown in Figure 10, the sub-pixels d11, d21, d31, and d41 are arranged in the same position in each pixel, and they are all in the first row and first column. The sub-pixels d12, d22, d32, and d42 are in each pixel. The arrangement positions in are the same in the first row and second column, and the sub-pixels d13, d23, d33, and d43 are arranged in the same position in each pixel, and they are all in the second row and first column, and the sub-pixel d14 , D24, d34 and d44 are arranged in the same position in each pixel, and they are all in the second row and second column.

In one embodiment, as shown in FIG. 11, operation 1102 uses a 2*2 PDAF image sensor to capture light information, specifically, obtaining a target brightness map according to the brightness value of each pixel point included in each pixel point group on the image sensor; Perform segmentation processing on the target brightness map to obtain the first split brightness map and the second split brightness map; perform operation 1104, according to the position difference of the pixels matching each other in the first split brightness map and the second split brightness map, Determine the phase difference value of the pixels that match each other; determine the phase difference value in the first direction and the phase difference value in the second direction according to the phase difference values of the pixels that match each other.

Perform operation 1106 to obtain gyroscope data; perform operation 1108 to determine the moving direction of the electronic device according to the gyroscope data; after determining the moving direction of the electronic device, perform operation 1110 to determine the phase difference between the first direction and the second direction Determine the target phase difference value in the phase difference value of. Specifically, when the moving direction of the electronic device is the first direction, the phase difference in the second direction is determined to be the target phase difference value; when the moving direction of the electronic device is the second direction, the phase difference in the first direction is determined to be Target phase difference value. Operation 1112 is performed to perform focusing based on the target phase difference value.

It should be understood that although the various operations in the flowcharts of FIGS. 6 to 9 are displayed in sequence as indicated by the arrows, these operations are not necessarily performed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these operations, and these operations can be executed in other orders. Moreover, at least part of the operations in FIGS. 6-9 may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily executed at the same time, but can be executed at different times. These sub-operations or The execution order of the stages is not necessarily carried out sequentially, but may be executed alternately or alternately with at least a part of other operations or sub-operations or stages of other operations.

Fig. 12 is a structural block diagram of a focusing device according to an embodiment. As shown in FIG. 12, a focusing device 1200 is provided, which is applied to an electronic device including a gyroscope, and includes: a phase difference value acquisition module 1202, a gyroscope data acquisition module 1204, a target phase difference value determination module 1206, and a focus module 1208, of which:

The phase difference value obtaining module 1202 is used to obtain the phase difference value during shooting, the phase difference value includes the phase difference value in the first direction and the phase difference value in the second direction; the first direction and the second direction form a preset angle.

The gyroscope data acquisition module 1204 is used for acquiring gyroscope data through the gyroscope.

The target phase difference value determination module 1206 is configured to determine the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data.

The focusing module 1208 is used for focusing based on the target phase difference value.

The above focusing device obtains the phase difference value during shooting, the phase difference value includes the phase difference value in the first direction and the phase difference value in the second direction; the first direction and the second direction form a preset angle; the gyroscope is obtained through the gyroscope Data; According to the gyroscope data, the moving direction of the electronic device can be determined, so that a more accurate target phase difference value can be determined from the phase difference value in the first direction and the phase difference value in the second direction; it can be more accurate based on the target phase difference value Focus.

In one embodiment, the above-mentioned target phase difference value determining module 1206 is further used to determine the moving direction of the electronic device according to the gyroscope data; according to the moving direction of the electronic device, the phase difference value from the first direction and the phase difference in the second direction Determine the target phase difference value in the value.

In one embodiment, the above-mentioned target phase difference value determining module 1206 is further configured to determine that the phase difference value in the second direction is the target phase difference value when the moving direction of the electronic device is the first direction; when the moving direction of the electronic device is In the second direction, it is determined that the phase difference in the first direction is the target phase difference value.

In an embodiment, the above-mentioned focusing module 1208 is further configured to determine the defocus distance value and the moving direction according to the target phase difference value; and control the movement of the lens to focus according to the defocus distance value and the moving direction.

In an embodiment, the above-mentioned focusing module 1208 is also used to obtain the confidence of the target phase difference value; when the confidence is greater than the confidence threshold, it is determined from the corresponding relationship between the phase difference value and the defocus distance value according to the target phase difference value. The corresponding defocus distance value.

In one embodiment, the above-mentioned focusing device 1200 further includes a subject detection module for acquiring a first image; subject detection is performed on the first image to obtain a region of interest. Obtaining the phase difference value during shooting includes: obtaining the phase difference value in the region of interest during shooting. Focusing based on the target phase difference value includes: focusing in the region of interest based on the target phase difference value to obtain the second image.

In one embodiment, the electronic device includes an image sensor, and the image sensor includes a plurality of pixel point groups arranged in an array, and each pixel point group includes a plurality of pixel points arranged in an array; each pixel point corresponds to a photosensitive unit. The above-mentioned phase difference value acquisition module 1202 is also used to acquire the target brightness map according to the brightness values of the pixels included in each pixel point group; perform segmentation processing on the target brightness map to obtain the first segmented brightness map and the second segmented brightness According to the position difference of the matching pixels in the first split brightness map and the second split brightness map, the phase difference value of the matching pixels is determined; the phase difference value of the matching pixels is determined according to the phase difference value of the matching pixels in the first direction. The phase difference value or the phase difference value in the second direction.

In one embodiment, the above-mentioned phase difference value acquisition module 1202 is further configured to, for each pixel point group, obtain the corresponding pixel point group according to the brightness value of the sub-pixel point at the same position of each pixel point in the pixel point group. Sub-brightness map: Generate the target brightness map according to the sub-brightness map corresponding to each pixel group.

The division of the modules in the above-mentioned focusing device is only used for illustration. In other embodiments, the focusing device can be divided into different modules as required to complete all or part of the functions of the above-mentioned focusing device.

Fig. 13 is a schematic diagram of the internal structure of an electronic device in an embodiment. As shown in FIG. 13, the electronic device includes a processor and a memory connected through a system bus. Among them, the processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by a processor to implement a focusing method provided in the following embodiments. The internal memory provides a cached operating environment for the operating system computer program in the non-volatile storage medium. The electronic device can be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device.

The implementation of each module in the focusing device provided in the embodiment of the present application may be in the form of a computer program. The computer program can be run on a terminal or a server. The program module composed of the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, the operation of the method described in the embodiment of the present application is realized.

The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to perform the operations of the focusing method.

A computer program product containing instructions that, when run on a computer, causes the computer to execute the focusing method.

Any reference to memory, storage, database, or other media used in the embodiments of the present application may include non-volatile and/or volatile memory. Suitable non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which acts as external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims (18)

A focusing method, characterized in that it is applied to an electronic device including a gyroscope, and includes: Acquiring a phase difference value during shooting, where the phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; the first direction and the second direction form a preset angle; Obtain gyroscope data through the gyroscope; Determining a target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data; and Focusing is performed based on the target phase difference value. The method according to claim 1, wherein the determining the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data comprises: Determining the moving direction of the electronic device according to the gyroscope data; and According to the moving direction of the electronic device, a target phase difference value is determined from the phase difference value in the first direction and the phase difference value in the second direction. The method according to claim 2, wherein the determining the target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the moving direction of the electronic device comprises : When the moving direction of the electronic device is the first direction, determining that the phase difference in the second direction is the target phase difference value; and When the moving direction of the electronic device is the second direction, it is determined that the phase difference in the first direction is the target phase difference value. The method according to claim 1, wherein the focusing based on the target phase difference value comprises: Determining the defocus distance value according to the target phase difference value; and The lens is controlled to move to focus according to the defocus distance value. The method according to claim 4, wherein the determining a defocus distance value according to the target phase difference value comprises: Obtaining the confidence level of the target phase difference value; and When the confidence is greater than the confidence threshold, the corresponding defocus distance value is determined from the corresponding relationship between the phase difference value and the defocus distance value according to the target phase difference value. The method according to claim 1, wherein the method further comprises: Get the first image; Subject detection on the first image to obtain a region of interest; The obtaining of the phase difference value during shooting includes: Acquiring the phase difference value in the region of interest during shooting; and The focusing based on the target phase difference value includes: Focusing in the region of interest based on the target phase difference value to obtain a second image. The method according to claim 1, wherein the electronic device comprises an image sensor, the image sensor comprises a plurality of pixel point groups arranged in an array, and each of the pixel point groups comprises M* arranged in an array. N pixels; each pixel corresponds to a photosensitive unit, where M and N are both natural numbers greater than or equal to 2; The obtaining the phase difference value includes: Acquiring a target brightness map according to the brightness value of the pixel points included in each pixel point group; Perform segmentation processing on the target brightness map to obtain a first segmented brightness map and a second segmented brightness map, and pixels matching each other according to the first segmented brightness map and the second segmented brightness map Determine the phase difference value of the mutually matched pixels; and The phase difference value in the first direction and the phase difference value in the second direction are determined according to the phase difference values of the mutually matched pixels. The method according to claim 7, wherein each of the pixel points includes a plurality of sub-pixel points arranged in an array, and the target brightness map is obtained according to the brightness value of the pixel points included in each of the pixel point groups ,include: For each pixel point group, obtain the sub-luminance map corresponding to the pixel point group according to the brightness value of the sub-pixel point at the same position of each pixel point in the pixel point group; and The target brightness map is generated according to the sub-brightness map corresponding to each pixel point group. A focusing device, characterized in that it is applied to an electronic device including a gyroscope, and includes: The phase difference value acquisition module is used to acquire the phase difference value during shooting. The phase difference value includes a phase difference value in a first direction and a phase difference value in a second direction; Set angle A gyroscope data acquisition module, configured to acquire gyroscope data through the gyroscope; A target phase difference value determination module, configured to determine a target phase difference value from the phase difference value in the first direction and the phase difference value in the second direction according to the gyroscope data; and The focusing module is used for focusing based on the target phase difference value. The apparatus according to claim 9, wherein the target phase difference value determining module is further configured to determine the moving direction of the electronic device according to the gyroscope data; and According to the moving direction of the electronic device, a target phase difference value is determined from the phase difference value in the first direction and the phase difference value in the second direction. The apparatus according to claim 10, wherein the target phase difference value determining module is further configured to determine that the phase difference value in the second direction is the target phase difference when the moving direction of the electronic device is the first direction. Value; and When the moving direction of the electronic device is the second direction, it is determined that the phase difference in the first direction is the target phase difference value. The device according to claim 9, wherein the focusing module is further configured to determine a defocus distance value according to the target phase difference value; and The lens is controlled to move to focus according to the defocus distance value. The apparatus according to claim 12, wherein the focusing module is further configured to obtain the confidence level of the target phase difference value; and When the confidence is greater than the confidence threshold, the corresponding defocus distance value is determined from the corresponding relationship between the phase difference value and the defocus distance value according to the target phase difference value. The device according to claim 9, characterized in that the device further comprises a subject detection module for acquiring the first image; Subject detection on the first image to obtain a region of interest; Acquiring the phase difference value in the region of interest during shooting; and Focusing in the region of interest based on the target phase difference value to obtain a second image. The device according to claim 9, wherein the electronic device comprises an image sensor, the image sensor comprises a plurality of pixel point groups arranged in an array, and each of the pixel point groups comprises M* arranged in an array. N pixels; each pixel corresponds to a photosensitive unit, where M and N are both natural numbers greater than or equal to 2; The phase difference value obtaining module is further configured to obtain a target brightness map according to the brightness value of the pixel points included in each pixel point group; Perform segmentation processing on the target brightness map to obtain a first segmented brightness map and a second segmented brightness map, and pixels matching each other according to the first segmented brightness map and the second segmented brightness map Determine the phase difference value of the mutually matched pixels; and The phase difference value in the first direction and the phase difference value in the second direction are determined according to the phase difference values of the mutually matched pixels. The device according to claim 15, wherein each pixel point comprises a plurality of sub-pixel points arranged in an array, and the phase difference value acquisition module is further configured to, for each pixel point group, according to the The brightness value of the sub-pixel at the same position of each pixel in the pixel group, and obtaining the sub-brightness map corresponding to the pixel group; and The target brightness map is generated according to the sub-brightness map corresponding to each pixel point group. An electronic device, comprising a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes any one of claims 1 to 8. The operation of the focus method. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the operation of the method according to any one of claims 1 to 8 when the computer program is executed by a processor.

Images