WO2022000975A1 - Image processing method and apparatus for stereoscopic endoscope, and storage medium - Google Patents

Abstract

Disclosed is an image processing method for a stereoscopic endoscope. The method comprises: acquiring first image data and second image data; acquiring color components in the first image data and the second image data and corresponding target components, and calculating and obtaining a first group of correction parameters and a second group of correction parameters according to the color components and the target components; and correcting, according to the first group of correction parameters, a first target image of the current target object that is acquired by a first camera, and correcting, according to the second group of correction parameters, a second target image of the current target object that is acquired by a second camera. Further disclosed are an image processing apparatus and a computer-readable storage medium. The present application can make different pieces of image data that are acquired by different cameras of a stereoscopic endoscope tend to be consistent, thereby improving the visual comfort of an operator.

Description

Image processing method, device and storage medium for stereoscopic endoscope

This application claims the priority of the Chinese patent application filed on June 30, 2020 with the application number 202010614933.5 and the application name "Image processing method, device and storage medium for stereoscopic endoscope", the entire content of which is approved by Reference is incorporated in this application.

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method of a stereoscopic endoscope, an image processing device, and a computer-readable storage medium.

Background technique

Endoscopes are widely used in medical and industrial fields. For example, in an endoscope system in the medical field, a doctor performs an endoscopic examination on a patient or performs a minimally invasive surgery through an image obtained by an endoscope. In the endoscope system in the industrial field, the observer inspects the interior of various machines, equipment, etc. through the images acquired by the endoscope, without disassembling or destroying the inspected object.

Stereoscopic endoscope, due to the influence of factors such as sensor, lens, light source position, manufacturing and assembly, may cause the image color of the two sensors of the stereoscopic endoscope to be inconsistent, thereby causing visual discomfort to the operator.

Application content

The main purpose of the present application is to provide an image processing method, an image processing device and a computer-readable storage medium for a stereoscopic endoscope, which aims to solve the problem that the image colors of the two sensors of the existing stereoscopic endoscope are inconsistent, causing the operator to Technical issues with visual discomfort.

To achieve the above purpose, the present application provides an image processing method for a stereoscopic endoscope, and the image processing method for a stereoscopic endoscope includes:

Obtain first image data and second image data, the first image data and the second image data are obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope respectively; The color components in the first image data and the second image data and the target components corresponding to the first image data and the second image data, and the first group is calculated according to the color components and the target components Correction parameters and the second group of correction parameters;

The first target image of the current target object obtained by the first camera is corrected according to the first set of correction parameters, and the first target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. Two target images are corrected.

Preferably, the acquiring color components in the first image data and the second image data and the target components corresponding to the first image data and the second image data, and according to the color components and The steps of calculating the target component to obtain the first set of correction parameters and the second set of correction parameters include:

acquiring color components in the first image data and the second image data;

determining the target component according to the color component of the first image data or the color component of the second image data;

calculating the ratio between the target component and the color component of the first image data to obtain the first set of correction parameters;

The ratio between the target component and the color component of the second image data is calculated to obtain the second set of correction parameters.

Preferably, the acquiring color components in the first image data and the second image data and the target components corresponding to the first image data and the second image data, and according to the color components and The steps of calculating the target component to obtain the first set of correction parameters and the second set of correction parameters include:

acquiring color components in the first image data and the second image data;

Determine a first target component corresponding to the first image data according to the color component of the first image data, and determine a second target component corresponding to the second image data according to the color component of the second image data;

According to the first target component and the second target component, calculating the equalization component;

According to the equalization component and the color component of the first image data, a first set of correction parameters is obtained by calculation, and a second set of correction parameters is calculated and obtained according to the equalization component and the components of the second image data.

Preferably, the step of calculating the equalized component according to the first target component and the second target component includes:

Obtain a first preset equalization scale coefficient corresponding to the first image data and a second preset equalization scale coefficient corresponding to the second image data, the first preset equalization parameter and the second preset The sum of the equalization parameters is the default value;

The equalization component is calculated and obtained according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component.

Preferably, the step of calculating the equalization component according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component includes:

multiplying the first preset equalization proportional coefficient by the first target component to obtain a first product;

multiplying the second preset equalization proportional coefficient by the second target component to obtain a second product;

A sum of the first product and the second product is calculated to obtain the equalized component.

Preferably, the image processing method of the stereoscopic endoscope further comprises:

obtaining the preset color components of the predetermined calibration plate;

Calculate the first difference between the first target component and the preset color component of the preset calibration board, and the second difference between the second target component and the preset color component of the preset calibration board. difference;

Comparing the magnitude of the first difference and the second difference, if the first difference is smaller than the second difference, the first preset equalization scale factor is greater than the second preset equalization scale factor ; if the first difference is greater than the second difference, the second preset equalization scale factor is greater than the first preset equalization scale factor; if the first difference is equal to the second difference value, the first preset equalization scale coefficient is equal to the second preset equalization scale coefficient.

Preferably, the first target image of the current target object acquired by the first camera is corrected according to the first set of correction parameters, and the The steps of correcting the second target image of the current target object include:

obtaining the components of the first target image and the components of the second target image;

Calculate the product between the first set of correction parameters and the components of the first target image to obtain the corrected components of the first target image, and calculate the difference between the second set of correction parameters and the components of the second target image The product of , obtains the corrected component of the second target image.

Preferably, the color components are any of RGB values, HSV values, HSI values, CMYK values, HSL values, HSB values, Ycc values, XYZ values, Lab values, and YUV values corresponding to different color models.

In order to achieve the above object, the present application also provides an image processing device, which includes a memory, a processor, and an image processing program for a stereoscopic endoscope that is stored in the memory and can run on the processor. ;

The image processing program of the stereoscopic endoscope, when executed by the processor, implements the steps of the method as described above.

In order to achieve the above object, the present application also provides a computer-readable storage medium, where an image processing program of a stereoscopic endoscope is stored on the computer-readable storage medium, and the image processing program of the stereoscopic endoscope is executed by a processor The steps of realizing the image processing method of the image processing apparatus as described above.

The image processing method, image processing device and computer-readable storage medium of a stereoscopic endoscope provided by the present application, by acquiring first image data and second image data, the first image data and the second image data are respectively represented by The first camera and the second camera of the stereoscopic endoscope are obtained by shooting a preset calibration plate, and then the color components in the first image data and the second image data and the color components in the first image data and the second image data and the first image data and the color components are obtained. The target component corresponding to the second image data, and the first set of correction parameters and the second set of correction parameters are calculated according to the color component and the target component, and finally obtained from the first camera according to the first set of correction parameters The first target image of the current target object is corrected, and the second target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. In this way, different image data obtained by different cameras of the stereoscopic endoscope can be made consistent, thereby improving the visual comfort of the operator.

Description of drawings

1 is a schematic diagram of a hardware structure of an image processing apparatus provided by an embodiment of the present application;

2 is a schematic flowchart of an embodiment of an image processing method for a stereoscopic endoscope of the present application;

FIG. 3 is a schematic diagram of the refinement flow of the first embodiment of step S2 in FIG. 2;

FIG. 4 is a schematic diagram of the refinement process of the second embodiment of step S2 in FIG. 2;

FIG. 5 is a schematic diagram of the refinement process of step S27 in FIG. 4;

FIG. 6 is a schematic diagram of the refinement process of step S272 in FIG. 5;

FIG. 7 is a schematic diagram of the refinement flow of step S3 in FIG. 2 .

detailed description

It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes, and should not be construed as indicating or implying their relative importance or implicitly indicating the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the scope of protection claimed in this application.

As shown in FIG. 1 , FIG. 1 is a schematic diagram of a hardware structure of an image processing apparatus according to an embodiment of the present application.

As shown in FIG. 1 , the image processing apparatus may include: a processor 1001 , such as a CPU, a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface); the memory 1005 can be a high-speed RAM memory, or a non-volatile memory, such as a disk memory, and the memory 1005 can optionally be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the hardware structure shown in FIG. 1 does not constitute a limitation on the image processing apparatus, and may include more or less components than the one shown, or combine some components, or arrange different components.

As shown in FIG. 1 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module and an image processing program.

In the image processing apparatus shown in FIG. 1, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can be used to call the image processing program stored in the memory 1005, and perform the following operations :

acquiring first image data and second image data, wherein the first image data and the second image data are respectively obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope;

Acquiring color components in the first image data and the second image data and target components corresponding to the first image data and the second image data, and calculating according to the color components and the target components obtain the first set of calibration parameters and the second set of calibration parameters;

The first target image of the current target object obtained by the first camera is corrected according to the first set of correction parameters, and the first target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. Two target images are corrected.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

acquiring color components in the first image data and the second image data;

determining the target component according to the color component of the first image data or the color component of the second image data;

calculating the ratio between the target component and the color component of the first image data to obtain the first set of correction parameters;

The ratio between the target component and the color component of the second image data is calculated to obtain the second set of correction parameters.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

acquiring color components in the first image data and the second image data;

Determine a first target component corresponding to the first image data according to the color component of the first image data, and determine a second target component corresponding to the second image data according to the color component of the second image data;

According to the first target component and the second target component, calculating the equalization component;

According to the equalization component and the color component of the first image data, a first set of correction parameters is obtained by calculation, and a second set of correction parameters is calculated and obtained according to the equalization component and the components of the second image data.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

Obtain a first preset equalization scale coefficient corresponding to the first image data and a second preset equalization scale coefficient corresponding to the second image data, the first preset equalization parameter and the second preset The sum of the equalization parameters is the default value;

The equalization component is calculated and obtained according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

multiplying the first preset equalization proportional coefficient by the first target component to obtain a first product;

multiplying the second preset equalization proportional coefficient by the second target component to obtain a second product;

A sum of the first product and the second product is calculated to obtain the equalized component.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

obtaining the preset color components of the predetermined calibration plate;

Calculate the first difference between the first target component and the preset color component of the preset calibration board, and the second difference between the second target component and the preset color component of the preset calibration board. difference;

Comparing the magnitude of the first difference and the second difference, if the first difference is smaller than the second difference, the first preset equalization scale factor is greater than the second preset equalization scale factor ; if the first difference is greater than the second difference, the second preset equalization scale factor is greater than the first preset equalization scale factor; if the first difference is equal to the second difference value, the first preset equalization scale coefficient is equal to the second preset equalization scale coefficient.

Further, the processor 110 can call the image processing program of the stereoscopic endoscope stored in the memory 109, and also perform the following operations:

obtaining the components of the first target image and the components of the second target image;

Calculate the product between the first set of correction parameters and the components of the first target image to obtain the corrected components of the first target image, and calculate the difference between the second set of correction parameters and the components of the second target image The product of , obtains the corrected component of the second target image.

Referring to FIG. 2 , in the first embodiment, the present application provides an image processing method for a stereoscopic endoscope, including the following steps:

Step S1, acquiring first image data and second image data, the first image data and the second image data are respectively obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope;

Step S2: Acquire the color components in the first image data and the second image data and the target components corresponding to the first image data and the second image data, and according to the color components and the Calculate the target component to obtain the first set of correction parameters and the second set of correction parameters;

In this embodiment, the stereoscopic endoscope has a first camera and a second camera and two left and right cameras. Of course, in other embodiments, the stereoscopic endoscope may also have three or more cameras, and the specific number of cameras may be reasonably set according to actual needs. It can be understood that when the stereoscopic endoscope has multiple cameras, multiple cameras shoot the preset calibration plate, and multiple corresponding image data can be obtained, and the corresponding image processing method is basically the same as the method with two left and right cameras. .

The first image data may be the first frame image obtained by the first camera of the stereoscopic endoscope shooting the preset calibration plate, or may be the Nth frame image; the second image data may be the stereoscopic image data. The first frame of image obtained by the second camera of the endoscope shooting the preset calibration board may also be the Nth frame of image. To specifically select the first frame image or the Nth frame image, the image data captured by the first camera can be compared with the image data captured by the second camera. If the difference between the two does not exceed a preset ratio range such as 10%, it can be determined The image data currently captured by the first and second cameras are the first image data and the second image data, respectively. If the difference between the two ranges from 10% to 30%, the Nth frame image can be selected. If the difference between the two exceeds a preset ratio range such as 30%, it may indicate that the camera of the stereoscopic endoscope is faulty.

In addition, it should be understood that the first camera and the second camera of the stereoscopic endoscope may simultaneously photograph the preset calibration plate, or may successively photograph the preset calibration plates to obtain the corresponding first image data. and the second image data. The solution of the present application does not limit the time sequence in which the first camera and the second camera respectively shoot the preset calibration board.

The color components may be any of RGB values, HSV values, HSI values, CMYK values, HSL values, HSB values, Ycc values, XYZ values, Lab values, and YUV values corresponding to different color models. The preset calibration board may be a calibration board with known color components, for example, when the color components are RGB values, R=G=B, and R, G, B∈[0,255]. Specifically, the preset calibration plate can be: a pure white calibration plate with R=G=B=255, or can be a pure black calibration plate with R=G=B=0; it can also be: R=G= B = 200 or 100, etc. gray calibration plate. Of course, in other embodiments, the relationship between the three components of RGB may also be: R=G≠B, R≠G=B, or R≠G≠B.

The image data may include information such as color components, pixel values, grayscale values, saturation, brightness, and contrast. That is, the image data may be at least one of the above-mentioned pieces of information.

It can be understood that, due to the influence of factors such as sensors, lenses, light source positions, manufacturing, and assembly, the obtained first image data and second image data of a stereoscopic endoscope are usually not identical. The solution of the present application mainly corrects the color components of the first image data and the second image data, so that the colors of the images obtained by the first and second cameras of the stereoscopic endoscope are consistent. Of course, in other embodiments, other parameters such as saturation and contrast in the image data may also be corrected together, so that different image data obtained by different cameras of the stereoscopic endoscope tend to be consistent, thereby improving the operation. visual comfort.

In this embodiment, the target components of the first image data and the second image data may be the same or different. The target component may be determined according to one of the color components of the first image data, may also be determined according to one of the color components of the second image data, or may be determined according to one of the color components of the first image data. The component and one of the color components of the second image data are comprehensively determined. The details will be described in detail below, and will not be repeated here.

Since there are multiple parameter values corresponding to the color component and the target component, the corresponding correction parameters are a set of parameters. For example, when the color components are R, G, and B, the corresponding target components are also R, G, and B. At this time, the calculated correction parameters are a set of parameters {R, G, B}.

The calculation method of the correction parameter may be: calculating the ratio between the target component and the color component; also calculating the ratio between the color component and the target component; or calculating the ratio between the target component and the target component. The difference between the color components; it can also be calculated to obtain the above ratio or difference, plus or minus a correction coefficient, the correction coefficient can be an empirical value, or it can be an adjusted value after many tests . Of course, the calculation methods of the correction parameters are not limited to the above listed situations, and other reasonable calculation methods can also be selected according to actual needs.

It can be understood that, in order to make the different image data obtained by different cameras of the stereoscopic endoscope tend to be consistent, so as to further improve the visual comfort of the operator, other parameters in the image data can be corrected separately, or can be adjusted separately. Computational corrections are performed in conjunction with other parameters in the image data. For example, after the above-mentioned color components are corrected, other parameters in the image data, such as grayscale value, saturation, brightness, and contrast, may also be corrected separately. It is also possible to combine all image data such as color components, pixel values, grayscale values, saturation, brightness, and contrast, etc., to calculate an overall eigenvalue, and then correct the eigenvalues.

Step S3, correcting the first target image of the current target object obtained by the first camera according to the first set of correction parameters, and correcting the current target obtained by the second camera according to the second set of correction parameters. The second target image of the real object is corrected.

In this embodiment, the current target object may be an object corresponding to different application scenarios. For example, in an industrial equipment detection scenario, the current target object may be a detected object inside various machines or equipment; in medical surgery In the scenario, the current target object may be the affected tissue of the patient.

Likewise, the order of correcting the first target image acquired by the first camera and correcting the second target image acquired by the second camera in the present application is not limited. Specifically, when the corresponding correction parameters and the target image are obtained, the corresponding correction process can be executed; the corresponding correction process can also be executed simultaneously; or the corresponding calibration process can be executed sequentially in a preset order.

The specific correction method corresponds to the method of calculating the correction parameter. For example, when the correction parameter is the ratio between the target component and the color component, when the target image of the current target object is obtained, the feature data corresponding to the color components in the target image, and then calculate the product between the feature data and the correction parameter.

When the correction parameter is the difference between the target component and the color component, then when the target image of the current target object is obtained, the feature data corresponding to the color component in the target image is obtained, The sum between the characteristic data and the correction parameter is then calculated. Of course, in other embodiments, other reasonable correction manners may also be used, and the present application is not limited to the manners listed above.

The image processing method for a stereoscopic endoscope provided by the present application, by acquiring first image data and second image data, the first image data and the second image data are respectively obtained by the first camera of the stereoscopic endoscope and the second image data. The two cameras are obtained by shooting a preset calibration board, and then the color components in the first image data and the second image data and the target components corresponding to the first image data and the second image data are obtained, and The first set of correction parameters and the second set of correction parameters are calculated according to the color components and the target components, and finally the first target image of the current target object acquired by the first camera is processed according to the first set of correction parameters. Correction is performed, and the second target image of the current target object acquired by the second camera is corrected according to the second set of correction parameters. In this way, different image data obtained by different cameras of the stereoscopic endoscope can be made consistent, thereby improving the visual comfort of the operator.

3, in the second embodiment, the step S2 includes:

Step S21, acquiring color components in the first image data and the second image data;

In this embodiment, the color components in the whole text below are described by taking RGB values as an example. It is assumed that the acquired color component RGB values in the first image data are (200, 180, 210), and the acquired color component RGB values in the second image data are (190, 200, 220). It should be understood that the specific numerical values listed in the following full text are only used to help the understanding of the solution of the present application, and do not play a specific limiting role.

Step S22, determine the target component according to the color component of the first image data or the color component of the second image data;

In this embodiment, the target component may be determined according to the color component of the first image data or the color component of the second image data. Specifically, to select the color component of the first image data or the color component of the second image data, the color component of the first image data can be preset as the target component. Further, because the color component RGB has three R, G, and B components component, one of the components can be preset: if the G value of the first image data is preset as the target component, in this case, the corresponding target component is (180, 180, 180); it can also be preset as The color component of the second image data is the target component. For example, the R value of the second image data is set as the target component by default. In this case, the corresponding target component is (190, 190, 190). It can also be selected by the operator according to his own preferences. Therefore, the operator can choose according to actual needs. For example, when a more vivid red color is required, the R component can be selected as the target component.

In other embodiments, the R, G, and B values of the preset calibration plate are all known. Therefore, the R, G, and B values corresponding to each color component in the image data and the preset calibration board can be compared respectively, the difference value of each color component can be calculated, and the color component with the smallest difference value can be selected as the target component. . For example, assuming that the R, G, and B values of the preset calibration board are: R=G=B=200, since the R value of the color component in the first image data is 200 and the R value of the preset calibration board is the smallest ( is 0), the difference between the color component G value in the second image data is 200 and the G value of the preset calibration board is the smallest (0), then the target component can be directly determined to be (200, 200, 200) . If the R, G, and B values of the preset calibration board are: R=G=B=189, since the color component G value in the first image data is 180 and the G value of the preset calibration board is the smallest ( is 9), the R value of the color component in the second image data is 190 and the difference between the R value of the preset calibration plate is the smallest (1), then the target component can be directly determined to be (190, 190, 190) .

Step S23, calculating the ratio between the target component and the color component of the first image data to obtain the first set of correction parameters;

In this embodiment, assuming that RGB values (200, 200, 200) are used as the target components in step S23, the first set of correction parameters K _L0 (Kr ₀ , Kg ₀ , Kb ₀ ) are calculated:

Kr ₀ =R _L0 /R _L0 =200/200=1;

Kg ₀ =R _L0 /G _L0 =200/180=10/9;

Kb ₀ =R _L0 /B _L0 =200/210=20/21.

That is, K _L0 =(1, 10/9, 20/21).

Step S24: Calculate the ratio between the target component and the color component of the second image data to obtain the second set of correction parameters.

In this embodiment, it is assumed that RGB values (200, 200, 200) are used as the target components in step S23, and the second set of correction parameters K _R0 (Kr ₀ , Kg ₀ , Kb ₀ ) are calculated:

Kr ₀ =200/190=20/19;

Kg ₀ =200/200=1;

Kb ₀ =200/220=10/11.

That is, K _R0 =(20/19, 1, 10/11).

4, in the third embodiment, the step S2 includes:

Step S25, acquiring color components in the first image data and the second image data;

In this embodiment, it is assumed that the color component RGB values in the acquired first image data are (200, 180, 210), and the acquired color component RGB values in the second image data are (190, 200) , 220). It should be understood that the specific numerical values listed are only for helping the understanding of the solution of the present application, and do not play a specific limiting role.

Step S26: Determine a first target component corresponding to the first image data according to the color component of the first image data, and determine a second target component corresponding to the second image data according to the color component of the second image data. target component;

In this embodiment, the color component of the first image data may be preset as the first target component, and the color component of the second image data may be preset as the second target component. Further, since the color component RGB has three components of R, G, and B, one of the components may be preset: if the value of G of the first image data is preset as the first target component, at this time, the corresponding The target component is (180, 180, 180); it is also possible to preset the color component of the second image data as the target component, for example, the R value of the second image data is preset as the target component, at this time, the corresponding The second target component is (190, 190, 190). The operator may also select any color component in the first image data as the first target component and any color component in the second image data as the second target component according to his own preferences. For example, when a more vivid red color is required, the R component in the first image data may be selected as the first target component, and the R component in the second image data may be selected as the second target component.

In other embodiments, the R, G, and B values of the preset calibration plate are all known. Therefore, the R, G, and B values corresponding to each color component in the first image data and the preset calibration board can be compared respectively, the difference value of each color component can be calculated, and the color component with the smallest difference value can be selected as the the first target component; similarly, the R, G, and B values corresponding to each color component in the second image data and the preset calibration board can be compared respectively, and the difference value of each color component can be calculated, and therefrom. The color component with the smallest difference is selected as the second target component. For example, assuming that the R, G, and B values of the preset calibration board are: R=G=B=200, since the R value of the color component in the first image data is 200 and the R value of the preset calibration board is the smallest ( is 0), then it can be determined that the first target component is (200, 200, 200); since the color component G value in the second image data is 200 and the G value of the preset calibration board is the smallest difference (for 0), the second target component can be determined to be (200, 200, 200). If the R, G, and B values of the preset calibration board are: R=G=B=195, since the color component R value in the first image data is 200 and the R value of the preset calibration board is the smallest ( is 5), it can be determined that the first target component is (200, 200, 200); since the color component R value in the second image data is 190 and the R value of the preset calibration plate has the smallest difference (0 ), the second target component can be determined to be (190, 190, 190).

Step S27, calculating the equalization component according to the first target component and the second target component;

In this embodiment, after the first target component and the second target component are obtained, an equalized component of the first target component and the second target component may be calculated and obtained. Specifically, weight coefficients corresponding to the first target component and the second target component can be assigned respectively, and the magnitude relationship between the two weight coefficients can be preset, or multiple options can be provided for the operator to choose, or Calculated.

In other embodiments, an average value between the first target component and the second target component may also be calculated, and the obtained average value is the equalized component.

It can be understood that, after the above-mentioned equalized component is obtained by calculation, other parameters, coefficients or constants may also be introduced, so as to make the calculated equalized component more accurate. In addition, the methods for calculating the equalization component are not the two listed above, and in other embodiments, other reasonable calculation methods may also be used, which are not exhaustive here.

Based on the third embodiment, referring to FIG. 5 , the step S27 further includes:

Step S271: Obtain a first preset equalization scale coefficient corresponding to the first image data and a second preset equalization scale coefficient corresponding to the second image data, where the first preset equalization parameter is the same as the first preset equalization scale coefficient. The sum of the two preset equalization parameters is the preset value;

Step S272: Calculate and obtain the equalization component according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component.

In this embodiment, a first preset equalization scale coefficient K ₁ corresponding to the first image data and a second preset equalization scale coefficient K ₂ corresponding to the second image data are obtained. The sum of the first preset equalization parameter and the second preset equalization parameter is a preset value, and the preset value may be 1 or other reasonable values. For example, K ₁ +K ₂ =1, _{and the values of K 1} and K ₂ can be reasonably set according to actual needs. _{K 1 = 0, K 2 =} 1; K 1 = 1, K 2 = 0; K 1 = 0.5, K 2 = 0.5; K 1 = 0.7, K 2 = 0.3 and the like.

_{The equalization component RGB R1} is calculated and obtained according to the first preset equalization scale coefficient K ₁ , the second preset equalization scale coefficient K ₂ , the first target component RGB _L1 , and the second target component, and the calculated Describe the equalized component RGB _A .

Wherein, the magnitude relationship between the first preset equalization scale coefficient K ₁ and the second preset equalization scale coefficient K ₂ corresponding to the second image data may be determined in the following manner:

Obtain the preset color component of the predetermined calibration plate; calculate the first difference between the first target component and the preset color component of the preset calibration plate, and calculate the second target component and the preset color component. calibrating the second difference between the preset color components of the board; comparing the magnitude of the first difference and the second difference; if the first difference is smaller than the second difference, the first difference The preset equalization scale coefficient is greater than the second preset equalization scale coefficient; if the first difference is greater than the second difference, the second preset equalization scale coefficient is greater than the first preset equalization scale coefficient; if the first difference is equal to the second difference, the first preset equalization scale coefficient is equal to the second preset equalization scale coefficient. The specific value of the weight coefficient is set reasonably according to the actual needs, and the specific value is not given as an example here.

Further, referring to FIG. 6 , the step S272 includes:

Step S2721, multiplying the first preset equalization scale coefficient by the first target component to obtain a first product;

In this embodiment, the first preset equalization scale coefficient K _{1 is} multiplied by the first target component RGB _{L1 to} obtain a first product: K ₁ *RGB _L1 .

Step S2722, multiplying the second preset equalization proportional coefficient by the second target component to obtain a second product;

In this embodiment, the second preset equalization scale coefficient K _{2 is} multiplied by the second target component RGB _{R1 to} obtain a second product: K ₂ *RGB _R1 .

Step S2723: Calculate the sum of the first product and the second product to obtain the equalized component.

In this embodiment, the calculated sum of the first product K ₁ *RGB _L1 and the second product K ₂ *RGB _R1 is calculated to obtain the equalized component RGB _A :

RGB _A =K ₁ *RGB _L1 +K ₂ *RGB _R1 .

Assuming K ₁ =0.5, K ₂ =0.5, the first target component RGB _L1 in the above step S26 is (200, 200, 200), and the second target component RGB _R1 is (220, 220, 220).

At this time, RGB _A =K ₁ *RGB _L1 +K ₂ *RGB _R1 =0.5*(200, 200, 200)+0.5*(220, 220, 220)=(210, 210, 210).

Step S28: Calculate a first set of correction parameters according to the equalization component and the color component of the first image data, and calculate and obtain a second set of correction parameters according to the equalization component and the components of the second image data.

In this embodiment, the calculation method of the first group of correction parameters may be: calculating the ratio between the equalization component and the color component of the first image data; or calculating the difference between the color component of the first image data and the color component of the first image data. The ratio between the equalized components; it can also be calculated as the difference between the equalized component and the color component of the first image data; it can also be calculated to add or subtract the above ratio or difference. To get a correction coefficient, the correction coefficient can be an empirical value or an adjusted value after many trials. Of course, the calculation methods of the first group of correction parameters are not limited to the above listed situations, and other reasonable calculation methods can also be selected according to actual needs.

Similarly, the calculation method of the second group of correction parameters may be: calculating the ratio between the equalization component and the color component of the second image data; or calculating the color component of the second image data and the color component of the second image data. The ratio between the equalized components; it can also be calculated as the difference between the equalized component and the color component of the second image data; it can also be calculated to obtain the above ratio or difference, plus or minus a Correction coefficient, which can be either an empirical value or an adjusted value after many tests. Of course, the calculation methods of the second group of correction parameters are not limited to the above listed situations, and other reasonable calculation methods can also be selected according to actual needs.

The calculation methods between the first group of correction parameters and the second group of correction parameters may be the same or different. It can be understood that, in order to ensure that the first image data and the second image data tend to be consistent, it may be preferable that the two are calculated in the same manner.

Specifically, it is assumed that the acquired color component RGB value in the first image data is: RGB _L0 =(200, 180, 210), and the acquired color component RGB value in the second image data is: RGB _R0 =(190, 200, 220), then the _{calculation methods of the first group of correction parameters K L1} and the second group of correction parameters K _R1 may be as follows:

KL1=RGBA/RGBL0=(210, 210, 210)/(200, 180, 210)=(21/20, 7/6, 1),

KR1=RGBA/RGBR0=(210, 210, 210)/(190, 200, 220)=(21/19, 21/20, 21/22).

7 , in the fifth embodiment, based on the fourth embodiment, the step S30 includes:

Step S31, acquiring the components of the first target image and the components of the second target image;

In this embodiment, it is assumed that the acquired components of the first target image are: URGB _L0 =(200, 200, 220), and the acquired components of the second target image are: URGB _R0 =(200, 180, 210 ). It can be understood that the components of the first target image and the components of the second target image correspond to the color components in the first image data and the color components in the second image data, respectively, such as: When the color components are RGB values, the components of the first target image or the second target image are also RGB values; similarly, when the color components are YCbCr values, the first target image or the second target image The components of the target image are also YCbCr values.

Step S32: Calculate the product between the first set of correction parameters and the components of the first target image to obtain the corrected first target image, and calculate the difference between the second set of correction parameters and the components of the second target image. to obtain the corrected second target image.

In this embodiment, the product between the first set of correction parameters K _{L1 and the component URGB L0 of the} first target image is calculated: K _L1 *URG _L0 , at this time, the corrected component of the first target image is K _L1 *URGB _L0 , so as to obtain the corrected first target image.

Calculate the product between the second set of correction parameters K _R1 and the component URGB _{R0 of the second target image: K R1} *URG _R0 , at this time, the corrected component of the second target image is K _R1 *URG _R0 , so as to obtain the corrected second target image.

The present application also provides an image processing device, which includes a memory, a processor, and an image processing program for a stereoscopic endoscope that is stored on the memory and can run on the processor; The image processing program of the speculum, when executed by the processor, implements the steps of the method as described above.

In this embodiment, the image processing device may be a stereoscopic endoscope, a device independent of the stereoscopic endoscope, or a surgical robot.

Specifically, the surgical robot may generally include a master console and a slave operation device. The master console is used to send control commands to the slave operation device according to the operation of the doctor to control the slave operation device; the slave operation device is used to respond to the command sent by the master console. Control commands and perform corresponding surgical operations. The slave operating device includes a mechanical arm, a power mechanism disposed on the mechanical arm, and an operating arm, the operating arm is used to extend into the body under the driving action of the power mechanism, and perform surgical operations through its distal end instrument, and/or acquire in vivo images through its distally located stereoscopic endoscope. The main console is also used for displaying images obtained by operating the device.

The present application further provides a computer-readable storage medium on which an image processing program of a stereoscopic endoscope is stored, and the image processing program of the stereoscopic endoscope is executed by a processor to achieve the above-mentioned The steps of the image processing method of the image processing apparatus.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make an image processing device (which may be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

Claims (17)

An image processing method for a stereoscopic endoscope, wherein the image processing method for a stereoscopic endoscope comprises: acquiring first image data and second image data, wherein the first image data and the second image data are respectively obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope; Acquiring color components in the first image data and the second image data and target components corresponding to the first image data and the second image data, and calculating according to the color components and the target components obtain the first set of calibration parameters and the second set of calibration parameters; The first target image of the current target object obtained by the first camera is corrected according to the first set of correction parameters, and the first target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. Two target images are corrected. The image processing method of a stereoscopic endoscope according to claim 1, wherein the acquiring color components in the first image data and the second image data and the color components in the first image data and the The target components corresponding to the second image data, and the steps of calculating and obtaining the first set of correction parameters and the second set of correction parameters according to the color components and the target components include: acquiring color components in the first image data and the second image data; determining the target component according to the color component of the first image data or the color component of the second image data; calculating the ratio between the target component and the color component of the first image data to obtain the first set of correction parameters; The ratio between the target component and the color component of the second image data is calculated to obtain the second set of correction parameters. The image processing method of a stereoscopic endoscope according to claim 1, wherein the acquiring color components in the first image data and the second image data and the color components in the first image data and the The target components corresponding to the second image data, and the steps of calculating and obtaining the first set of correction parameters and the second set of correction parameters according to the color components and the target components include: acquiring color components in the first image data and the second image data; Determine a first target component corresponding to the first image data according to the color component of the first image data, and determine a second target component corresponding to the second image data according to the color component of the second image data; According to the first target component and the second target component, calculating the equalization component; According to the equalization component and the color component of the first image data, a first set of correction parameters is obtained by calculation, and a second set of correction parameters is calculated and obtained according to the equalization component and the components of the second image data. The image processing method of a stereoscopic endoscope according to claim 3, wherein the step of calculating the equalized component according to the first target component and the second target component comprises: Obtain a first preset equalization scale coefficient corresponding to the first image data and a second preset equalization scale coefficient corresponding to the second image data, the first preset equalization parameter and the second preset The sum of the equalization parameters is the default value; The equalization component is calculated and obtained according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component. The image processing method of a stereoscopic endoscope according to claim 4, wherein the first preset equalization scale factor, the second preset equalization scale factor, the first target component, and the The step of calculating the second target component to obtain the equalized component includes: multiplying the first preset equalization proportional coefficient by the first target component to obtain a first product; multiplying the second preset equalization proportional coefficient by the second target component to obtain a second product; A sum of the first product and the second product is calculated to obtain the equalized component. The image processing method for a stereoscopic endoscope according to claim 4, wherein the image processing method for a stereoscopic endoscope further comprises: obtaining the preset color components of the predetermined calibration plate; Calculate the first difference between the first target component and the preset color component of the preset calibration board, and the second difference between the second target component and the preset color component of the preset calibration board. difference; Comparing the magnitude of the first difference and the second difference, if the first difference is smaller than the second difference, the first preset equalization scale factor is greater than the second preset equalization scale factor ; if the first difference is greater than the second difference, the second preset equalization scale factor is greater than the first preset equalization scale factor; if the first difference is equal to the second difference value, the first preset equalization scale coefficient is equal to the second preset equalization scale coefficient. The image processing method for a stereoscopic endoscope according to claim 1, wherein the first target image of the current target object acquired by the first camera is corrected according to the first set of correction parameters, and the first target image is corrected according to the first set of correction parameters. The step of correcting the second target image of the current target object obtained by the second camera with the second set of correction parameters includes: obtaining the components of the first target image and the components of the second target image; Calculate the product between the first set of correction parameters and the components of the first target image to obtain the corrected components of the first target image, and calculate the difference between the second set of correction parameters and the components of the second target image The product of , obtains the corrected component of the second target image. The image processing method of a stereoscopic endoscope according to claim 1, wherein the color components are RGB values, HSV values, HSI values, CMYK values, HSL values, HSB values, and Ycc values corresponding to different color models , XYZ value, Lab value, YUV value. An image processing device, characterized in that the image processing device comprises a memory, a processor, and an image processing program of a stereoscopic endoscope that is stored on the memory and can run on the processor; When the image processing program of the stereoscopic endoscope is executed by the processor, the following steps are implemented: acquiring first image data and second image data, wherein the first image data and the second image data are respectively obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope; Acquiring color components in the first image data and the second image data and target components corresponding to the first image data and the second image data, and calculating according to the color components and the target components obtain the first set of calibration parameters and the second set of calibration parameters; The first target image of the current target object obtained by the first camera is corrected according to the first set of correction parameters, and the first target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. Two target images are corrected. A computer-readable storage medium, characterized in that an image processing program of a stereoscopic endoscope is stored on the computer-readable storage medium, and the image processing program of the stereoscopic endoscope is implemented as follows when executed by a processor The steps of the image processing method of the image processing apparatus: acquiring first image data and second image data, wherein the first image data and the second image data are respectively obtained by shooting a preset calibration plate with the first camera and the second camera of the stereoscopic endoscope; Acquiring color components in the first image data and the second image data and target components corresponding to the first image data and the second image data, and calculating according to the color components and the target components obtain the first set of calibration parameters and the second set of calibration parameters; The first target image of the current target object obtained by the first camera is corrected according to the first set of correction parameters, and the first target image of the current target object obtained by the second camera is corrected according to the second set of correction parameters. Two target images are corrected. 11. The computer-readable storage medium of claim 10, wherein the acquiring color components in the first image data and the second image data and the color components in the first image data and the second image data The target components corresponding to the image data, and the steps of obtaining the first set of correction parameters and the second set of correction parameters according to the color components and the target components include: acquiring color components in the first image data and the second image data; determining the target component according to the color component of the first image data or the color component of the second image data; calculating the ratio between the target component and the color component of the first image data to obtain the first set of correction parameters; The ratio between the target component and the color component of the second image data is calculated to obtain the second set of correction parameters. 11. The computer-readable storage medium of claim 10, wherein the acquiring color components in the first image data and the second image data and the color components in the first image data and the second image data The target components corresponding to the image data, and the steps of obtaining the first set of correction parameters and the second set of correction parameters according to the color components and the target components include: acquiring color components in the first image data and the second image data; Determine the first target component corresponding to the first image data according to the color component of the first image data, and determine the second target component corresponding to the second image data according to the color component of the second image data; According to the first target component and the second target component, calculating the equalization component; According to the equalization component and the color component of the first image data, a first set of correction parameters is obtained by calculation, and a second set of correction parameters is calculated and obtained according to the equalization component and the components of the second image data. The computer-readable storage medium of claim 12, wherein the step of calculating the equalized component according to the first target component and the second target component comprises: Obtain a first preset equalization scale coefficient corresponding to the first image data and a second preset equalization scale coefficient corresponding to the second image data, the first preset equalization parameter and the second preset The sum of the equalization parameters is the default value; The equalization component is calculated and obtained according to the first preset equalization scale coefficient, the second preset equalization scale coefficient, the first target component, and the second target component. 14. The computer-readable storage medium of claim 13, wherein the first preset equalization scale factor, the second preset equalization scale factor, the first target component, and the second The step of calculating the target component to obtain the equalized component includes: multiplying the first preset equalization proportional coefficient by the first target component to obtain a first product; multiplying the second preset equalization proportional coefficient by the second target component to obtain a second product; A sum of the first product and the second product is calculated to obtain the equalized component. The computer-readable storage medium of claim 12, wherein the image processing method for a stereoscopic endoscope further comprises: obtaining the preset color components of the predetermined calibration plate; Calculate the first difference between the first target component and the preset color component of the preset calibration board, and the second difference between the second target component and the preset color component of the preset calibration board. difference; Comparing the magnitude of the first difference and the second difference, if the first difference is smaller than the second difference, the first preset equalization scale factor is greater than the second preset equalization scale factor ; if the first difference is greater than the second difference, the second preset equalization scale factor is greater than the first preset equalization scale factor; if the first difference is equal to the second difference value, the first preset equalization scale coefficient is equal to the second preset equalization scale coefficient. The computer-readable storage medium according to claim 10, wherein the first target image of the current target object acquired by the first camera is corrected according to the first set of correction parameters, and the first target image is corrected according to the first set of correction parameters. The step of correcting the second target image of the current target object obtained by the second camera with the two sets of correction parameters includes: obtaining the components of the first target image and the components of the second target image; Calculate the product between the first set of correction parameters and the components of the first target image to obtain the corrected components of the first target image, and calculate the difference between the second set of correction parameters and the components of the second target image The product of , obtains the corrected component of the second target image. The computer-readable storage medium of claim 10, wherein the color components are RGB values, HSV values, HSI values, CMYK values, HSL values, HSB values, Ycc values, XYZ values corresponding to different color models , Lab value, or YUV value.

Images