WO2015005623A1 - Apparatus, method and system for automatically removing visual noise in surgical operation image in real time - Google Patents

Abstract

Disclosed are an apparatus, a method and a system for removing noise in a surgical operation image. The apparatus for removing noise in a surgical operation image can comprise an input unit for receiving the surgical operation image, and a control unit for enhancing the surgical operation image by removing noise included in the received surgical operation image. The control unit can further include a function for analyzing features of the received surgical operation image by using at least one of a histogram, a mean value, a maximum value, a minimum value and standard deviation, and for determining the type of noise included in the surgical operation image by using the analyzed features of the surgical operation image. The control unit can further include a function for determining the type of noise of the received surgical operation image according to a user input.

Description

Apparatus, Method and System for Automatic Elimination of Real-time Visual Noise in Surgical Images

The present invention relates to a surgical image processing apparatus, and more particularly, to an apparatus, method and system for automatically removing real-time visual noise of a surgical image.

Recently, surgery performed by a doctor includes laparotomy and laparoscopic surgery.

Laparotomy is a surgery that splits open the stomach and treats or removes the organs in the stomach.

Laparoscopic surgery is a surgical procedure that involves drilling three to four small holes without opening the patient, and then inserting an endoscope equipped with a special camera and surgical instruments. In addition, laparoscopic surgery has fewer complications than laparotomy, and can start treatment much faster after the procedure, and maintain the stamina or immune function of the surgical patient. Therefore, considering the rapid recovery of beauty and patients, which is a modern trend, the field is expected to be developed in the future.

However, laparoscopic surgery may be affected by the endoscope image because the surgery is performed depending on the image taken from the endoscope. Endoscopic images may degrade image quality due to various causes, which may cause doctors to feel visual discomfort during observation and surgery. Doctors also waste a lot of time in surgery by removing and cleaning the endoscope from the patient's body to clear the endoscope image.

Currently, there are various image enhancement algorithms and systems, but the image enhancement system for laparoscopic surgery images is still difficult to find in the market or academia.

Although the University of Erlangen-Nuremberg Martesnsstr, Germany, developed Open CV-based image enhancement system in PC environment and adopted the method of displaying the original image improved image at the same time, it did not overcome the limitation of resolution and processing time of image.

Sweden's LYYN company has developed an image enhancement system, but its applications are different, its resolution is low, and its price is high.

Korean Patent Laid-Open Publication No. 10-2011-0036453 relates to a surgical image processing apparatus and a method thereof, and discloses a robot and image processing for laparoscopic surgery. However, the patent shows a limitation that the image itself cannot be improved by using an image processing method that matches an endoscope image and a modeled image to produce an output image.

Therefore, as the laparoscopic surgery system is miniaturized and developed at high resolution, an image enhancement system that provides additional functions must also be capable of real-time image processing and remain a problem to be implemented as an embedded device.

The technical problem to be achieved by the present invention is to provide a clearer and improved image to the user by removing the noise contained in the surgical image using the endoscope in real time and automatically.

Another technical problem to be achieved by the present invention is to shorten the operation time by reducing the endoscope cleaning occurring during surgery.

Another object of the present invention is to select the automatic control or manual control to remove the noise included in the surgical image.

The surgical image noise removing apparatus may include an input unit for receiving a surgical image and a controller for automatically removing noise included in the received surgical image to improve the surgical image in real time.

The apparatus may further include a storage unit in which the improved surgical image is stored.

The control unit analyzes a characteristic of the surgical image by using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and uses the characteristic of the analyzed surgical image to determine the surgical image. It may include a function for determining the type of noise included.

The controller may include a function of determining a type of noise based on a user's input of the received surgical image.

The controller is configured to display at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of each pixel of the received image using at least one color model of RGB, YCbCr, HSV, and Gray formats. An image analyzer for calculating and analyzing the features of the surgical image, an image determining unit for determining the type of noise according to a preset criterion by using the features of the analyzed image, and removing the type of the determined noise. It may include at least one of the image improving unit for improving the.

The image improvement unit may include a mist treatment unit for removing fog, a droplet processing unit for removing water droplets, a smoke processing unit for removing smoke, a particle processing unit for removing particles, a low contrast processing unit for restoring low contrast, and blurring due to reflection and the reflection It may include at least one of the reflection and blur processing unit for removing.

Surgical image noise removing system is a surgical image noise removing device according to the above,

A camera controller for photographing a surgical image, a light source for adjusting the brightness of the surgical image by providing light to the probe, a camera for photographing the surgical image, and providing the surgical image to the surgical image noise removing device. And a monitor displaying the improved image in the surgical image noise removing device.

The surgical image noise removing device may be detachable.

The surgical image noise removing method may include receiving a surgical image and improving the surgical image by removing noise included in the received surgical image.

Analyze the characteristics of the surgical image using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and noise included in the surgical image by using the characteristics of the analyzed surgical image. The method may further include determining a type of.

The method may further include determining a type of noise based on a user's input of the received surgical image.

The method may further include storing the improved surgical image in an analog storage device.

The method may further include storing the improved surgical image in a digital storage device.

According to the apparatus, method and system for automatically removing the real-time visual noise of the surgical image according to the present invention, noise included in the surgical image using the endoscope may be removed in real time and automatically to provide a clearer and improved image to the user.

In addition, the operation time can be shortened by reducing endoscopic cleaning occurring during surgery.

In addition, by selecting the automatic control or manual control to remove the noise included in the surgical image, it can reflect the user's convenience and intention.

In addition, the surgical imaging noise removing device may be detachable and compatible with the surgical imaging system.

1 is an exemplary view showing a surgical image noise removal system according to an embodiment of the present invention.

2 is a block diagram illustrating a surgical image noise removing apparatus according to an exemplary embodiment of the present invention.

3 is a block diagram illustrating a control unit according to an embodiment of the present invention.

4 is an exemplary diagram illustrating a kind of noise generated during surgery according to an embodiment of the present invention.

5 is an exemplary diagram illustrating image analysis of an image analyzer according to an exemplary embodiment of the present invention.

6 is an exemplary view illustrating image analysis of an image analyzer according to another exemplary embodiment of the present invention.

7 is an exemplary view illustrating image analysis of an image analyzer according to another exemplary embodiment of the present invention.

8 is a block diagram illustrating an image improving unit according to an exemplary embodiment of the present invention.

9 is an exemplary view illustrating image improvement of an image improving unit according to an exemplary embodiment of the present invention.

10 is an exemplary view illustrating image improvement of an image improving unit according to another exemplary embodiment of the present invention.

11 is an exemplary view illustrating image improvement of an image improving unit according to another exemplary embodiment of the present invention.

12 is a flowchart illustrating a surgical image noise removing method according to an embodiment of the present invention.

The surgical image noise removing apparatus may include an input unit for receiving a surgical image and a controller for automatically removing noise included in the received surgical image to improve the surgical image in real time.

The apparatus may further include a storage unit in which the improved surgical image is stored.

The control unit analyzes a characteristic of the surgical image by using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and uses the characteristic of the analyzed surgical image to determine the surgical image. It may include a function for determining the type of noise included.

The controller may include a function of determining a type of noise based on a user's input of the received surgical image.

The controller is configured to display at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of each pixel of the received image using at least one color model of RGB, YCbCr, HSV, and Gray formats. An image analyzer for calculating and analyzing the features of the surgical image, an image determining unit for determining the type of noise according to a preset criterion by using the features of the analyzed image, and removing the type of the determined noise. It may include at least one of the image improving unit for improving the.

The image improvement unit may include a mist treatment unit for removing fog, a droplet processing unit for removing water droplets, a smoke processing unit for removing smoke, a particle processing unit for removing particles, a low contrast processing unit for restoring low contrast, and blurring due to reflection and the reflection It may include at least one of the reflection and blur processing unit for removing.

Surgical image noise removing system is a surgical image noise removing device according to the above,

A camera controller for photographing a surgical image, a light source for adjusting the brightness of the surgical image by providing light to the probe, a camera for photographing the surgical image, and providing the surgical image to the surgical image noise removing device. And a monitor displaying the improved image in the surgical image noise removing device.

The surgical image noise removing device may be detachable.

The surgical image noise removing method may include receiving a surgical image and improving the surgical image by removing noise included in the received surgical image.

Analyze the characteristics of the surgical image using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and noise included in the surgical image by using the characteristics of the analyzed surgical image. The method may further include determining a type of.

The method may further include determining a type of noise based on a user's input of the received surgical image.

The method may further include storing the improved surgical image in an analog storage device.

The method may further include storing the improved surgical image in a digital storage device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, in adding reference numerals to the elements of each drawing, it may be noted that the same elements are denoted by the same reference numerals as much as possible even though they are displayed on different drawings. In addition, in the following description of the present invention, if it is determined that the detailed description of the related well-known configuration or function is apparent to those skilled in the art or may obscure the gist of the present invention, the detailed description may be omitted.

1 is an exemplary view showing a surgical image noise removal system according to an embodiment of the present invention.

Referring to FIG. 1, the surgical image noise removing system 1 may provide a clearer and improved image than an image generated in an operation using an endoscope. The surgical image noise removing system 1 may provide a clear and improved image by removing noise included in the image. The surgical image noise removing system 1 may include a surgical image noise removing device 10 and a surgical imaging system 20.

The surgical image noise removing apparatus 10 may be a device for removing noise included in the surgical image. The surgical image noise removing apparatus 10 may be detachable, and may be compatible with the surgical imaging system 20 and at least one of hardware and software. In addition, the surgical image noise removing apparatus 10 may receive a source image, that is, a surgical image from the surgical imaging system 20, and then provide the surgical imaging system 20 with an improved image from which noise is removed through image processing. Can be.

Surgical imaging system 20 may be a system for providing an image for the user to perform the surgery using the endoscope. The surgical imaging system 20 may include an endoscope 110, a light source 120, a camera controller 130, and a monitor 140.

Endoscope 110 may be a medical device made to directly see the internal organs or the body cavity. The endoscope 110 may be used for not only laparoscopic but also at least one of thoracoscopic, arthroscopic, parenteral, bladder, rectal, duodenum, mediastinal and cardiac. The endoscope 110 may include a CCD chip 111, a camera objective lens 112, a focus 113, an alternative lens section 114, a relay lens section 115, and an objective lens section 116.

The objective lens section 116 may be a section in which an image of the surgical image is formed. The objective lens section 116 may be located at the other end of the endoscope handle. In addition, the objective lens section 116 may be at a position where the endoscope 110 may be most in contact with the body.

The relay lens section 115 may be a section for forming an image from the image of the body formed in the objective lens section 116. Therefore, the relay lens section 115 may serve to adjust the focal length of the body image. The relay lens section 115 may be located between the objective lens section 116 and the alternative lens section 114.

The alternative lens section 114 may be a section for further enlarging an image made in the relay lens section 115. The alternative lens section 114 may be located in the direction of the endoscope handle.

The focus 113 may be a point at which the magnified image converges in the lens or the concave mirror in the enlarged image in the alternative lens section 114.

The camera objective lens 112 may be a lens that receives the light converged at the focus 114 to form an image.

The CCD chip 111 may be a chip that stores an image formed by the camera objective lens 112. Also, the CCD chip 111 is a charge coupled device, and unlike the conventional transistor device, the CCD chip 111 has two functions of accumulating and transmitting signals.

The endoscope 110 may include a camera including the above components.

The light source 120 may provide light to the endoscope 110 to adjust the brightness of the surgical image. The light source 120 may be connected to the relay lens section 115 and may provide the light to the relay lens section 115. Therefore, the light source 120 may provide light into the body of the patient through the endoscope 110.

The camera controller 130 may control a camera included in the endoscope 110. The camera controller 130 may manually or automatically control adjustment of the electrode voltage, beam current and focus, screen size, position, shading, brightness signal level, color, color signal level, synchronization, etc. of the camera. Can be. The camera controller 130 may provide a source image photographed by the endoscope 110 to the surgical image noise removing system 10. The source image may be a surgical image.

The monitor 140 may display an improved image provided by the surgical image noise removing device 10. The monitor 140 may display the surgery image so that the user may perform the surgery using the endoscope 110.

2 is a block diagram illustrating a surgical image noise removing apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the surgical image noise removing apparatus 10 may receive a surgical image and calculate an improvement image through image processing of the received surgical image. The surgical image noise removing apparatus 10 may remove noise included in the surgical image by automatic control or manual control. The automatic control or the manual control may be selected from at least one of software and hardware. The surgical image noise removing apparatus 10 may include an input unit 210, a control unit 220, an output unit 230, and a storage unit 240.

The input unit 210 may receive a surgical image. The input unit 210 may receive a surgical image provided by the camera controller 130. The input unit 210 may receive the surgical image through a cable compatible with the camera controller 130. In addition, the surgical image may be a surgical image including noise.

The controller 220 may image the surgical image including the noise to calculate an improved image. The controller 220 may perform image processing by automatic control or manual control.

When the controller 220 performs automatic control, the controller 220 may analyze at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the surgical image received from the input unit 210. The controller 220 may determine the type of noise included in the surgical image based on a preset criterion.

When the controller 220 performs manual control, the controller 220 may determine noise included in the surgical image according to an external input. The external input may be a user input. The user may select the type of noise included in the surgical image.

The controller 220 may select an algorithm according to the type of noise obtained through automatic control or manual control to remove the noise. The controller 220 may remove at least one noise from water vapor mist on the lens surface, water droplets formed on the lens surface, smoke generated during laser surgery, particles flying during laser surgery, low contrast and strong reflection, and consequent blurring. have.

The output unit 230 may transmit the surgical image from which the noise is removed from the controller 220 to the monitor 140. The output unit 230 may provide the improved surgical image to the monitor 140 through a cable compatible with the monitor 140.

The storage unit 240 may include at least one of an algorithm for analyzing an image, an algorithm for determining a noise type of the analyzed image, and an algorithm for removing noise. The storage unit 240 may store a criterion for determining the type of noise. The criterion may be setting of data for a specialized feature according to the type of noise. Storage unit 240 may be compatible because the information of the existing surgical imaging system 20 is stored. In addition, the storage unit 240 may update the information of the image processing algorithm and the surgical imaging system. The storage 240 may be physically independent, and the surgical image improved by the controller 220 may be stored. The storage unit 240 may be an analog storage device or a digital storage device.

3 is a block diagram illustrating a control unit according to an embodiment of the present invention.

Referring to FIG. 3, the controller 220 may remove noise included in the surgical image received by the input unit 210. The controller 220 may remove the noise through image processing. The controller 220 may include an image analyzer 310, an image determiner 320, and an image enhancer 330.

The image analyzer 310 may analyze the surgical image by expressing the surgical image in a histogram. The image analyzer 310 may calculate a histogram for each pixel using at least one color model of an RGB format, a YCbCr format, an HSV format, and a gray format.

The RGB format is the most basic color model and represents a combination of three components: red, green, and blue. YCbCr format is a color model that expresses the brightness component (Y) and the color difference information (Cb, Cr) separately from RGB colors. The HSV format is a color model that expresses three components: Hue, Saturation, and Value. Gray format is a color model that expresses the image using only brightness information without using color information.

The image analyzer 310 may calculate an average value, a maximum value, a minimum value, and a standard deviation of each pixel. In addition, the image analyzer 310 may calculate the size and speed of the moving object. The image analyzer 310 may use an algorithm stored in the storage 240.

The image determiner 320 may determine the type of noise included in the surgical image by using the data analyzed by the image analyzer 310. The image determiner 320 may determine the type of noise based on a predetermined criterion for the characteristic of the surgical image. According to the kind of noise, the image determining unit 320 may include at least one of water vapor mist on the surface of the lens, water droplets formed on the lens surface, smoke generated during laser surgery, particles flying during laser surgery, low contrast and strong reflections, and the resulting blurring. One noise can be determined. In addition, the image determiner 320 may determine noise that is unnecessary for the surgical image, not limited to the above-mentioned noise.

The image determiner 320 may determine the water vapor fog on the lens surface. The image determiner 320 may check the contrast of the surgical image to determine the vapor fog.

The image determiner 320 may determine the water droplets formed on the lens surface by using three methods. The three methods may be model based droplet recognition, droplet recognition using a continuous frame, and droplet recognition using a droplet model and a continuous frame.

The image determiner 320 may perform model-based water droplet recognition. The image determining unit 320 may set the water droplets to an elliptical shape, and set the pixel value to be a very high value in the surgical image using strong illumination. In addition, the image determiner 320 may set that the reduced form of the entire surgical image is represented inside the droplet in the case of transparent droplets, and that the pixel value is expressed in a Gaussian form in the case of opaque droplets.

The image determining unit 320 may determine the smoke generated during the laser surgery. The image determiner 320 may use at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the surgical image. If there is smoke, the image determining unit 320 may set the surgery image to be generated in a signal having a high histogram distribution, having a high average value, and having a high minimum value because the surgery image is generally displayed as a bright image.

The image determiner 320 may determine particles flying during laser surgery. The image determining unit 320 may use the motion information in the surgical image in determining the particles. The image determiner 320 may determine the size and speed of the moving object.

The image determiner 320 may determine a low contrast. The image determiner 320 may set that the low contrast image has a feature that the histogram distribution is concentrated in a specific region. Accordingly, the image determiner 320 may use a maximum value-minimum value (dynamic range) and an average value.

The image determiner 320 may determine the strong reflection and the resulting blur. The image determiner 320 may use the color and the ratio of the pixels whose strong reflection exceeds a predetermined threshold.

The image improving unit 330 may improve the surgical image by using a noise removing algorithm according to the type of noise determined by the image determining unit 320. The image enhancer 330 may use a noise removing algorithm stored in the storage 240. The image enhancement unit 330 removes at least one noise of smoke and water vapor generated from the laser surgery, water droplets formed on the lens surface, particles flying during the laser surgery, low contrast and strong reflections, and consequent blurring. Removal can improve the surgical image. The image improving unit 330 is not limited to the noise and may remove noise that is unnecessary for the surgical image.

4 is an exemplary diagram illustrating a kind of noise generated during surgery according to an embodiment of the present invention.

Referring to FIG. 4, the surgical image may include various kinds of noises.

4 (a) is a surgical image including a vapor mist of the lens surface. Water vapor fog noise on the lens surface may be a case where the quality of the surgical image is severely degraded as water vapor is liquefied in the lens due to the temperature difference during the process of entering the body.

4B is a surgical image including water droplets formed on the lens surface. The water droplet noise formed on the lens surface may be a case in which distilled water splashes on the lens surface and forms a water droplet-like pattern during laparoscopic surgery.

Figure 4 (c) is a surgical image including the smoke generated during the laser surgery. The smoke noise generated during laser surgery may be a case where smoke is generated due to laser surgery, resulting in visual discomfort during surgery.

Figure 4 (d) is a surgical image containing particles flying during the laser surgery. Particle noise flying during laser surgery may be a case where particles are generated inside by laser surgery.

4 (e) is a surgical image with low contrast. Low contrast noise may be a case of causing visual inconvenience due to uneven brightness and illumination.

4 (f) is a surgical image that includes strong reflections and consequent bleeding. The strong reflection and the resulting blurring noise may be a case where the surrounding screen is smeared when the surgical tool is reflected by strong lighting during surgery.

5 is an exemplary diagram illustrating image analysis of an image analyzer according to an exemplary embodiment of the present invention, FIG. 6 is an exemplary diagram illustrating image analysis of an image analyzer according to another exemplary embodiment of the present invention, and FIG. An example diagram illustrating image analysis of an image analyzer according to another exemplary embodiment of the present invention.

5 to 7, the image analyzer 310 may calculate a surgical image as a histogram. The image analyzer 310 may calculate a histogram using at least one color model of an RGB format, a YCbCr format, an HSV format, and a gray format.

Figure 5 (a) is a surgical image that does not include noise, Figure 6 (a) is a surgical image including a water vapor mist on the lens surface, Figure 7 (a) is a surgical image including water droplets formed on the lens surface to be.

The image analyzer 310 may calculate a histogram of FIG. 5A. The image analyzer 310 may calculate a histogram of the RGB format of FIG. 5A as shown in FIG. 5B. 5 (b) may be divided into histograms of R 510, G 520, and B 530. Since the surgical image is an internal image of the body, the distribution of R 510 may be larger than that of G 520 and B 530. The image analyzer 310 may calculate the histogram of the YCbCr format of FIG. 5A as shown in FIG. 5C. 5 (c) may be divided into histograms of Y 540, Cb 550, and Cr 560.

The image analyzer 310 may calculate a histogram of FIG. 6A. The image analyzer 310 may calculate the histogram of the RGB format of FIG. 6A as shown in FIG. 6B. 6 (b) may be divided into histograms of R 610, G 620, and B 630. 6 (a) is a bright image as a whole because of the surgical image including the fog. Therefore, the distribution of the histogram may be closer to 255 than to 0 as a whole. The image analyzer 310 may calculate a histogram of the YCbCr format of FIG. 6A as shown in FIG. 6C. 6 (c) may be divided into Y 640, Cb 650, and Cr 660 histograms.

The image analyzer 310 may calculate a histogram of FIG. 7A. The image analyzer 310 may calculate a histogram of the RGB format of FIG. 7A as shown in FIG. 7B. 7 (a) may be divided into R (710), G (720), and B (730) histograms. 7 (a) is a surgical image including a drop of water, it can be seen that the distribution of the histogram is distributed in a specific range. The specific range may be 50 to 70. The image analyzer 310 may calculate a histogram of the YCbCr format of FIG. 7A as shown in FIG. 7C. A histogram of the YCbCr format of FIG. 7A can be calculated as shown in FIG. 7C. FIG. 7C may be divided into a histogram of Y 740, Cb 750, and Cr 760.

8 is a block diagram illustrating an image improving unit according to an exemplary embodiment of the present invention.

Referring to FIG. 8, the image improving unit 330 may remove the type of noise determined by the image determining unit 320. The image improving unit 330 may include a mist processor 810 for removing mist, a droplet processor 820 for restoring a droplet region, a smoke processor 830 for removing smoke, a particle processor 840 for removing particles, and low contrast. It may include at least one of a low contrast processing unit 850 to restore the reflection and blur and processing unit 860 to restore the reflection and blur.

The fog processor 810 may remove smoke by using a temporal median filter and a histogram smoothing method. The fog processor 810 may restore detailed information lost during histogram smoothing using edge information. The fog processor 810 may calculate the edge information by using an image processing method such as a sobel, canny algorithm, or the like.

The droplet processing unit 820 may basically apply three methods to restore the droplet region. The droplet processor 820 may restore the droplet region by using inpainting, image registration, and image contrast increase of the image.

The droplet processor 820 may perform image infancy. The droplet processor 820 restores the droplet region using pixel pixel values around the droplet region.

The droplet processor 820 may perform image registration. The droplet processor 820 may replace the current droplet region with pixel values of the previous image by using previous frame information of the droplet region. Accordingly, the droplet processing unit 820 may use an image feature extraction and matching method such as template matching and optical flow.

The droplet processor 820 may increase an image contrast. The droplet processing unit 820 may apply the method to the droplet region as it is to remove the fog when the droplet region is a simple spreading zero.

The smoke processor 830 may remove the smoke using the same method as the fog removal method. The smoke processor 830 may restore the detailed information lost during the histogram peace by using the histogram smoothing method and the edge information.

Since the particle processing unit 840 suddenly appears and disappears in a small sized particle, the particle processing unit 840 may remove particles that flutter during surgery using a Spatio-temporal median filter in consideration of the above characteristics. The spatio-temporal median filter refers to a filter that takes a median value as a final representative value among all pixels and surrounding pixels in the space and time domain.

The low contrast processor 850 may improve the contrast using a histogram smoothing or histogram correction method. The histogram smoothing is a method of increasing the contrast by extending the range of the surgical image to 0 to 255 range. In addition, the histogram correction method is a method of increasing the contrast in the situation that the minimum value, the maximum value does not change.

The reflection and blur processing unit 860 may improve the image by performing a linear transformation on the pixel because the strong reflection exceeds a certain threshold. The threshold value may be 200 to 210. In addition, the reflection and blur processing unit 860 may restore the detailed information lost after the linear transformation by using the edge information.

9 is an exemplary view showing an image improvement unit of an image improving unit according to an embodiment of the present invention, FIG. 10 is an exemplary view showing an image improvement unit of an image improving unit according to another embodiment of the present invention, and FIG. FIG. 3 is an exemplary diagram illustrating image improvement of an image improving unit according to another exemplary embodiment of the present invention.

9 to 11, the image improving unit 330 may improve a surgical image by removing noise using various methods.

FIG. 9 illustrates removal of smoke and vapor mist from the surface of the lens generated by the laser surgery 330. The image improving unit 330 may display the surgical image 900 as a graph 910 using a cumulative distribution function. The graph 910 of the surgical image is curved in proportion to the graph. Accordingly, the image enhancement unit 330 may remove noise by using a temporal median filter and a histogram smoothing method. The image enhancement unit 330 may restore detailed information lost when the histogram is smoothed by using the edge information. It can be seen that the improved surgical image 920 is clearer than the surgical image 900. In addition, the graph 930 of the cumulative distribution function of the improved surgical image indicates that the color distribution of the surgical image is uniform because the graph is proportional to each other.

FIG. 10 illustrates a spatio-temporal median filter used by the image improving unit 330 to remove flying particles generated during laser surgery. The image improving unit 330 may be removed using a spatio-temporal median filter because the particle size is small and suddenly appears and disappears in the video. The size of the filter may be 3 or 5. The filter 1010 is when the time of the frame is t, the filter 1020 is when the time of the frame is t + 1, and the filter 1030 is when the time of the frame is t + 2. The image improving unit 330 may take the median value among all pixels and neighboring pixels in the space and time domain as the final representative value. Thus, filter 1010, filter 1020, and filter 1030 may take representative values of 150, 144, and 168, respectively.

FIG. 11 illustrates that the image enhancement unit 330 restores low contrast. The image improvement unit 330 may improve the contrast using histogram correction or histogram smoothing. FIG. 11 (a) shows a correction of the histogram, and FIG. 11 (b) shows that the image quality is improved by using edge information after smoothing the histogram.

FIG. 11A illustrates that the image improving unit 330 improves the contrast by correcting the histogram. The image improving unit 330 may convert the surgical image 1110 into a cumulative distribution function graph 1120 and increase the contrast in a situation where the minimum value and the maximum value do not change. Through this, the image improving unit 330 may generate an improved surgical image 1130. Comparing the cumulative distribution function graph 1120 of the surgical image with the cumulative distribution function graph 1140 of the improved surgical image, it can be seen that the graph 1140 is closer to the proportional curve than the graph 1120.

FIG. 11B illustrates that the image improving unit 330 improves the contrast using smoothing of the histogram. The image improving unit 330 may convert the surgical image 1150 into a cumulative distribution function graph 1160 and perform a histogram smoothing 1170 to increase the contrast by extending the range of the image to a range of 0 to 255. However, since the histogram smoothing may cause a missing part, the image improving unit 330 restores the missing part of the histogram smoothing 1170 using the edge information 1180 to generate the final improved image 1190. can do.

12 is a flowchart illustrating a surgical image noise removing method according to an embodiment of the present invention.

Referring to FIG. 12, the surgical image noise removing apparatus 10 may remove noise included in the surgical image.

The surgical image noise removing apparatus 10 receives the surgical image (S100). The surgical image noise removing apparatus 10 may receive the surgical image through the camera controller 130. The surgical image may be a surgical image including noise.

The surgical image noise removing apparatus 10 selects automatic image determination or manual image determination (S110). The surgical image noise removing apparatus 10 may select whether to determine noise included in the surgical image through automatic image determination or determine noise included in the surgical image through manual image determination. Automatic image determination and manual image determination may be performed by selecting at least one of hardware and software.

The surgical image noise removing apparatus 10 analyzes characteristics of the surgical image (S120). In the case of automatic image determination, the surgical image noise removing apparatus 10 may analyze characteristics of the received image. The surgical image noise removing apparatus 10 may calculate at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the surgical image. In addition, the surgical image noise removing apparatus 10 may calculate the size and speed of the moving object.

The surgical image noise removing apparatus 10 determines the noise included in the surgical image based on the analysis result of step S120 (S130). The surgical image noise removing apparatus 10 may determine the type of noise included in the surgical image by using the analyzed surgical image. The surgical image farmland removal apparatus 10 may determine the type of noise according to a preset criterion. The type of farmland may be at least one of water vapor mist on the surface of the lens, smoke generated during laser surgery, water droplets formed on the lens surface, particles flying during laser surgery, low contrast and strong reflection, and consequent bleeding.

The surgical image noise removing apparatus 10 receives a user input (S140). In the case of the manual image determination, the surgical image noise removing apparatus 10 may receive a user input. The surgical image noise removing apparatus 10 may receive a type of noise according to a user's decision. In addition, the surgical image noise removing apparatus 10 may not input the user when the user does not need to improve the image.

The surgical image noise removing apparatus 10 improves the surgical image (S150). The surgical image noise removing apparatus 10 may remove the determined kind of noise. The determined noise may be noise determined by the surgical image noise removing apparatus 10 in step S130, or may be noise determined by a user in step S140. The surgical image noise removing apparatus 10 may improve the surgical image by removing noise related to fog, water droplets, smoke, particles, low contrast, and reflection and bleeding included in the surgical image.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer device is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed fashion.

Although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific preferred embodiments described above, and the present invention belongs to the present invention without departing from the gist of the present invention as claimed in the claims. Various modifications can be made by those skilled in the art, and such changes are within the scope of the claims.

Disclosed are a surgical image noise canceling apparatus, method and system. It may include an input unit for receiving a surgical image and a control unit for removing the noise included in the received surgical image to improve the surgical image. The controller analyzes the characteristics of the surgical image using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and uses the characteristics of the analyzed surgical image to determine the noise of the surgical image. It may further include a function for determining the type. The controller may further include a function of determining the type of noise based on a user's input of the received surgical image.

Claims (13)

An input unit for receiving a surgical image; And And a controller for automatically removing the noise included in the received surgical image to improve the surgical image in real time. The method of claim 1, The apparatus for automatically removing real-time visual noise of a surgical image further comprising a storage unit for storing the improved surgical image. The method of claim 1, The control unit, Analyze the characteristics of the surgical image using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and noise included in the surgical image by using the characteristics of the analyzed surgical image. Apparatus for automatically removing real-time visual noise of a surgical image comprising the function of determining the type of surgery. The method of claim 1, The control unit, And a function of determining a type of noise based on a user's input of the received surgical image. The method of claim 1, The control unit, Each pixel of the received image is calculated using at least one color model of RGB, YCbCr, HSV, and Gray formats to calculate at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation. An image analyzer for analyzing a feature of the image; An image determination unit that determines a type of noise based on a preset criterion by using the analyzed feature of the image; And And at least one of an image improving unit for improving the image by removing the determined kind of noise. The method of claim 5, The image improvement unit, A mist treatment unit for removing mist; A droplet treatment unit for removing droplets; A smoke processor to remove smoke; A particle processing unit for removing particles; A low contrast processor to restore low contrast; And at least one of a reflection and a bleed processing unit for removing reflection and peripheral bleeding caused by the reflection. A surgical image noise removing device according to claim 1; Probes for taking a surgical image; A light source providing light to the probe to adjust brightness of the surgical image; A camera controller which adjusts a camera for photographing the surgical image and provides the surgical image to the surgical image noise removing device; And Automatic visual real-time noise reduction system of the surgical image including a monitor for displaying the improved image in the surgical image noise removing device. The method of claim 7, wherein The surgical image noise removing device is a real-time visual noise automatic removal system of the surgical image, characterized in that detachable. Receiving a surgical image; And Automatically removing the real-time visual noise of the surgical image comprising the step of removing the noise contained in the received surgical image to improve the surgical image. The method of claim 9, Analyze the characteristics of the surgical image using at least one of a histogram, an average value, a maximum value, a minimum value, and a standard deviation of the received surgical image, and noise included in the surgical image by using the characteristics of the analyzed surgical image. And automatically determining the type of the real-time visual noise of the surgical image. The method of claim 9, And automatically determining a type of noise based on a user's input of the received surgical image. The method of claim 9, And automatically storing the improved surgical image in an analog storage device. The method of claim 9, And automatically storing the improved surgical image in a digital storage device.

Images