WO2019107226A1 - Endoscopic apparatus - Google Patents

Abstract

The present invention provides an endoscope navigation device which navigates, by means of a computer, procedures for flexure operations and operations for advancing/rotating an insertion part according to the site of a lumen through which an endoscope is to be passed. Also provided is an endoscopic apparatus with which it is possible to automate, by means of an image processing device and a CPU, adjustment in manipulation quantity of a flexure operation done by an operator while observing a change in viewing field of an endoscopic image. This endoscopic apparatus is equipped with: an imaging device for an endoscopic image; a navigation function which simultaneously or concurrently presents, by way of an image or sound, two or more procedures for inserting an endoscope into a lumen in the body; a means for designating a target on the endoscopic image and inputting a moving vector of the target; and a means for controlling tip position of the endoscope according to the inputted moving vector of the target on the endoscopic image.

Description

Endoscope device

The present invention relates to an endoscope apparatus for performing diagnosis and treatment in the medical field. Further, in the industrial field, the present invention relates to an endoscope apparatus which inspects and repairs a defect of an object in a closed place.

Endoscope devices are conventionally and widely used in the medical and industrial fields. The endoscope apparatus acquires an image of an object at the tip of the insertion unit by the insertion unit, an imaging device provided at the tip of the insertion unit, and the illumination device, and displays the image by the imaging device and the image display device And the operator can perform the inspection. In addition, a bending mechanism is provided at the distal end of the endoscope, and for example, bending operation with two degrees of freedom in the vertical and horizontal directions can be performed. By this bending operation, for example, in the medical field, steering when inserting an endoscope into the digestive tract, control of the visual field when observing or diagnosing a tissue, or when excising a tumor etc. You can control the position and direction of the treatment tool.

Conventionally, when inserting an endoscope into the digestive tract, the operator performs bending operation according to each part of the lumen through which the endoscope passes while grasping the bending state of the endoscope and the state of the insertion portion It is necessary to follow the procedure of the operation of the insertion and withdrawal of the insertion and withdrawal. A great deal of experience was needed to develop these procedures.

For example, Japanese Patent No. 2710384 detects the insertion direction of an endoscope by extracting a dark region of an endoscopic image, and the endoscope operator looks at the detected insertion direction of the endoscope and performs bending operation and insertion. It has been devised to insert an endoscope by automatically performing an operation and inserting an endoscope, or automatically with respect to a detected insertion direction.
Furthermore, for the bending operation, a bending structure consisting of a bending tube connected to the tip, a pulley provided in the operation part at hand, a wire connected to the bending structure and the pulley and disposed along the insertion part It has a drive transmission mechanism which consists of an operation knob fixed to a pulley. The bending operation is performed by rotating the operation knob. In the conventional endoscope, the operator holds the insertion portion of the endoscope with the right hand and advances or retracts the insertion portion while rotating the two operation knobs of the upper and lower and left and right simultaneously with the finger of the left hand to perform bending operation In order to perform the bending operation in the desired direction, it is necessary to be skilled.

For example, in Japanese Patent No. 3353938, in order to improve the operability of the two operation knobs of the upper and lower and left and right sides which require this skill, instead of manual operation by two operation knobs, they are driven by two motors and rotate both motors. It has been devised to move up and down, left and right with one finger using operation means such as a joystick.

For example, in JP-A-2014-109630, a touch panel is provided on a display unit for displaying an endoscopic image in order to move an arbitrary position in the endoscopic image to the central part of the screen quickly and easily. It has been devised to calculate the angle of the bending portion from the touch position information on the screen detected in the above, and operate the bending portion based on this.

Patent 2710384 Patent No. 3353938 JP, 2014-109630, A

Patent 2710384 describes the case where the insertion direction of the endoscope is detected by extracting the dark area of the endoscopic image and the tip is automatically directed to the dark area. As to what input operation is to be directed, there is no description other than turning the tip to the direction of the dark area. In the actual insertion of the endoscope, the dark part may be brought to the center of the endoscopic image or may be brought to the lower side of the endoscopic image. It is necessary to have a means to conduct and operate easily under the recognition.

In addition, the dark part is only one index in endoscope operation, and in addition, the position of air bubbles appearing in the endoscope image, the bending state of the endoscope and the state of the insertion portion are also indexes. It is necessary to follow the procedure of the bending operation and the operation of advancing / retracting / rotation of the insertion part for each part of the lumen through which the lens passes. A great deal of experience was needed to develop these procedures.

Even if the operator uses the joystick disclosed in Japanese Patent No. 3353938 or the touch panel disclosed in JP-A-2014-109630 as the operation means for causing the bending operation, the operator still changes the field of view of the endoscopic image for the following reasons. It is necessary to adjust the amount of operation of the bending operation while watching, which requires skill.

First, when the operator moves the insertion unit back and forth or rotates in observation or treatment with an endoscope, the endoscope image moves, rotates, or changes in size, such as enlargement or reduction of the image. It is necessary to adjust the operation amount of the bending operation while observing the change of the image. If the distance between the object and the tip of the endoscope is changed by advancing and retracting the insertion portion, not only the size on the endoscopic image changes but also the position on the image moves. If the insertion portion is rotated, the endoscopic image rotates in the opposite direction to the rotation direction, and the position on the image of the object moves, so it is necessary to adjust the operation amount of the bending operation while watching these movements .

Second, when performing a bending operation using a touch panel as disclosed in JP-A-2014-109630, the touch position information on the screen detected by the touch panel and the bending angle are determined by the distance between the observation target and the tip of the endoscope. The relationship changes a lot. That is, as the observation target is closer to the distal end of the endoscope, the position on the endoscopic image moves largely due to the bending operation, and as the observation target is farther from the distal end of the endoscope, the position on the endoscopic image is curved due to the bending operation. Move small. Therefore, even if the observation object on the endoscopic image is pointed with the touch panel, the distance between the observation object and the tip of the endoscope can not be known, so it is necessary to calculate the operation amount of the bending operation for moving the observation object on the image Can not.

Thirdly, due to the friction of the bending portion of the bending mechanism of the endoscope and the wire, the amount of bending operation and the change in the field of view of the endoscopic image do not have a linear relationship, so the bending corresponding to the observation site of the endoscopic image The amount of operation can not be calculated accurately. Therefore, it is necessary to adjust the operation amount of the bending operation while observing the change in the field of view of the endoscopic image. For example, when the bending portion is bent 90 ° and 90 ° right at the same time from the straight state, it moves to the middle between 4 o'clock and 5 o'clock as viewed from the front. However, if the bending operation is performed 90 degrees to the right while the bending portion is bending downward by 90 degrees, the bending portion moves in a twisting direction. Even if the curve is manipulated at the same angle, changing the path results in different bending angles. This difference is an example showing a non-linear relationship between the operation amount of the bending operation and the change of the visual field of the endoscopic image. Also in the above reference, it has been necessary for the operator to adjust the operation amount of the bending operation while observing the change in the field of view of the endoscope image.

The computer navigates the procedure of the bending operation and the operation of advancing / retracting / rotation of the insertion part according to each part of the lumen through which the endoscope passes. In addition, information concerning endoscope insertion by navigation is accumulated in a database, machine learning is performed based on the database, and the navigation program is updated to improve accuracy. Furthermore, the image processing apparatus and the CPU automatically adjust the operation amount of the bending operation while the operator looks at the change of the visual field of the endoscopic image, and specifically, the following configuration is It features.

The endoscope apparatus of the first invention is
In the endoscope apparatus,
An imaging device for endoscopic images,
It is characterized by having a function of instructing at least two or more of the procedures for inserting the endoscope into the lumen of the body simultaneously or sequentially.

An endoscope navigation device according to a second aspect of the invention is
An imaging device for endoscopic images,
It is characterized by including a navigation function for presenting at least two or more of the procedures for inserting the endoscope into the lumen of the body simultaneously or sequentially with images or sounds.

An endoscope navigation device according to a third aspect of the present invention is the second aspect according to the present invention,
An image processing apparatus for detecting image information data of an endoscope image;
A function of detecting insertion state data consisting of at least one of a bending state of an endoscope, rotation of an insertion portion, and an insertion length;
According to the image information data and / or the insertion state data, referring to the endoscope insertion procedure data predetermined based on predetermined parameters, at least two or more simultaneously or sequentially with the image or voice insertion procedure And a navigation function to convey information.

A navigation method of an endoscope navigation device according to a fourth aspect of the invention is:
In the endoscope apparatus,
Detecting the insertion state of the endoscope;
Referencing insertion procedure data of an insertion operation predetermined based on the insertion state of the endoscope;
Presenting at least two or more of the endoscope insertion procedures simultaneously or sequentially with images or sounds.

An endoscope navigation device according to a fifth aspect of the invention is
In the endoscope apparatus,
An imaging device for endoscopic images,
An image processing apparatus for detecting image information data of the endoscopic image;
A function of detecting insertion state data consisting of at least one of a bending state of an endoscope, rotation of an insertion portion, and an insertion length;
A navigation function of referring to insertion procedure data of an endoscope determined in advance based on predetermined parameters according to the image information data and / or the insertion state data, and conveying the insertion procedure by image or voice;
A function of recording, as operation data, contents of endoscope operation based on the insertion procedure data;
A function of calculating effect determination result data for measuring / evaluating a result of endoscope operation;
A function of storing the image information data, the insertion state data, the predetermined parameter, the insertion procedure data, the operation data, and the effect determination result data in association with each other in a database;
A function of machine learning the relationship between the data stored in the database;
A function of changing the predetermined parameter and / or the insertion procedure data using the result of the machine learning is characterized.

An endoscope apparatus according to a sixth aspect of the present invention is the endoscope apparatus according to the first aspect, wherein
A means for specifying a target on the endoscopic image;
And means for inputting the movement vector of the target and
And means for controlling the position of the distal end of the endoscope according to the input of the movement vector of the target on the endoscopic image.

An endoscope apparatus according to a seventh aspect of the present invention is the endoscope apparatus according to the sixth aspect, wherein
A means for measuring the movement vector of the target on the endoscopic image,
The input movement vector is compared with the measured movement vector, and feedback control of the position of the endoscope tip is performed so that the difference between the two approaches zero.

An endoscope apparatus according to an eighth invention is any one of the sixth to seventh inventions,
A treatment device is provided at the tip of the endoscope, and movement of the point of action is performed by inputting a movement vector of the point of action designated on the endoscopic image.

An endoscope apparatus according to a ninth invention is any one of the sixth to eighth inventions,
The means for controlling the position of the distal end of the endoscope is any one or more of a bending mechanism provided at the distal end of the endoscope, rotation of the insertion portion, and advancement / retraction of the insertion portion.

An endoscope apparatus according to a tenth invention is any one of the sixth to ninth inventions,
It is characterized in that the designation means of the target on the endoscopic image and the input means of the movement vector are a touch panel and / or a joystick and / or a track ball.

An endoscope apparatus according to an eleventh invention is the seventh invention,
It is characterized in that the means for measuring the movement vector of the target on the endoscopic image is a block matching method, a representative point matching method, or an optical flow method.

The control method of the endoscope apparatus of the twelfth invention is
In the endoscope apparatus,
Specifying a target on the endoscopic image;
Inputting a movement vector of a target on the endoscopic image;
Controlling the position of the tip of the endoscope according to the input movement vector;
Measuring a movement vector of the target on the endoscopic image;
Comparing the input movement vector with the measured movement vector;
And feedback controlling the position of the tip of the endoscope such that the difference between the two approaches zero.

It facilitates endoscope operation, improves the accuracy of endoscopic diagnosis and treatment, and reduces the time without requiring high skill and skill of the operator.

It is a figure showing composition of a system of an endoscope apparatus. It is a figure which shows the structure of an endoscope insertion position sensor. It is a figure which shows the operation interface which touches the dark part in an endoscopic image, and performs bending operation. FIG. 7 is a view showing a menu of navigation in the insertion operation of the large intestine endoscope of the first embodiment. FIG. 8 is a view showing the procedure in the insertion operation of the large intestine endoscope of the first embodiment. FIG. 16 is a diagram showing a flowchart performed in machine learning of navigation in the insertion operation of the large intestine endoscope of the second embodiment. It is a figure which shows the flowchart which performs automatic adjustment of bending operation by the image processing apparatus and endoscopic image of an endoscope image. It is a figure which shows operation of exfoliation of endoscopic submucosal dissection surgery using the endoscope apparatus by this invention of Example 3, and incision. It is a figure which shows the coupling method of the drive transmission system of the endoscope of Example 6, and the structure of an operation part.

Example 1
The present embodiment relates to navigation in insertion of an endoscope. The configuration of this embodiment is shown in FIG. The endoscope apparatus 1 includes an insertion portion 10, an imaging device 2 provided at the tip of the insertion portion 10, an illumination device 3, and an illumination fiber 4 for guiding illumination light at the tip of the insertion portion 10. An image signal of an object ahead of the unit 10 is acquired, an image is constructed by the imaging device 5, and the image is displayed by the display monitor 17. Thus, the operator can view endoscopic images and perform endoscopic diagnosis and treatment. In addition, a bending mechanism 7 is provided at the tip of the endoscope, and, for example, it is possible to perform bending operation with two degrees of freedom, vertically and horizontally. By this bending operation, for example, in the medical field, steering when inserting an endoscope into the digestive tract, control of the visual field when observing or diagnosing a tissue, or when excising a tumor etc. You can control the position and direction of the treatment tool.

For the bending operation, a bending mechanism 7 consisting of a connected bending tube 8 provided at the tip of the endoscope, a pulley 12 provided at the operation unit 11 at hand, and both along the insertion unit 10 It is coupled by the deployed wire 9 and is transmitted by driving. The pulley 12 is connected to the gear 13, the gear 13 is engaged with the gear 14, and the gear 14 is connected to the motor 15. Further, the motor 15 is provided with an encoder 16 for detecting a rotational position.

An endoscope image display monitor 17 is provided near the operation unit 11, and a touch panel 18 is integrated with the display monitor 17. In addition to the display monitor 17, a monitor for displaying an endoscope image may be provided.

The encoder 16 and the motor 15 are electrically connected to the motor control circuit 19. The motor control circuit 19 is connected to the CPU 20. The display monitor 17 is connected to the image processing apparatus 21, and the touch panel 18 is connected to the CPU 20. The signal of the imaging device 5 is connected to the image processing device 21. The image processing device 21 is connected to the CPU 20.

In addition, the insertion position sensor 30 can detect the forward and backward position and the rotation position of the insertion portion 10. The insertion portion position sensor 30 is connected to the CPU 20.

FIG. 2 shows the structure of the insertion portion position sensor 30 for detecting the advancing / retracting position and the rotational position of the insertion portion 10. The illumination 31 and the image sensor 32 are provided on the inner surface of the cylindrical portion into which the insertion portion 10 of the endoscope can be loosely inserted, and the scale 33 and the insertion length marked on the outer periphery of the insertion portion 10 of the endoscope The numeral 34 is read by the image sensor 32, and the insertion position of the insertion unit 10 is detected from the longitudinal position X of the scale line 33 and the numeral 34, and the rotational position of the insertion unit 10 is detected from the position Y in the circumferential direction of the numeral 34. be able to. In FIG. 2, the position Y in the rotational direction of the center of the number "30" and the position X in the longitudinal direction are determined by the image sensor 32, and the insertion length and the rotational position are calculated by the equations in the figure. There is.

The insertion portion position sensor 30 is installed so that the positional relationship with the entrance of the lumen into which the endoscope is inserted does not shift. For example, in the case of the upper endoscope, it is built in or fixed to the mouthpiece to be added to the mouth, and in the case of the colonoscope, it is directly to the patient near the anus or bed. It is fixed through. When the insertion portion position sensor 30 is installed away from the mouth or anus, the offset distance of the insertion length displayed on the display monitor 17 described later can be corrected by inputting the distance to the CPU 20. .

Here, an example is shown in which the advancing / retreating position and the rotation of the insertion unit 10 are detected using the scale line 33 and the numeral 34 indicating the insertion length printed on the insertion unit 10 of the conventional endoscope. The contents described in the insertion section 10 of the endoscope are not limited to this. In order to be able to detect the advancing / retreating position and the rotational position 44 of the insertion unit 10, for example, it may be a symbol that can identify the insertion length instead of a numeral, or a line indicating the rotational direction. . Furthermore, instead of printing on the surface of the insertion portion 10, a Hall element may be embedded, and instead of the image sensor 32, a magnetic sensor may detect the forward and backward position and the rotational position of the insertion portion 10. In fact, since the numbers printed on the insertion section 10 of the conventional endoscope are printed every 10 cm, in order to always recognize the numbers as an image, the length of the insertion section position sensor 30 is at least 10 cm. As described above, printing of numbers at short pitches and providing a symbol for recognizing the length for each scale line shortens the length of the insertion portion position sensor 30, and the mouthpiece for the upper endoscope This contributes to not extending the length of the subject or restricting the working space near the anus in the lower endoscope.

In the hole for inserting the endoscope of the insertion portion position sensor 30, a transparent sleeve-like sterilization cover 35 is detachably provided, and the insertion portion position sensor 30 contaminates the insertion portion 10 of the endoscope. To prevent.

A movement vector (referred to as an input movement vector) when the touch panel 18 is touched and dragged with a finger is calculated by the CPU 20. A table of the correspondence between this input movement vector and the movement vectors of the upper and lower and left and right motors 15 is stored in the internal memory of the CPU 20. Referring to this table, the position and / or speed of the motor control circuit 19 from the CPU 20 Send a command. As a result, the motor 15 is rotated, and the bending mechanism 7 operates via the drive transmission mechanism to move the endoscopic image.

On the other hand, the image processing device 21 and the CPU 20 extract feature amounts of an endoscopic image including the portion touched by the touch panel 18 and the periphery thereof, and obtain movement vectors (referred to as output movement vectors) on the image from the feature amounts. . Furthermore, the CPU 20 calculates a difference between the input movement vector and the output movement vector, and performs feedback control to send a command of position and / or speed to the motor control circuit 19 so that the difference approaches zero.

In the following, “The image processing apparatus 21 and the CPU 20 extract the feature amount of the endoscope image including the portion touched by the touch panel 18 and the periphery thereof and move the object having the feature amount on the image” An example of “determining a vector (output movement vector)” is shown.

For example, it uses a block matching method that is also used for electronic image stabilization of video. The CPU 20 stores an image of a portion including the touch panel 18 touched by the finger and the periphery thereof in the CPU 20, and shifts the superposition position with respect to the image having a change including movement, rotation, and enlargement / reduction by bending operation. The sum of absolute differences of pixels (SAD: Sum of Absolute Difference) is calculated, and the position where the SAD is minimum is detected as the matching position. The movement vector to this matching position is determined as the aforementioned output movement vector.

At this time, pixels having the feature quantities extracted in the endoscopic image are sampled as representative points, and matching is performed by shifting the overlapping positions of the images of the representative points. Can be reduced.

Alternatively, it is possible to use an optical flow mainly used for detection of a moving object, analysis of its operation, and the like. This represents the distribution of movement vectors between frames of representative points on the image, and is used to relate the movement direction and speed of the part including the touch panel 18 touched by the finger and the surrounding area, and the depth. You can get information. That is, the information about the depth of the object can be used as the bending operation of the endoscope by the optical flow.

Subsequently, referring to FIG. 3, the dark portion 53 of the endoscopic image 6 is touched with the touch panel 18, and the touch panel 18 is dragged in a desired direction to perform a bending operation to assist insertion of the endoscope. Will be explained.

For example, in order to insert the colonoscope through the anus and travel through the rectum, the sigmoid colon, the descending colon, the transverse colon, and the ascending colon so as to reverse the movement of the stool, to allow the tip to reach the colon cecum, One indication in the operation of the endoscope is the position of the dark section 53 at a distance from the lumen. The illumination light from the tip of the endoscope is difficult to reach far in the lumen, and as a result, the image becomes darker as it goes further. Endoscope insertion aiming at the dark part 53 is conventionally performed in actual clinical practice using this fact. In the present embodiment, as shown in FIGS. 3A and 3B, the image of the dark portion 53 is touched with the touch panel 18, and the touch panel 18 is dragged in a desired direction to perform a bending operation, as shown in FIG. Assist in the insertion of the mirror.

FIG. 4 exemplifies a navigation menu 56 displayed on the display monitor 17 when the colonoscope is inserted. A colonoscopy model image 40 when the large intestine is viewed from the right side of the body, a colonoscopy model image 41 when the colon is viewed from the front of the body, an insertion unit model image 42 of an endoscope superimposed on the colonoscopy image, The bending state 43, the rotational position 44 and the upward marker 45 of the endoscopic image, the bending arrow 46 instructing the bending operation, the advancing arrow 47 / retraction arrow 48 indicating the advancing / retreating of the insertion portion 10, the insertion portion A rotation arrow 49 for instructing the operation of rotation of 10, an insertion length 50 indicating the length of insertion of the insertion unit 10, and an insertion method selection 51 are displayed together with the endoscopic image 6. These are the rotational position 44 and insertion length 50 of the insertion portion 10 detected by the insertion portion position center 30, and the vertical and horizontal positions of the bending portion detected by the encoder 16 via the motor 15 and the drive transmission mechanism, It is calculated and displayed by the CPU 20 from the bending state 43 and the position of the dark portion 53 and / or the air bubble 54 designated as the target of the endoscopic image 6.

FIG. 5 shows a part of the contents displayed as the navigation menu 56 of FIG. 4 in the step of insertion of the colonoscope. First, select the method of inserting the colonoscope. As a method of inserting a colonoscope, there are a loop insertion method, an axis holding shortening insertion method, and the like. Navigation programs corresponding to these insertion methods are prepared in advance, and first, one of the colonoscope insertion methods is selected. The subsequent steps are an example of navigation when the axis holding shortening insertion method is selected.

In colonoscopy, the patient takes a left-sided decubitus position (left-side down) at the beginning of insertion. First, the tip of the endoscope is passed through the hole of the insertion portion position sensor 30, and insertion is started from the anus. At this time, an arrow is displayed on the display monitor so that the rotational position 44 points in the 9 o'clock direction. For example, as shown in FIG. 5 (1), when the rotational position 44 of the insertion portion 10 detected by the insertion portion position sensor 30 is 12 o'clock, the rotational arrow 49 is displayed to be 9 o'clock. When the operator grips the insertion portion 10 and rotates it in the direction indicated by the rotation arrow 49, “STOP” is displayed as shown in FIG. 5 (2) when the rotation position 44 reaches 9 o'clock. At this time, as shown in FIG. 5 (3), the dark portion 53 is displayed in the endoscopic image 6 at 12 o'clock, and the air bubble 54 is displayed in the 3 o'clock direction.

When the positions of the dark portion 53 and the air bubble 54 are measured by the image processing device 21 and the CPU 20 and it is confirmed that they are in the 12 o'clock and 3 o'clock directions respectively, the insertion portion 10 is advanced as shown in FIG. A forward arrow 47 is displayed. In accordance with this, the operator grips the insertion portion 10 and starts advancing. While keeping the bubble 54 in the direction of 3 o'clock, as shown in FIG. 5 (5), while moving the dark part 53 downward from the upper part of the endoscopic image 6, the forward movement is advanced to insert the insertion length 50 Becomes 13 cm, "STOP" is displayed at the tip of the forward arrow 47 as shown in FIG. 5 (6). At this time, in the endoscopic image 6, as shown in FIG. 5 (7), strong bending and dark portions 53 of the lumen appear in the 7 o'clock direction.

Here, it is confirmed by the image processing of the image processing apparatus 21 that the dark part 53 is in the lower part and the air bubble 54 is in the 3 o'clock direction, and the length inserted from the anus of the insertion part 10 of the endoscope. When it is confirmed that 50 is 13 cm, as shown in FIG. 5 (8), an arrow is displayed on the display monitor so that the rotational position 44 turns to the left at 3 o'clock. When the operator grips the insertion portion 10 and rotates it in the direction indicated by the arrow, “STOP” is displayed as shown in FIG. 5 (9) when the rotational position 44 reaches 3 o'clock. In the endoscopic image 6 at that time, as shown in FIG. 5 (10), the dark portion 53 is displayed at 12 o'clock and the air bubble 54 is displayed at 9 o'clock.

The positions of the dark portion 53 and the air bubble 54 were measured by the image processing device 21 and the CPU 20, and it was confirmed that they were in the 12 o'clock and 9 o'clock directions, respectively. As shown in FIG. The upward direction is presented. Seeing this, the operator touches the dark part 53 above the endoscopic image 6 of the touch panel 18 and drags it downward. The operation of touching and dragging downward is repeated, and when the upward bending reaches 120 °, the upward bending arrow 46 disappears, and "STOP" is displayed as shown in FIG. 5 (12). As a result, as shown in the colon model image 40 seen from the right side of the body in FIG. 5 (13), the bending portion 55 is in a state of being caught in the bending at the start region of the sigmoid colon.

Subsequently, as shown in FIG. 5 (14), a retreating arrow 48 instructing to retreat the insertion part 10 is displayed. When the operator holds the insertion portion 10 and retracts slowly according to this, as shown in the colonoscopy model image 40 seen from the right side of the body in FIG. 5 (15), the colon sigmoid colon gradually approaches the anus side. The dark portion 53 appears on the upper right side or the left side of the endoscopic image 6.

The position of the dark portion 53 is measured by the image processing device 21 and the CPU 20, and the left rotation arrow 49 is displayed when the position of the dark portion 53 is the upper left, and the right is displayed when the position of the dark portion 53 is the upper right. A rotating arrow 49 is displayed. FIG. 5 (16) shows an example in which the right rotation arrow 49 is displayed when the dark part 53 appears on the upper right side. The operator applies a right twist in accordance with the direction of the rotation arrow 49 of the insertion portion 10 as shown in FIG. 5 (17) while continuing to retract the insertion portion 10 slowly. As a result, as shown in the colon model image 40 viewed from the right side of the body in FIG. 5 (18), the sigmoid colon of the large intestine is attracted and the lumen of the downstream portion of the sigmoid colon is opened.

Finally, as shown in FIG. 5 (19), in the endoscopic image 6, the dark part 53 appears to be widely opened and moved to the center. When the image processing apparatus 21 and the CPU 20 measure this state, that is, that the dark portion 53 is at the center and that the air bubble 54 is in the direction of 9 o'clock, as shown in FIG. 5 (20) 49 and the backward arrow 48 disappear, and "STOP" is displayed.

The above steps show an example of the navigation of the present invention in the case of the insertion of the sigmoid colon, which is considered to be the most difficult in the insertion of the colonoscope. The insertion is further advanced, and the appropriate navigation of the operation is performed in the same manner as described above until the cecal is finally reached. Conventionally, the operator judges the bending operation and the rotation and advancing / retracting operation of the insertion unit 10 in light of experience with the information on the position of the dark portion 53 and the position of the air bubble 54 and the rotation and length of the insertion unit 10. However, by navigating the procedure of determination and operation as shown in FIG. 5, the insertion operation of the endoscope can be supported.

Since the insertion length 50 and the rotational position 44 displayed on the screen can be recorded simultaneously with the endoscopic image 6 of the lesion, for example, when the colonoscopy is performed again for follow-up observation at a later date, the lesion It is useful because the location of can be confirmed accurately and efficiently.

The large intestine model image 40 viewed from the right side of the body displayed in FIG. 4, the large intestine model image 41 viewed from the front of the body, and the insertion model image 42 of the endoscope superimposed on the large intestine image are the large intestine. Image using computer graphics considering the anatomy of Anatomical information such as extension and deformation of the large intestine using data of the insertion length 50 of the insertion portion 10, the rotational position 44, and the bending state 43 and these processes from the insertion start to the current position Is an image created with reference to.

The present embodiment shows an example in which the bending operation is performed by electric drive by the motor 15. However, the bending operation may be performed manually. Further, as a means for detecting the bending state, a sensor may be provided directly on the bending portion.
(Example 2)
The present embodiment relates to machine learning of navigation in the insertion operation of the large intestine endoscope in the first embodiment. As shown in FIG. 6, the navigation program 60 is a program in which an experienced rule accumulated by a doctor skilled in colonoscope insertion is dropped into the program, and specifically, image information of dark parts, air bubbles, and luminal features appearing in an image. Endoscope using predetermined parameters 66 from state data 61 such as the insertion portion and bending shape of the endoscope, patient information, insertion method, body position, operator information, pre-treatment state, etc. Assist in insertion by deriving navigation data 62 which is operation output information such as advancing / retracting / rotation of the insertion portion, bending operation, air supply / water supply / suction operation, abdominal pressure, body position conversion etc. To do.

The navigation program 60 records, as the operation data 67, the contents actually subjected to the endoscope operation according to the navigation data 62, and measures patient pain, insertion time, patient monitor and the like in the process of insertion as a result of the operation These functions are analyzed to calculate effect determination result data 63 for evaluating the performance of the insertion method. Further, the state data 61, the navigation data 62, the operation data 67, and the effect determination result data 63 described above can be accumulated in the database 64 as needed while being associated with each other. The database 64 is installed on the cloud, is connected by a network, and shares the database 64 with a plurality of endoscope apparatuses.

For example, machine learning 65 using an AI algorithm 68 such as Convolutional Neural Network (CNN) 69 is performed on this database, and the relationship between effect determination result data 63, predetermined parameters 66, navigation data 62, and operation data 67. Derive The performance of the navigation program 60 is improved by changing the predetermined parameter 66 so that the effect determination result data 63 becomes high. The means of machine learning 65 is not limited to the above-mentioned CNN.

(Example 3)
The steps for the operator to automatically adjust the operation amount of the bending operation while watching the change in the field of view of the endoscopic image are shown in FIG. The operator first recognizes a target that serves as a guideline for the operation of a tissue, a lumen, etc. appearing on an endoscopic image. In order to move the field of view of the endoscope in a predetermined direction with respect to this target, an operation for advancing / retracting or rotating the insertion portion, or performing a bending operation is performed. Along with this, the field of view moves, but at this time, if the movement of the target, that is, the position or movement vector, is different from that intended by the operator, the image processing is applied to the difference between the intended one and the actual one. Automatically recognize. If the difference is small enough and the adjustment is not necessary, the operation proceeds to the next operation, but if the difference is large and the adjustment is necessary, the adjustment amount by the bending operation is automatically estimated. Subsequently, using this adjustment amount, the bending operation is controlled so that the difference approaches zero.

The configuration of the system of the endoscope apparatus 1 for realizing the steps shown in FIG. 7 is shown in FIG.

The endoscope apparatus 1 includes an insertion portion 10, an imaging device 2 provided at the tip of the insertion portion 10, an illumination device 3, and an illumination fiber 4 for guiding illumination light at the tip of the insertion portion 10. An image signal of an object ahead of the unit 10 is acquired, an image is constructed by the imaging device 5, and the image is displayed by the display monitor 17. Thus, the operator can view endoscopic images and perform endoscopic diagnosis and treatment. In addition, a bending mechanism 7 is provided at the tip of the endoscope, and, for example, it is possible to perform bending operation with two degrees of freedom, vertically and horizontally. By this bending operation, for example, in the medical field, steering when inserting an endoscope into the digestive tract, control of the visual field when observing or diagnosing a tissue, or when excising a tumor etc. You can control the position and direction of the treatment tool.

For the bending operation, a bending mechanism 7 consisting of a connected bending tube 8 provided at the tip of the endoscope, a pulley 12 provided at the operation unit 11 at hand, and both along the insertion unit 10 It is coupled by the deployed wire 9 and is transmitted by driving. The pulley 12 is connected to the gear 13, the gear 13 is engaged with the gear 14, and the gear 14 is connected to the motor 15. Further, the motor 15 is provided with an encoder 16 for detecting a rotational position.

An endoscope image display monitor 17 is provided near the operation unit 11, and a touch panel 18 is integrated with the display monitor 17. In addition to the display monitor 17, a monitor for displaying an endoscope image may be provided.

The touch panel 18 may use a projection capacitance method, an ultrasonic surface acoustic wave method, an analog resistive film method or the like, but in the case of using a latex rubber glove for the operation of the endoscope It is preferable to use an ultrasonic surface acoustic wave system or an analog resistive film system which can be operated even when using a glove. In addition, in the case of using a conductive glove, a projected capacitive type may be used.

The encoder 16 and the motor 15 are electrically connected to the motor control circuit 19. The motor control circuit 19 is connected to the CPU 20. The display monitor 17 is connected to the image processing apparatus 21, and the touch panel 18 is connected to the CPU 20. The signal of the imaging device is connected to the image processing device 21. The image processing device 21 is connected to the CPU 20.

In addition, the advancing / retreating position and the rotational position of the insertion portion 10 can be detected by the insertion portion position sensor 30. The insertion portion position sensor 30 is connected to the CPU 20. In the insertion portion position sensor 30, the illumination and the image sensor are provided on the inner surface of the cylindrical portion into which the insertion portion 10 of the endoscope can be loosely inserted, and the scale written on the outer periphery of the insertion portion 10 of the endoscope The image sensor reads the line 33 and the numeral 34 representing the insertion length, and the insertion position of the insertion portion 10 from the longitudinal position of the scale line 33 and the numeral 34 and the rotational position of the insertion portion 10 from the circumferential position of the numeral 34 Can be detected.

A movement vector (referred to as an input movement vector) when the touch panel 18 is touched and dragged with a finger is calculated by the CPU 20. A table of the correspondence between this input movement vector and the movement vectors of the upper and lower and left and right motors 15 is stored in the internal memory of the CPU 20. Referring to this table, the position and / or speed of the motor control circuit 19 from the CPU 20 Send a command. As a result, the motor 15 is rotated, and the bending mechanism 7 operates via the drive transmission mechanism to move the endoscopic image.

On the other hand, the image processing device 21 and the CPU 20 extract feature amounts of an endoscopic image including the portion touched by the touch panel 18 and the periphery thereof, and obtain movement vectors (referred to as output movement vectors) on the image from the feature amounts. . Furthermore, the CPU 20 calculates a difference between the input movement vector and the output movement vector, and performs feedback control to send a command of position and / or speed to the motor control circuit 19 so that the difference approaches zero.

In the following, “The image processing apparatus 21 and the CPU 20 extract the feature amount of the endoscope image including the portion touched by the touch panel 18 and the periphery thereof and move the object having the feature amount on the image” An example of “determining a vector (output movement vector)” is shown.

For example, it uses a block matching method that is also used for electronic image stabilization of video. The CPU 20 stores an image of a portion including the touch panel 18 touched by the finger and the periphery thereof in the CPU 20, and shifts the superposition position with respect to the image having a change including movement, rotation, and enlargement / reduction by bending operation. The sum of absolute differences of pixels (SAD: Sum of Absolute Difference) is calculated, and the position where the SAD is minimum is detected as the matching position. The movement vector to this matching position is determined as the aforementioned output movement vector.

At this time, pixels having the feature quantities extracted in the endoscopic image are sampled as representative points, and matching is performed by shifting the overlapping positions of the images of the representative points. Can be reduced.

Alternatively, it is possible to use an optical flow mainly used for detection of a moving object, analysis of its operation, and the like. This represents the distribution of movement vectors between frames of representative points on the image, and is used to relate the movement direction and speed of the part including the touch panel 18 touched by the finger and the surrounding area, and the depth. You can get information. That is, the information about the depth of the object can be used as the bending operation of the endoscope by the optical flow.

When matching of the entire endoscopic image is performed, an object consisting of non-moving pixels is recognized as being stationary. A scalpel, a transparent hood, and the like, which will be described later, correspond to those which are stationary because they do not move on the image even if the bending operation of the distal end portion of the endoscope or the insertion / retraction of the insertion unit 10 is performed. The pixel corresponding to this stationary one is not used as a target for obtaining the above-mentioned movement vector.

For example, when dragging the finger, the image of the portion where the touch panel 18 is touched with the finger and the periphery thereof changes not only in parallel movement but also rotation, scaling, deformation, etc. every moment In order to do so, the tracking accuracy can be improved by updating the feature quantity extracted by the image processing for matching at a constant cycle.

The feature amount of the object may be poor. For example, when the endoscope tip is in direct contact with the mucous membrane, the image has the front face, mucous membrane color, and one color, so the feature amount of the object is scarce and the above-mentioned output movement vector can not be determined. In such a case, an image processing error is displayed. Even in the case of an image processing error, since it is necessary to continue the bending operation of the endoscope, the speed of the bending operation at this time is set to be slow in consideration of safety. As described above, the closer the object is from the tip of the endoscope, the larger the position on the endoscopic image is moved when the bending operation is performed. That is, when the object is in contact with the tip of the endoscope, the sensitivity of the bending operation to the movement of the finger dragged by the touch panel 18 should be the smallest, so the speed (sensitivity) of the bending operation at that time is It is reasonable to use in case of errors.

The portion touched by the touch panel 18 with the finger and the range around it can be changed. For example, when the touch panel 18 is touched with two fingers, a feature amount to be matched is extracted from a portion including the area between the two fingers and the periphery thereof. Alternatively, by combining the pressure sensor with the touch panel 18 and detecting the pressing force on the surface of the touch panel 18, change the target peripheral part to be narrow when lightly touched and to be wide when strongly pressed. Can.

The following shows the application in which the invention is used in the medical field.
In recent years, cases of esophagus cancer, gastric cancer and colon cancer that can be detected early as endoscopy has spread are increasing. Early detection will enable minimally invasive treatment with endoscopic submucosal dissection (ESD). It has an electric scalpel knife insulated with a ceramic ball at the tip, a knife whose tip electrode is hooked, or an electrode with a short length to the insulated outer diameter in the channel of the endoscope Peel around and under the lesion by operating the endoscope through a knife or the like. The advantage of this method is that the lesion can be excised in free form and size. On the other hand, the disadvantage is that the bleeding is a lot or it takes time to exfoliate a wide area. The operation of the knife in the ESD is a combination of the advancing and retracting operation of the insertion portion 10 of the endoscope, and the rotation and bending operations, and in order to precisely exfoliate a wide range accurately, the operator can It is necessary to instantaneously repeatedly adjust the operation amount of the bending operation while observing the change of the visual field.

The main steps of the ESD are as follows.
a) Infusion: Infusion of a local injection solution such as hyaluronic acid into the submucosa layer around the lesion using the endoscope injection needle through the channel for the endoscope treatment tool to float the mucosa together with the lesion Do.
b) Incision: Cutting the mucous membrane around the lesion area with an electric knife (connected to a high frequency power supply) so as to surround the lesion
c) Peeling: The knife is peeled little by little under the mucous membrane including the lesion with a knife
d) Excision: As a result of exfoliation, the lesion area is cut off The image captured in the endoscopic image is the tissue such as the lesion, the knife 23 and the transparent hood 22. A part of the transparent hood 22 attached to the tip of the endoscope is captured at four corners of the image. By the way, the visual field of the endoscope and the operation space of the knife 23 can be secured by the transparent hood 22. Since this transparent hood is connected to the endoscope, it does not move on the image even if the bending operation of the distal end portion of the endoscope or the advancing / retracting / rotation of the insertion portion is performed. Also, the knife 23 does not move on the image like the transparent hood 22 unless the length of the portion protruding from the tip of the endoscope is changed or the direction is not rotated.

In the following, an example and the effect of the operating method according to the invention in the case of performing b) incision in the aforementioned ESD will be shown.
As shown in FIG. 8A, the vicinity of the lesion of the endoscopic image 6 displayed on the display monitor 17 is touched with a touch panel, and the position where the knife 23 is inserted approaches the position of the tip of the knife 23 Make your finger drag slowly. Accordingly, the target portion approaches the position of the tip of the knife 23 so as to follow the movement of the finger by the above-described feedback control.

Subsequently, the operator advances the knife 23 to bring it into contact with the mucosal surface, and at the same time, operates the high frequency power supply device to make an incision in the mucous membrane. Next, the finger is released from the touch panel 18, and the part in the direction in which the incision should be made is touched again with a finger, and the finger is slowly dragged so as to be pulled toward the tip of the knife 23 as it is (FIG. 8 (2)).

As shown in FIG. 8 (3), in the case where the incision is advanced to a site far from the tip of the endoscope, it is necessary to push the insertion portion 10 of the endoscope to advance the tip of the endoscope. According to the above-described feedback control, the tip of the knife 23 moves so that the movement of the image at the position of the finger on the touch panel does not occur, and an incision is made.

Similarly, by touching the touch panel 18 with a region in the direction in which the incision is to be advanced and dragging it so as to draw it toward the tip of the knife 23, the entire mucous membrane including the periphery of the tumor as shown in FIG. The circumference can be incised.

Subsequently, when performing c) exfoliation in ESD, as shown in FIG. 8 (5), the touch panel 18 is touched on the image of the incision displayed on the display monitor 17 to cut the site where the incision was made. Drag it to the position of the tip of. Next, as shown in FIG. 8 (6), the high frequency power supply device is operated, the knife 23 is advanced, and exfoliation is advanced while dragging the incised site to the left and right. Subsequently, as shown in FIG. 8 (7), the insertion portion 10 is pushed to advance the tip of the endoscope or the insertion portion 10 is rotated so that the tip of the endoscope is under the mucous membrane from the site where the incision was made. Embark on. Furthermore, as shown in FIG. 8 (8), the finger is moved slowly and repeatedly so as to follow the line of peeling the finger on the touch panel 18, and the peeling is continued until the mucous membrane is separated from the submucosal tissue.

As described above, in the ESD, since the bending operation of the endoscope and the operation of advancing and retracting the insertion portion of the endoscope are performed in combination and performed in combination, the movement of the image originally changes complexly, but the present invention According to the above, it is not necessary for the operator to adjust for complex movement in order to feedback control the bending operation so as to follow the movement vector of the target pointed by the touch panel. In addition, it is possible to intuitively instruct the movement of the action point of the therapeutic device such as a knife on the endoscopic image with the touch panel.

In this embodiment, the movement vector of the target point on the endoscopic image is indicated using the touch panel 18 mounted on the display monitor 17. However, a joystick may be used. In the case of the joystick, the cursor displayed on the display monitor 17 is moved, and the portion pointed by this cursor corresponds to the portion touched by the touch panel 18.

In the present embodiment, an example is shown in which the operator manually performs the advancing and retracting and / or the rotating operation of the insertion unit 10, but these may be electrically controlled.
(Example 4)
The fourth embodiment shows an embodiment in which the endoscope apparatus 1 shown in the third embodiment is a 3D endoscope. The endoscope includes an optical system having parallax, a means for detecting two images by an imaging device, an image processor for constructing a 3D image from two images, and a 3D monitor for displaying a 3D image. Install this 3D monitor near the operation unit. The 3D monitor plays back right-eye and left-eye video images at a high speed, and alternately opens and closes the glasses alternately at the timing synchronized with that, displaying a 3D video frame sequential method, right-eye and left-eye video A polarization method in which images are alternately displayed and the image is viewed stereoscopically with polarizing plate glasses, a naked eye parallax barrier method in which images arranged in a stripe shape are viewed through continuous vertical slits, and the like are used.

The touch panel is mounted on the 3D monitor. The touch panel uses the touch panel described in the third embodiment. Alternatively, a 3D touch panel may be used instead. In this case, the portion pointed by the 3D touch panel is the surface of an object in the 3D space on the extension of the fingertip (the surface of a tissue such as a mucous membrane or a tumor in the case of a digestive tract). In this case, unlike the touch panel described in the third embodiment, a switch serving as a trigger is prepared in order to capture the position indicated by the fingertip as data without touching an actual finger in the air and touching the surface of the panel. When this switch is turned on, the position of the fingertip is recognized and taken as an input of the bending operation.

In this embodiment, three-dimensional position information can be handled by measuring the distance in the depth direction from two images having parallax by triangulation.
(Example 5)
The present embodiment is the same in configuration and operation as the third embodiment, but the application in which the present invention is used relates to the detection of a dark part distant from a lumen in the body in the medical field. For example, in order to insert the colonoscope through the anus and travel through the rectum, the sigmoid colon, the descending colon, the transverse colon, and the ascending colon so as to reverse the movement of the stool, to allow the tip to reach the colon cecum, One indication in the operation of the endoscope is the position of the dark section 53 at a distance from the lumen. The illumination light from the tip of the endoscope is difficult to reach far in the lumen, and as a result, the image becomes darker as it goes further. Endoscope insertion aiming at the dark part 53 is conventionally performed in actual clinical practice using this fact. In the present embodiment, as shown in FIGS. 3A and 3B, the image of the dark portion 53 is touched with the touch panel 18, and the touch panel 18 is dragged in a desired direction to perform a bending operation, as shown in FIG. Support mirror insertion operation.

Similarly, when advancing the bronchoscope to the bronchus or lung lobe bronchus, by using the present invention, the image of the dark part 53 is pointed with the touch panel 18, and the touch panel 18 is dragged in a desired direction, A bending operation can be performed to support the insertion operation of the endoscope.
(Example 6)
The present embodiment relates to a method of coupling a transmission system between the motor 15 and the pulley 12 in the third embodiment. In the third embodiment, the gear size 13 connected to the pulley 12 and the small gear 14 meshing with the gear size 13 are used in the drive transmission mechanism. The gear size 13 is used as the operation knob 24 of the endoscope. The configuration of the sixth embodiment is the configuration of the sixth embodiment. In the conventional endoscope, as described in the background art, the operator performs the bending operation by rotating the two upper and lower and left and right operation knobs 24 with the finger. In the present embodiment, as shown in FIG. 9B, the large gear 13 is fixed to each of the two upper and lower and left and right operation knobs 24 so that the small gear meshes with this. Further, the touch panel 18, the display monitor 17, the motor 15, the encoder 16, the large gear 13, and the small gear 14 are entirely covered with a cover 25 and configured as a single operation unit 26. As shown in FIG. 9A, the operation unit 26 and the operation unit 11 are detachably connected, and the drive transmission system is configured to be detachable between the large gear 13 and the small gear 14. There is. This has the following merits.

First, the endoscope is cleaned and / or disinfected / sterilized using a dedicated endoscope cleaner. As in the present embodiment, when the operation unit 11 and the operation unit 26 are separated when cleaning and / or sterilization / sterilization is performed, the endoscope can be a conventional dedicated endoscope cleaner. On the other hand, as shown in FIG. 9C, the operation unit 26 is covered with the sterile drape 27 when using the endoscope, so that there is no need to perform cleaning and / or sterilization / sterilization.

Second, even if there is a defect in the part other than the endoscope, ie, in the part related to the operation unit, during the diagnosis and treatment with the endoscope, the operation unit 26 is By separating and directly operating the operation knob 24, emergency response can be made.

The operation unit 26 can be rotatably fixed to a gripping arm 28 fixed to a floor or a bed or the like. As a result, when the operator performs an operation, it is not necessary to keep holding the operation unit 11, so fatigue can be reduced.

In the first and second embodiments, the following configuration may be made.
1. An endoscope navigation apparatus according to a third invention, wherein the endoscope image data is a bubble and / or a dark part.
2. An endoscope navigation device according to a second invention and a third invention characterized in that the insertion portion position sensor is provided with a sterilization sleeve for preventing contamination of the endoscope insertion portion.
3. An endoscope navigation apparatus according to a second invention and a third invention characterized in that a position of a lesion can be recorded by displaying and recording an insertion length and a rotational position together with an endoscopic image.
4. It is characterized by comprising a colonoscopy model image created by computer graphic using the insertion state data and operation procedure data from the start of insertion to the present, and an insertion part model image of an endoscope superimposed thereon. An endoscope navigation apparatus according to the second invention and the third invention.
5. The endoscope navigation system according to the second invention and the third invention, characterized in that the insertion operation data includes an insertion portion advancing / retracting / rotating operation, a bending operation, air supply / water supply / suction operation, abdominal pressure, body position change. Station equipment.
6. The endoscope navigation apparatus according to the third invention, wherein the effect determination result data comprises patient pain, partial and entire insertion time, and patient monitor.

The third embodiment may be configured as follows.
7. The means for inputting a target movement vector according to the sixth aspect of the invention is performed by touching and dragging a finger on the touch panel.
8. The touch panel uses an ultrasonic surface acoustic wave method or an analog resistive film method that can be operated even when using a glove.
9. A stationary area is detected by matching of a plurality of consecutive endoscopic images, and a pixel corresponding to the stationary area is not used for the bending operation of the endoscope.
10. In the bending control of the endoscope, an optical flow is used as arithmetic processing means for calculating a movement vector of a target between a plurality of endoscopic images, and information on the depth of the object is used as the bending operation amount of the endoscope.
11. Motorized control is performed to move the insertion part back and forth and / or rotate it.
12. In the case where the feature amount of the object is scarce and the movement of the image can not be adjusted, the speed of the bending operation is switched to the safe speed.

The fourth embodiment may be configured as follows.
13. The endoscope apparatus according to any one of the sixth to ninth inventions, wherein the endoscope image imaging apparatus images a 3D image.
14. The touch panel uses a 3D touch panel.
15. Measure the distance in the depth direction by triangulation from the two images with parallax constituting the 3D image.

The fifth embodiment may be configured as follows.
16. The endoscope apparatus according to any one of the sixth to eighth inventions, wherein the target designated on the endoscopic image is a dark area.

The sixth embodiment may be configured as follows.
17. In the means for changing the position of the endoscope tip, the bending portion and the control motor are mechanically coupled via the drive transmission mechanism of the endoscope, and can be separated along the drive transmission mechanism. An endoscope apparatus according to a sixth aspect of the invention.
18. The gear size is fixed to each of the two upper and lower and left and right operation knobs, and the gear small is engaged with it. The controller from the gear large to the gear small and to that at the time of endoscope cleaning or motor control failure occurs. The endoscope apparatus according to the sixth invention, which is configured to be able to be separated.

Reference Signs List 1 endoscope apparatus 2 imaging element 3 illumination apparatus 4 illumination fiber 5 imaging apparatus 6 endoscope image 7 bending mechanism 8 bending tube 9 wire 10 insertion unit 11 operation unit 12 pulley 13 gear large 14 gear small 15 motor 16 encoder 17 Display monitor 18 Touch panel 19 Motor control circuit 20 CPU
21 image processing device 30 insertion portion position sensor 31 illumination 32 image sensor 33 scale line 34 numeral 35 sterile cover 40 colonoscopy model image 41 viewed from the right side of the body colon vision model image 42 viewed from the front of the body insertion region model image 43 Curved state 44 Rotational position 45 Upward marker 46 Curved arrow 47 Forward arrow 48 Backward arrow 49 Rotational arrow 50 Insertion length 51 Insertion method selection 53 Dark part 54 Air bubble 55 Curved part 56 Navigation menu 60 Navigation program 61 State data 62 Navigation data 63 Effect Judgment result data 64 Database 65 Machine learning 66 Predetermined parameters 67 Operation data 68 AI algorithm 69 Convolutional Neural Network

Claims (12)

In the endoscope apparatus,
An imaging device for endoscopic images,
An endoscope apparatus comprising a function of instructing at least two or more of the procedures for inserting an endoscope into a body lumen simultaneously or sequentially. In the endoscope apparatus,
An imaging device for endoscopic images,
2. The endoscope according to claim 1, further comprising: a navigation function for instructing at least two or more of the procedures for inserting the endoscope into the body lumen simultaneously or sequentially with an image or a sound. Mirror navigation device. An image processing apparatus for detecting image information data of an endoscope image;
And a function of detecting insertion state data consisting of at least one of a bending state of an endoscope, rotation of an insertion portion, and an insertion length,
As the navigation function, according to the image information data and / or the insertion state data, at least two or more images are simultaneously or continuously referred to by referring to endoscope insertion procedure data predetermined based on predetermined parameters. The endoscope navigation device according to claim 2, wherein the insertion procedure is transmitted by voice. In the endoscope apparatus,
Detecting the insertion state of the endoscope;
Referencing insertion procedure data of an insertion operation predetermined based on the insertion state of the endoscope;
A navigation method of an endoscope navigation apparatus, comprising the steps of: presenting at least two or more of the endoscope insertion procedures simultaneously or sequentially with images or voices. In the endoscope apparatus,
An imaging device for endoscopic images,
An image processing apparatus for detecting image information data of the endoscopic image;
A function of detecting insertion state data consisting of at least one of a bending state of an endoscope, rotation of an insertion portion, and an insertion length;
A navigation function of referring to insertion procedure data of an endoscope determined in advance based on predetermined parameters according to the image information data and / or the insertion state data, and conveying the insertion procedure by image or voice;
A function of recording, as operation data, contents of endoscope operation based on the insertion procedure data;
A function of calculating effect determination result data for measuring / evaluating a result of endoscope operation;
A function of storing the image information data, the insertion state data, the predetermined parameter, the insertion procedure data, the operation data, and the effect determination result data in association with each other in a database;
A function of machine learning the relationship between the data stored in the database;
An endoscope navigation apparatus comprising: a function of changing the predetermined parameter and / or the insertion procedure data using a result of the machine learning. A means for specifying a target on the endoscopic image;
Means for inputting a movement vector of the target;
The endoscope apparatus according to claim 1, further comprising: means for controlling the position of the tip of the endoscope in accordance with the input of the movement vector of the target on the endoscope image. A means for measuring the movement vector of the target on the endoscopic image,
7. The endoscope according to claim 6, wherein the input movement vector is compared with the measured movement vector, and feedback control is performed on the position of the endoscope tip so that the difference between the two approaches zero. apparatus. The therapeutic device is provided at the tip of the endoscope, and the movement of the point of action is performed by the input of the movement vector of the point of action designated on the endoscopic image. The endoscope apparatus according to any one. The means for controlling the position of the tip of the endoscope is any one or more of a bending mechanism provided at the tip of the endoscope, rotation of the insertion portion, and advancement / retraction of the insertion portion. The endoscopic device described in any one of the above. The specification means of the target on the endoscopic image and the input means of the movement vector are a touch panel and / or a joystick and / or a track ball, according to any one of claims 6 to 9. Endoscope device. The endoscope apparatus according to claim 7, wherein the means for measuring the movement vector of the target on the endoscopic image is a block matching method, a representative point matching method, or an optical flow method. In the endoscope apparatus,
Specifying a target on the endoscopic image;
Inputting a movement vector of the target on the endoscopic image;
Controlling the position of the tip of the endoscope according to the input movement vector;
Measuring the movement vector of the target on the endoscopic image;
Comparing the input movement vector with the measured movement vector;
Feedback control of the position of the tip of the endoscope so that the difference between the two approaches zero, the control method of the endoscope apparatus.

Images