US20180184883A1

US20180184883A1 - Medical image processing apparatus

Info

Publication number: US20180184883A1
Application number: US15/908,579
Authority: US
Inventors: Hirotake Ando
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2012-06-08
Filing date: 2018-02-28
Publication date: 2018-07-05
Also published as: US20130329024A1; JP2013252387A; US9943213B2

Abstract

A medical image processing apparatus is provided which generates a three-dimensional superposed image showing an inserted medical device in a preoperative image. More specifically, the medical image processing apparatus comprises; an input unit; an image signal generating unit; an image information generating unit which generates image information for acquiring a superposed image having a three-dimensional image of the medical device over a three-dimensional image a preoperative structure; and an output unit.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of co-pending U.S. patent application Ser. No. 13/910,947, filed Jun. 5, 2013, which claims foreign priority benefit of Japanese Patent Application No. 2012-131118, filed Jun. 8, 2012, both which are hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to medical image processing apparatuses which accurately visualize an operating environment for a medical device, such as an endoscope, to be inserted to an object.

Description of the Related Art

It is difficult to insert a surgical tool, such as an endoscope and a catheter, having a complicated form to an intended region through an internal luminal structure such as blood vessel or a body cavity, for example.
Accordingly, Japanese Patent Laid-Open No. 5-177000 discloses creating a map of an internal luminal structure from an image of an internal luminal structure captured with an X-ray or by CT, MRI or the like before an operation and performing navigation for inserting a tip of a surgical tool into an internal desirable position.
For an intraoperative navigation, a position of a tip of a surgical tool may be detected with an X-ray. Alternatively, a transmitting unit may be attached to the tip of the surgical tool and a signal transmitted from the transmitting unit may be received by a receiving unit. The position of the transmitting means is acquired from the received signal in real time. The position may be displayed over the map.
According to the method disclosed in Japanese Patent Laid-Open No. 5-177000, a two-dimensional map is used. Therefore, what kind of attitude a surgical tool has may not be known three-dimensionally.

SUMMARY OF THE INVENTION

According to the present invention, there is provided a medical image processing apparatus including: an input unit used to input change-of-form information on a medical device; an image signal generating unit which generates an image signal from the input change-of-form information; an image information generating unit which generates image information for acquiring a superposed image having a three-dimensional image of the medical device over a three-dimensional image a preoperative structure; and an output unit which outputs information on the superposed image.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a system having a medical image display apparatus according to a first embodiment.

FIG. 2 is a schematic diagram illustrating a configuration of a medical image display apparatus according to the first embodiment.

FIGS. 3A and 3B are schematic diagrams of a curvature deformation unit of a medical device of a system in the medical image display apparatus according to the first embodiment.

FIGS. 4A and 4B are schematic diagrams illustrating the curvature deformation unit having an angle sensor.

FIG. 5 is a schematic diagram illustrating the curvature deformation unit having an attitude sensor.

FIG. 6 is a schematic diagram illustrating the curvature deformation unit having a pressure sensor.

FIGS. 7A and 7B are schematic diagrams illustrating that a part abutted against the curvature deformation unit of an internal structure is corrected to be enlarged by an image correction in a superposed image.

DESCRIPTION OF THE EMBODIMENTS

First Embodiment

A medical image processing apparatus according to a first embodiment of the present invention includes an image information generating unit which causes a display apparatus to display a three-dimensional image of a medical device over a three-dimensional image of a preoperative object (or an object before an operation and/or before an examination).
First, reference numerals in drawings will be described. Reference numeral 1 denotes a medical device, reference numerals 4 and 401 denote curvature deformation units, reference numeral 5 denotes a controller, reference numeral 6 denotes a navigation unit, and reference numeral 101 denotes an internal structure.
FIG. 1 is a schematic diagram of a system including a medical image processing apparatus according to the first embodiment. A medical device 1 is inserted to an object 100 which is a subject human body during an operation. The medical device 1 has an operating unit 3 and an inserting unit 2. The inserting unit 2 has a curvature deformation unit 4 at its leading end. The curvature deformation unit 4 curves (or bends) to the direction of in-plane as illustrated, for example, with an internally-contained wire. When the medical device is an endoscope, the curvature deformation unit may internally contain an image optical system such as optical fiber and/or a treatment tool.
The medical device 1 is connected to the controller 5 through cables 8 and 9. The controller 5 automatically controls operating conditions such as an insertion depth, direction and/or angle of the inserting unit 2 or a position and/or direction of rotation of the medical device 1 itself, in addition to control over a deformation of the curvature deformation unit 4. The controller 5 is connected to the navigation unit 6 that is a medical image processing apparatus according to this embodiment through cables 10 and 11. The navigation unit 6 outputs image information, and a display apparatus 7 receives it and displays the corresponding image. In FIG. 1, the display apparatus 7 three-dimensionally displays images of a blood vessel to be observed, a part of the inserting unit 2, and the curvature deformation unit 4.
FIG. 2 is a schematic diagram illustrating a configuration of the navigation unit 6 that is a medical image processing apparatus according to this embodiment. The navigation unit 6 has a preoperative image input unit, an input unit which receives information from the medical device, an image signal generating unit which generates an image signal from information from the medical device, an image information generating unit, and an output unit which outputs image information generated by the image information generating unit to a display apparatus.
The preoperative image input unit receives a plurality of two-dimensional tomograms captured before an operation. The image information generating unit segments the plurality of input tomograms to generate a three-dimensional image.
A two-dimensional image captured before an operation here may refer to an image captured by an apparatus such as a CT and an MRI, for example. The image is an image of a wider area than an area to be displayed by a display apparatus. The two-dimensional image has in advance a marker that is a coordinate reference for image capturing such that the curvature deformation unit of the medical device may be displayed at a correct position in a preoperative image.
The image signal generating unit generates an image signal on the basis of change-of-form information input from the medical device 1 to the input unit. The change-of-form information from the medical device 1 may refer to tip position information, change-of-angle information, change-of-tilt information, and/or change-of-pressure information, for example, on the curvature deformation unit 4, which may be acquired by insertion of the curvature deformation unit to an object. From at least one information piece of these information pieces input through the controller 5, the image information generating unit generates image information for generating a three-dimensional image of a form of the curvature deformation unit 4 in the medical device 1. The form of the curvature deformation unit 4 may be a thickness of the curvature deformation unit or a contour of the leading end, for example.
Another information piece for identifying a form of the curvature deformation unit may be information known before the curvature deformation unit is inserted to an object, such as a structure of the curvature deformation unit, an angle of a curvature deformation, and a range that may be captured by an imaging optical system if the curvature deformation unit has one. These known information pieces may be input from the controller 5 to the navigation unit 6 or may be directly input to the navigation unit through an input unit, not illustrated, without through the controller 5.
The image information generating unit generates image information for displaying an image (superposed image) having a three-dimensional image of the curvature deformation unit 4 over a preoperative three-dimensional image of an object. Image information on a superposed image is transmitted from the output unit to the display apparatus, and the display apparatus displays a three-dimensional image that is the superposed image.
The display apparatus may be a display for a PC, for example. The navigation unit 6 generates image information on a rotatable and translational image from a viewpoint requested by an observer.
Thus, the medical image processing apparatus according to this embodiment may three-dimensionally display a figure of the medical device over a three-dimensional image generated on the basis of a preoperative image.
The figure of the medical device to be displayed in the medical image processing apparatus according to this embodiment is a form of the curvature deformation unit of the medical device as described above. The figure of the medical device to be displayed in the medical image processing apparatus according to the present invention may include both figures of the curvature deformation unit and the inserting unit. The figure of the inserting unit may be generated from information acquired by insertion of the curvature deformation unit or known information, like a figure of the curvature deformation unit.
In a system having a medical image processing apparatus according to this embodiment, the controller 5 may automatically perform operations on a position, a depth, an angle and/or a direction, for example, of the medical device itself, the inserting unit or the curvature deformation unit. On the other hand, in a system according to the present invention, one of those operations may be performed manually by an operator.
A system having a medical image processing apparatus according to this embodiment may perform an operation on a subject human body as the object 100. On the other hand, a system having a medical image processing apparatus according to the present invention may perform an operation on an animal as its object. A system having a medical image processing apparatus according to this embodiment may perform an operation on a subject human body as the object 100, and the medical device may be inserted into the illustrated body cavity. On the other hand, a system having a medical image processing apparatus according to the present invention may be inserted to the body from the mouth, an ear or the like or may be inserted to a lumen such as a gap in the brain, a colon, or a blood vessel.
In a system having a medical image processing apparatus according to this embodiment, information pieces as described above may be moved (received and/or transmitted) through a cable or by radio between the operation controller 5 of the medical device and the navigation unit 6.
In a system having a medical image processing apparatus according to this embodiment, information may be moved through a cable or by radio between the navigation unit 6 and the display apparatus 7.
In a system having a medical image processing apparatus according to this embodiment, information pieces as described above may be moved through a cable or by radio between components of the navigation unit 6.
FIGS. 3A and 3B are schematic diagrams illustrating a part (rigid unit) 21 and the curvature deformation unit 4 at its tip of the inserting unit of the medical device 1 in a system having the medical image processing apparatus. FIG. 3A illustrates the curvature deformation unit 4 that is bending, and FIG. 3B illustrates the curvature deformation unit 4 that is straight and is not bending.
The curvature deformation unit 4 has a plurality of node rings 41 in a longitudinal direction. A fulcrum 42 is provided between the node rings 41. The curvature deformation unit 4 is curved (rotated) about the fulcrums.
The straight state in FIG. 3B according to this embodiment is an initial state. The controller 5 recognizes the state as the initial state and recognizes a state resulting from a curvature deformation two-dimensionally, that is, simply recognizes a form of the state.
From the state, the position where the medical device 1 is mounted (or a tilt or a distance from an object, for example) and a marker on a preoperative image, the navigation unit 6 calculates the position of the medical device on a three-dimensional image for registration, and the image information generating unit generates superposed image information.
FIGS. 4A and 4B to FIG. 6 are schematic diagrams illustrating configurations of the various curvature deformation units 4. FIGS. 4A and 4B illustrate a configuration in which the curvature deformation unit has a potentiometer and which provides change-of-angle information. FIG. 5 illustrates a configuration in which the curvature deformation unit has an attitude sensor and which provides information on a change of direction or attitude. FIG. 6 illustrates a configuration in which the curvature deformation unit has a pressure sensor and which provides change-of-pressure information.
The configurations of the various curvature deformation units will be described with reference to FIGS. 4A and 4B to FIG. 6.
As illustrated in FIG. 4A, the curvature deformation unit has a plurality of node rings 41 and a potentiometer 441 at each fulcrum therebetween. Each of the potentiometers 411 detects an angle of the corresponding node ring as a value of resistance. FIG. 4B illustrates an equivalent circuit of three potentiometers illustrated in FIG. 4A. The three potentiometers 441 give voltages based on the corresponding angles as change-of-angle information to the controller 5.
As illustrated in FIG. 5, the curvature deformation unit has a plurality of node rings 41, and each of the node rings has an attitude sensor 442 at a same position on its side. The attitude sensor 442 may be a triaxial acceleration sensor, for example. The attitude sensor 442 may detect the direction of gravity in three directions applied to the sensor, that is, a tilt of the corresponding node ring (tilt of the attitude sensor) and give change-of-attitude information to the controller 5. Because the three-dimensional directions of the node rings 41 is provided by the attitude sensors, the controller 5 may grasp the attitude of the curvature deformation unit three-dimensionally rather than two-dimensionally. The change-of-attitude information may also be useful for generating a three-dimensional image fast and accurately by the navigation unit 6 even when the controller 5 does not grasp the attitude three-dimensionally.
Because the three-dimensional directions of the node rings 41 are provided by the attitude sensors, the attitude of the curvature deformation unit may be detected three-dimensionally even when the curvature deformation unit only moves two-dimensionally as in this embodiment. Thus, the attitude may be detected even when the curvature deformation unit 4 is compressed by an internal structure and is tilted to a direction where it is not operable.
The attitude sensor may be an encoder, an optical sensor or the like instead of an acceleration sensor.
As illustrated in FIG. 6, the curvature deformation unit has a plurality of node rings 41 and a potentiometer 441 and a pressure sensor 443 therebetween. The pressure sensor 443 outputs a voltage in accordance with a pressure. The pressure sensor may be a piezoresistive pressure sensor which is produced by an MEMS technology, for example.
The change-of-pressure information on the curvature deformation unit is given to the controller 5. On the basis of the change-of-pressure information, operation control information may be transmitted from the navigation unit 6 to the controller 5. For example, when the controller 5 inserts the curvature deformation unit automatically, the navigation unit 6 determines that the curvature deformation unit is in contact with an internal structure on the basis of the change-of-pressure information. How much the curvature deformation unit is to be retracted may be instructed through the controller 5.

Second Embodiment

A medical image processing apparatus according to a second embodiment of the present invention is a medical image processing apparatus which corrects an image of an abutted part of an image corresponding to an internal structure in a superposed image to enlarge the image of the form of the internal structure in the abutted part when the curvature deformation unit is abutted against the internal structure. Except for this, the second embodiment is the same as the first embodiment.
FIGS. 7A and 7B are schematic diagrams illustrating three-dimensional images based on image information output by the medical image processing apparatus according to this embodiment. FIGS. 7A and 7B illustrate superposed images in which images of an internal structure and a curvature deformation unit are superposed. Reference numeral 20 denotes an inserting unit, reference numeral 401 denotes a curvature deformation unit, and reference numeral 101 denotes an internal structure. FIG. 7A is a schematic diagram illustrating a tip of the curvature deformation unit immediately before it is abutted against an internal structure. FIG. 7B is a schematic diagram illustrating an image corrected to enlarge the image of the abutted internal structure.
The internal structure and the curvature deformation unit may be abutted against each other when insertion of the automatically controlled medical device gets slower during an operation. Thus, when the insertion condition changes, the fact is determined by the navigation unit on the basis of the change-of-form information on the medical device, and the image of the abutted internal structure (within a dotted circle) is corrected to be enlarged.
More specifically, if the tip of the curvature deformation unit having a pressure sensor in its leading part detects a pressure and it is known that an internal structure exists near there from a preoperative image or a preoperative three-dimensional image before a superposed image, the image is corrected to enlarge the image of the internal structure.
Thus, when the structure of an object to which the medical device is inserted is different from the structure grasped from a preoperative image in their sizes and/or positions, the image correction may provide a more accurate image thereof.
Having described this embodiment in which a medical device having a curvature deformation unit having a pressure sensor at its tip, a medical image processing apparatus according to the present invention may have pressure sensors at a plurality of parts such as a tube part and a stick part of the curvature deformation unit, and an image of an internal structure may be corrected to be enlarged from pressures on the tube part and/or stick part.
According to this embodiment, when the curvature deformation unit is abutted against an internal structure, an image correction is performed to enlarge the image of the contact part of the internal structure. On the other hand, a medical image processing apparatus according to the present invention may perform an image correction to delete or reduce an image of a part of an internal structure expected from a preoperative image to be abutted as a result of the insertion or deformation of the curvature deformation unit if it is not abutted against the part.
As specifically described with reference to the embodiments, a medical image processing apparatus of the present invention may provide a superposed image in which a three-dimensional image of the medical device is over a three-dimensional image of a preoperative structure. Thus, an accurate three-dimensional image based on information may be provided.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2012-131118, filed Jun. 8, 2012, which is hereby incorporated by reference herein in its entirety.

Claims

What is claimed is:

1. A medical image processing apparatus, comprising:

an image information generating unit configured to generate image information for acquiring a superposed image having a three-dimensional image of a medical device over a three-dimensional image of a structure; and

an output unit configured to output the image information,

wherein the image information generating unit corrects the structure in the superposed image on a basis of a state of contact, detected by a detector of the medical device, of the structure and the medical device, and

wherein the output unit outputs the corrected image information.

2. The medical image processing apparatus according to claim 1,

wherein, in a case where the contact of the structure and the medical device is detected by the detector, the image information generating unit corrects the structure in the superposed image in a manner that the structure approaches the medical device.

3. The medical image processing apparatus according to claim 2,

wherein the image information generating unit performs correction in a manner as to enlarge the structure in the superposed image.

4. The medical image processing apparatus according to claim 3,

wherein the image information generating unit performs the correction in a manner that, of the structure, a portion corrected to be enlarged and other portion are displayed continuously.

5. The medical image processing apparatus according to claim 1,

wherein, in a case where non-contact of the structure and the medical device is detected by the detector, the image information generating unit performs correction to reduce visibility of the structure in the superposed image.

6. The medical image processing apparatus according to claim 5,

wherein the image information generating unit reduces the visibility of the structure in the superposed image either by reducing size of the structure or by deleting the structure.

7. The medical image processing apparatus according to claim 1,

wherein the detector is a pressure sensor; and

wherein the state of contact is acquired on a basis of change-of-pressure information detected by the pressure sensor.

8. The medical image processing apparatus according to claim 1,

wherein the medical device includes a curvature deformation unit.

9. The medical image processing apparatus according to claim 8,

wherein the curvature deformation unit has a serially aligned structure of node rings in an axial direction of the node rings.

10. The medical image processing apparatus according to claim 8,

wherein the detector is provided on the curvature deformation unit.

11. The medical image processing apparatus according to claim 8,

wherein the detector is an attitude sensor configured to detect attitude of the curvature deformation unit.

12. The medical image processing apparatus according to claim 8, further comprising:

a control unit configured to control deformation of the curvature deformation unit.

13. The medical image processing apparatus according to claim 12,

wherein the control unit communicates with the medical device wirelessly.

14. The medical image processing apparatus according to claim 1,

wherein the three-dimensional image of the structure includes a marker; and

wherein the image information generating unit generates the image information by using the marker.

15. The medical image processing apparatus according to claim 1,

wherein a viewpoint of the superposed image is changeable.

16. The medical image processing apparatus according to claim 1, further comprising:

a display unit configured to display the corrected image information.

17. An image information generation method of generating image information for acquiring a superposed image having a three-dimensional image of a medical device over a three-dimensional image of a structure, comprising:

correcting the structure in the superposed image on a basis of a state of contact, detected by a detector of the medical device, of the structure and the medical device, and

outputting the corrected image information.

18. The image information generation method according to claim 17,

wherein, in the correction, in a case where the contact of the structure and the medical device is detected by the detector, the structure in the superposed image is corrected in a manner that the structure approaches the medical device.

19. The image information generation method according to claim 18,

wherein the correction is performed in a manner as to enlarge the structure in the superposed image.

20. The image information generation method according to claim 19,

wherein the correction is performed in a manner that, of the structure, a portion corrected to be enlarged and other portion are displayed continuously.

21. The image information generation method according to claim 17,

wherein, in a case where non-contact of the structure and the medical device is detected by the detector, the correction is performed to reduce visibility of the structure in the superposed image.

22. The image information generation method according to claim 21,

wherein the visibility of the structure in the superposed image is reduced either by reducing size of the structure or by deleting the structure.

23. The image information generation method according to claim 17,

wherein the three-dimensional image of the structure includes a marker; and

wherein the image information is generated by using the marker.

24. The image information generation method according to claim 17,

wherein a viewpoint of the superposed image is changeable.