JP7521629B2 - IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS, IMAGE PROCESSING PROGRAM, AND BLOOD VESSEL DIAMETER CALCULATION APPARATUS - Google Patents

Description

The present invention relates to an image processing method, an image processing device, an image processing program, and a blood vessel diameter calculation device.

Patent Document 1 discloses a method for visualizing the choroidal vascular network. There has been a demand for obtaining the diameter of choroidal blood vessels.

US Patent Application Publication No. 2016/0135683

The image processing method of the first aspect of the technology disclosed herein acquires a choroidal vessel image, identifies multiple vascular center points of the choroidal vessels from the choroidal vessel image along the direction of the choroidal vessels, and calculates the vascular diameter for each of the identified multiple vascular center points.

The image processing method of the second aspect of the technology disclosed herein acquires a choroidal blood vessel image, identifies multiple vascular center points of the choroidal blood vessels from the choroidal blood vessel image along the running direction of the choroidal blood vessels, calculates the vascular diameter for each of the identified multiple vascular center points, identifies the closest vascular center point from among the multiple vascular center points for each pixel of the choroidal blood vessel image, and stores in a storage medium each pixel of the choroidal blood vessel image, the identified vascular center point for that pixel, and the calculated vascular diameter for each vascular center point in correspondence with each other.

The image processing device of the third aspect of the disclosed technology includes a choroidal vessel image acquisition unit that acquires a choroidal vessel image, an identification unit that identifies multiple vascular center points of the choroidal vessels from the choroidal vessel image along the running direction of the choroidal vessels, and a calculation unit that calculates the vascular diameter for each of the identified multiple vascular center points.

The image processing device of the fourth aspect of the technology disclosed herein includes a choroidal vessel image acquisition unit that acquires a choroidal vessel image, an identification unit that identifies a plurality of vascular center points of each of the plurality of choroidal vessels along the running direction of each of the plurality of choroidal vessels from the choroidal vessel image, a calculation unit that calculates a vascular diameter for each of the plurality of identified vascular center points for each of the plurality of choroidal vessels, and a storage processing unit that stores the identified plurality of vascular center points and the calculated vascular diameter for each of the plurality of choroidal vessels in a storage medium.

The image processing device of the fifth aspect of the disclosed technology includes a choroidal vessel image acquisition unit that acquires a choroidal vessel image, a first identification unit that identifies a plurality of vascular center points of the choroidal vessels from the choroidal vessel image along the running direction of the choroidal vessels, a calculation unit that calculates the vascular diameter for each of the identified plurality of vascular center points, a second identification unit that identifies the closest vascular center point from among the plurality of vascular center points for each pixel of the choroidal vessel image, and a storage processing unit that stores in a storage medium each pixel of the choroidal vessel image, the identified vascular center point for that pixel, and the calculated vascular diameter for each vascular center point in correspondence with each other.

The image processing program of the sixth aspect of the disclosed technology causes a computer to execute the image processing method of the first or second aspect.

The vascular diameter calculation device according to the seventh aspect of the disclosed technology includes a storage medium that stores data on multiple vascular center points of choroidal blood vessels and data on the vascular diameter of each vascular center point in correspondence with each other, and a control unit that calculates the vascular diameter of the choroidal blood vessels based on the data on multiple vascular center points of the choroidal blood vessels stored in the storage medium and the data on the vascular diameter of each vascular center point.

The vascular diameter calculation device of the eighth aspect of the disclosed technology includes a storage medium that stores data on a plurality of vascular center points of each of a plurality of choroidal vessels and data on the vascular diameter of each of the vascular center points, and a control unit that calculates the vascular diameter of each of a plurality of choroidal vessels based on the data on the plurality of vascular center points of each of the choroidal vessels stored in the storage medium in correspondence with each of the plurality of choroidal vessels, the data on the vascular diameter of each of the vascular center points, and the data on the vascular diameter of each of the vascular center points.

The ninth aspect of the disclosed technique is a vascular diameter calculation device that includes a storage medium that stores corresponding data of each pixel of a choroidal vessel image, data of a plurality of vessel center points of the ocular vessels in the choroidal vessel image, and data of the vessel diameter of each vessel center point, and a control unit that calculates the vessel diameter of the choroidal vessel at a location indicated in the choroidal vessel image displayed on a display unit based on the data of each pixel of the choroidal vessel image stored in correspondence with the storage medium, the data of a plurality of vessel center points of the ocular vessels in the choroidal vessel image, and the vessel diameter data of each vessel center point.

FIG. 1 is a block diagram of an ophthalmology system 100. 1 is a schematic diagram showing the overall configuration of an ophthalmic apparatus 110. FIG. FIG. 2 is a block diagram of the electrical system configuration of the management server 140. FIG. 2 is a block diagram of the electrical configuration of the image viewer 150. FIG. 2 is a functional block diagram of a CPU 262 of the management server 140. 13 is a flowchart of an image processing program. 7 is a flowchart of a process for calculating a blood vessel diameter in step 308 of FIG. 6. 8 is a flowchart of the process of step 322 in FIG. 7. 8 is a flowchart of the process of step 324 in FIG. 7. 9B is a flowchart of a subroutine for searching for a blood vessel center point in step 1340 of FIG. 9A. 8 is a flowchart of the process of step 326 in FIG. 7. 7 is a flowchart of a process for creating a database of choroidal vessel diameters in step 309 of FIG. 6. 1A to 1C are diagrams showing various UWF-SLO images obtained by an ophthalmic apparatus 110. FIG. 13 is a diagram showing a choroidal blood vessel image CLA. FIG. 13 is a diagram showing a choroidal vessel image CLE obtained by processing a choroidal vessel image CLA to cut out a fundus region (removing eyelids, etc.). FIG. 13 is a diagram showing a vascular skeleton image VS. FIG. 13 is a diagram showing a vascular skeleton map VSM. FIG. 1 shows a vascular network VN. FIG. 2 shows a portion of a vascular network VN. FIG. 13 is a diagram showing a vascular skeleton image. FIG. 13 is a diagram showing the vascular diameter at the center point of the choroidal blood vessel in a binary image of the choroidal blood vessel projected onto the simulated ocular surface. FIG. FIG. 13 is a diagram showing the vascular diameter at each vascular center point of the choroidal blood vessel in the binarized choroidal blood vessel image. FIG. 13 is a diagram showing the relationship between positions P1, P2, . . . corresponding to blood vessel center points and pixels of a binarized choroidal blood vessel image. FIG. 13 is a diagram showing how pixels G1 to G6 of a binarized choroidal blood vessel image are associated with the number of a position P1 corresponding to a blood vessel center point and the blood vessel diameter. FIG. 13 is a diagram showing how pixels G7 to G12 of a binarized choroidal vessel image are associated with the number of a position P2 corresponding to a vessel center point and the vessel diameter. FIG. 1 shows a database of choroidal vessel diameters. FIG. 13 shows another database of choroidal vessel diameters. A diagram showing a display screen 800. 8 is a diagram showing a state in which an icon 844H is operated and choroidal vessels having a vessel diameter of 600 μm or more are displayed in the vessel image display field 834. FIG. FIG. 13 is a diagram showing a state in which the vascular diameter of a choroidal vessel indicated by a cursor is displayed. 8 is a diagram showing a state in which the vascular diameter of a specified choroidal vessel is displayed in a graph 838. FIG. FIG. 8 shows a second display screen 800A. FIG. 7 is a flowchart showing the process of calculating the blood vessel diameter in step 308 of FIG. 6 according to the second embodiment. 9 is a diagram showing the pixel range of the vascular region of a choroidal blood vessel 912 and a vascular skeleton image 910. FIG. FIG. 13 is a diagram showing a plurality of divided regions obtained by dividing a choroidal blood vessel 912. FIG. 13 is a diagram showing another plurality of divided regions obtained by dividing the choroidal blood vessels 912. 13 is a diagram for explaining how to obtain the area of each pixel of a divided region of the choroidal blood vessels 912 on the simulated eyeball surface. FIG.

The following describes in detail an embodiment of the present invention with reference to the drawings.

[First embodiment]

The following describes the artificial eye according to the first embodiment of the present invention with reference to the drawings.

The configuration of an ophthalmologic system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, an axial length measuring device 120, a management server apparatus (hereinafter referred to as "management server") 140, and an image display device (hereinafter referred to as "image viewer") 150. The ophthalmologic apparatus 110 acquires a fundus image. The axial length measuring device 120 measures the axial length of a patient. The management server 140 stores a plurality of fundus images, axial lengths, and tomographic images obtained by photographing the funduses of a plurality of patients by the ophthalmologic apparatus 110 in association with the patient ID. The image viewer 150 displays the fundus image acquired from the management server 140.
The management server 140 is an example of the “image processing device” and the “vascular diameter calculation device” of the technology of the present disclosure.

The ophthalmic device 110, axial length measuring device 120, management server 140, and image viewer 150 are interconnected via network 130.

The axial length measuring device 120 has two modes, a first mode and a second mode, for measuring the axial length, which is the length in the axial direction of the test eye 12. In the first mode, light from a light source (not shown) is guided to the test eye 12, and then the interference light between the reflected light from the fundus and the reflected light from the cornea is received, and the axial length is measured based on an interference signal indicating the received interference light. In the second mode, the axial length is measured using ultrasound (not shown).

The axial length measuring device 120 transmits the axial length measured in the first mode or the second mode to the management server 140. The axial length may be measured in both the first mode and the second mode, in which case the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length.

Next, the configuration of the ophthalmic device 110 will be described with reference to Figure 2.

For ease of explanation, Scanning Laser Ophthalmoscope will be referred to as "SLO." Optical Coherence Tomography will be referred to as "OCT."

When the ophthalmic device 110 is placed on a horizontal plane, the horizontal direction is the "X direction", the vertical direction to the horizontal plane is the "Y direction", and the direction connecting the center of the pupil of the anterior part of the subject's eye 12 and the center of the eyeball is the "Z direction". Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

The ophthalmic device 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the subject's eye 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. In addition, a tomographic image or an en-face image of the retina created based on the OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

The control device 16 includes a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input/output (I/O) port 16D.

The control device 16 is equipped with an input/display device 16E connected to the CPU 16A via an I/O port 16D. The input/display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from the user. An example of the graphic user interface is a touch panel display.

The control device 16 also includes an image processing device 17 connected to the I/O port 16D. The image processing device 17 generates an image of the subject's eye 12 based on the data obtained by the photographing device 14. The control device 16 is connected to the network 130 via a communication interface (not shown).

As described above, in FIG. 2, the control device 16 of the ophthalmic device 110 is equipped with an input/display device 16E, but the technology of the present disclosure is not limited to this. For example, the control device 16 of the ophthalmic device 110 may not be equipped with the input/display device 16E, but may be equipped with a separate input/display device that is physically independent from the ophthalmic device 110. In this case, the display device has an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image, etc., based on an image signal that is instructed to be output by the display control unit 204.

The imaging device 14 operates under the control of the imaging control unit 202 of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes a first optical scanner 22, a second optical scanner 24, and a wide-angle optical system 30.

The first optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The second optical scanner 24 performs two-dimensional scanning in the X and Y directions with the light emitted from the OCT unit 20. The first optical scanner 22 and the second optical scanner 24 may be optical elements capable of deflecting a light beam, such as a polygon mirror or a galvanometer mirror. Alternatively, a combination of these may be used.

The wide-angle optical system 30 includes an objective optical system (not shown in FIG. 2) having a common optical system 28, and a combining unit 26 that combines the light from the SLO unit 18 and the light from the OCT unit 20.

The objective optical system of the common optical system 28 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it is possible to photograph the retina not only in the center of the fundus but also in the peripheral part of the fundus.

When using a system including an elliptical mirror, the system using the elliptical mirror described in International Publication WO2016/103484 or International Publication WO2016/103489 may be used. The disclosures of International Publication WO2016/103484 and International Publication WO2016/103489 are each incorporated herein by reference in their entirety.

The wide-angle optical system 30 realizes observation of the fundus with a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be photographed by the photographing device 14. The FOV 12A can be expressed as a field of view. In this embodiment, the field of view can be defined by an internal irradiation angle and an external irradiation angle. The external irradiation angle is the irradiation angle of the light beam irradiated from the ophthalmic device 110 to the subject's eye 12, which is defined based on the pupil 27. The internal irradiation angle is the irradiation angle of the light beam irradiated to the fundus F, which is defined based on the center O of the eyeball. The external irradiation angle and the internal irradiation angle are in a corresponding relationship. For example, when the external irradiation angle is 120 degrees, the internal irradiation angle corresponds to approximately 160 degrees. In this embodiment, the internal irradiation angle is 200 degrees.

Here, an SLO fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. Note that UWF stands for Ultra Wide Field.

The SLO system is realized by the control device 16, SLO unit 18, and imaging optical system 19 shown in FIG. 2. The SLO system is equipped with a wide-angle optical system 30, which enables fundus imaging with a wide FOV 12A.

The SLO unit 18 includes a B (blue light) light source 40, a G (green light) light source 42, an R (red light) light source 44, and an IR (infrared (e.g., near-infrared) light) light source 46, as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them to one optical path. The optical systems 48, 50, and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The B light is reflected by the optical system 48, passes through the optical system 50, and is reflected by the optical system 54, the G light is reflected by the optical systems 50 and 54, the R light is transmitted through the optical systems 52 and 54, and the IR light is reflected by the optical systems 52 and 56, and is each guided to one optical path.

The SLO unit 18 is configured to be able to switch between a light source that emits laser light of different wavelengths or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in FIG. 2, three light sources are provided: a light source 42 of G light, a light source 44 of R light, and a light source 46 of IR light, but the technology of the present disclosure is not limited to this. For example, the SLO unit 18 may further include a light source of B light (blue light) and a light source of white light, and may emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

The light incident on the imaging optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the first optical scanner 22. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The light reflected by the fundus passes through the wide-angle optical system 30 and the first optical scanner 22 and is incident on the SLO unit 18.

The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits light other than B light from the posterior segment (fundus) of the subject's eye 12, and a beam splitter 58 that reflects G light and transmits light other than G light from the light that has passed through the beam splitter 64. The SLO unit 18 includes a beam splitter 60 that reflects R light and transmits light other than R light from the light that has passed through the beam splitter 58. The SLO unit 18 includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

The light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light passes through the beam splitter 58 in the case of R light, is reflected by the beam splitter 60, and is received by the R light detection element 74. The incident light passes through the beam splitters 58 and 60 in the case of IR light, is reflected by the beam splitter 62, and is received by the IR light detection element 76. The image processing device 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

As shown in FIG. 12, the UWF-SLO images include a UWF-SLO image (G-color fundus image) 502GG obtained by photographing the fundus in G color, and a UWF-SLO image (R-color fundus image) 504RG obtained by photographing the fundus in R color. The UWF-SLO images include a UWF-SLO image (B-color fundus image) 506BG obtained by photographing the fundus in B color, and a UWF-SLO image (IR fundus image) 508IRG obtained by photographing the fundus in IR.

The control device 16 also controls the light sources 40, 42, 44 to emit light simultaneously. The fundus of the subject's eye 12 is photographed simultaneously with the B light, G light, and R light, thereby obtaining a G-color fundus image 502GG, an R-color fundus image 504RG, and a B-color fundus image 506BG, each of which corresponds to the other positions. An RGB color fundus image is obtained from the G-color fundus image 502GG, the R-color fundus image 504RG, and the B-color fundus image 506BG. The control device 16 controls the light sources 42, 44 to emit light simultaneously, and the fundus of the subject's eye 12 is photographed simultaneously with the G light and R light, thereby obtaining a G-color fundus image 502GG and an R-color fundus image 504RG, each of which corresponds to the other positions. An RG color fundus image is obtained from the G-color fundus image 502GG and the R-color fundus image 504RG.

The UWF-SLO image also includes UWF-SLO image (video) 510ICGG, which was photographed using ICG fluorescence. When indocyanine green (ICG) is injected into a blood vessel, it reaches the fundus, first the retina, then the choroid, and passes through the choroid. UWF-SLO image (video) 510ICGG is a moving image from when indocyanine green (ICG) is injected into a blood vessel and reaches the retina to after it has passed through the choroid.

The image data of the B-color fundus image 506BG, the G-color fundus image 502GG, the R-color fundus image 504RG, the IR fundus image 508IRG, the RGB color fundus image, the RG color fundus image, and the UWF-SLO image 510ICGG are sent from the ophthalmic device 110 to the management server 140 via a communication IF (not shown).

The OCT system is realized by the control device 16, OCT unit 20, and imaging optical system 19 shown in FIG. 2. The OCT system includes a wide-angle optical system 30, and thus allows fundus imaging with a wide FOV 12A, similar to the above-mentioned SLO fundus image imaging. The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

The light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched lights is collimated by the collimating lens 20E as measurement light, and then enters the imaging optical system 19. The measurement light is scanned in the X and Y directions by the second optical scanner 24. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus is entered into the OCT unit 20 via the wide-angle optical system 30 and the second optical scanner 24, and enters the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

The other light emitted from the light source 20A and branched by the first optical coupler 20C is incident on the reference optical system 20D as reference light, and passes through the reference optical system 20D to be incident on the second optical coupler 20F.

These lights incident on the second optical coupler 20F, i.e., the measurement light reflected from the fundus and the reference light, are interfered with by the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. The image processing device 17, which operates under the control of the image processing control unit 206, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B.

Here, an OCT fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-OCT image.

Image data of the UWF-OCT image is sent from the ophthalmic device 110 to the management server 140 via a communication IF (not shown) and stored in the storage device 254.

In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various types of OCT systems, such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT), may also be used.

The axial length measuring device 120 in FIG. 1 has two modes, a first mode and a second mode, for measuring the axial length, which is the length in the axial direction (Z direction) of the subject's eye 12. In the first mode, light from a light source (not shown) is guided to the subject's eye 12, and then the interference light between the reflected light from the fundus and the reflected light from the cornea is received, and the axial length is measured based on an interference signal indicating the received interference light. In the second mode, the axial length is measured using ultrasound (not shown). The axial length measuring device 120 transmits the axial length measured in the first mode or the second mode to the management server 140. The axial length may be measured in the first mode and the second mode, in which case the average of the axial lengths measured in both modes is transmitted to the management server 140 as the axial length. The axial length is stored as patient information in the management server 140 as one of the subject's data, and is also used for fundus image analysis.

Next, the configuration of the electrical system of the management server 140 will be described with reference to FIG. 4. As shown in FIG. 4, the management server 140 includes a computer main body 252. The computer main body 252 includes a CPU 262, a RAM 266, a ROM 264, and an input/output (I/O) port 268. The input/output (I/O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I/F) 258. The storage device 254 is, for example, configured of a non-volatile memory. The input/output (I/O) port 268 is connected to the network 130 via the communication interface (I/F) 258. Therefore, the management server 140 can communicate with the ophthalmic device 110, the axial length measuring device 120, and the image viewer 150. The storage device 254 stores an image processing program, which will be described later. The image processing program may be stored in the ROM 264.
The storage device 254 and the ROM 264 are examples of the "storage medium" of the technology of the present disclosure.

The management server 140 stores the data received from the ophthalmic device 110 and the axial length measuring device 120 in the storage device 254.

Next, the configuration of the electrical system of the image viewer 150 will be described with reference to FIG. 4. As shown in FIG. 4, the image viewer 150 includes a computer main body 152. The computer main body 152 has a CPU 162, a RAM 166, a ROM 164, and an input/output (I/O) port 168. The input/output (I/O) port 168 is connected to a storage device 154, a display 156, a mouse 155M, a keyboard 155K, and a communication interface (I/F) 158. The storage device 154 is, for example, composed of a non-volatile memory. The input/output (I/O) port 168 is connected to the network 130 via the communication interface (I/F) 158. Therefore, the image viewer 150 can communicate with the ophthalmic device 110 and the management server 140.

Next, various functions realized by the CPU 262 of the management server 140 executing the image processing program will be described with reference to Fig. 5. The image processing program has a display control function, an image processing control function, and a processing function. By the CPU 262 executing the image processing program having these functions, the CPU 262 functions as a display control unit 204, an image processing control unit 206, and a processing unit 208 as shown in Fig. 6.
The image processing control unit 206 includes a “choroidal blood vessel image acquisition unit”, a “determination unit”, a “calculation unit”, a “storage processing unit”, a “first determination unit”, a “second determination unit”, and a “control unit”.
This is an example.

Next, the image processing by the management server 140 will be described in detail with reference to FIG. 6. The image processing (image processing method) shown in the flowchart of FIG. 6 is realized by the CPU 262 of the management server 140 executing the image processing program.

In step 302, the image processing control unit 206 acquires the UWF-SLO image from the storage device 254. In step 304, the image processing control unit 206 creates a choroidal vessel image from the acquired UWF-SLO image as follows.

First, we will explain the information contained in the R-color fundus image and the G-color fundus image.

The structure of the eye is such that the vitreous body is covered by multiple layers with different structures. The multiple layers include, from the innermost layer on the vitreous body side to the outermost layer, the retina, choroid, and sclera. Since R light has a long wavelength, it passes through the retina and reaches the choroid. Therefore, the R-color fundus image 504RG includes information on blood vessels present in the retina (retinal blood vessels) and blood vessels present in the choroid (choroidal blood vessels). In contrast, since G light has a shorter wavelength than R light, it only reaches the retina. Therefore, the G-color fundus image 502GG includes only information on blood vessels present in the retina (retinal blood vessels). Therefore, a choroidal blood vessel image CLA (see FIG. 13) can be obtained by extracting the retinal blood vessels from the G-color fundus image 502GG and removing the retinal blood vessels from the R-color fundus image 504RG. Specifically, the choroidal blood vessel image CLA is generated as follows.

The image processing control unit 206 extracts retinal blood vessels from the G-color fundus image 502GG by applying black hat filter processing to the G-color fundus image 502GG. Next, the image processing control unit 206 removes retinal blood vessels from the R-color fundus image 504RG by inpainting processing. That is, the image processing control unit 206 performs processing to paint the retinal blood vessel structure of the R-color fundus image 504RG with the same value as the surrounding pixels using the position information of the retinal blood vessels extracted from the G-color fundus image 502GG.
The image processing control unit 206 then performs contrast limited adaptive histogram equalization on the image data of the red fundus image 504RG from which the retinal blood vessels have been removed, thereby emphasizing the choroidal blood vessels in the red fundus image 504RG. As a result, a choroidal blood vessel image CLA shown in FIG. 13 is created. The created choroidal blood vessel image CLA is stored in the storage device 254.

In the above example, the choroidal blood vessel image is generated from the R-color fundus image 504RG and the G-color fundus image 502GG. However, the image processing control unit 206 may generate the choroidal blood vessel image CLA from the G-color fundus image 502GG and the UWF-SLO image 508IRG. The image processing control unit 206 may generate the choroidal blood vessel image CLA from the B-color fundus image 502BG and one of the R-color fundus image 504RG and the UWF-SLO image 508IRG.

Furthermore, a choroidal blood vessel image CLA may be generated from the UWF-SLO image (moving image) 510. As described above, the UWF-SLO image (moving image) 510 is a moving image from when indocyanine green (ICG) is injected into a blood vessel and reaches the retina to after it has passed through the choroid. A choroidal blood vessel image CLA may be generated from a moving image of the period from when indocyanine green (ICG) passes through the retina to when it passes through the choroid.

By the way, the diameter of choroidal blood vessels is generally larger than that of blood vessels in the retina. Specifically, blood vessels with a diameter larger than a predetermined threshold are choroidal blood vessels. Therefore, in the UWF-SLO image (video) 510, blood vessels may be extracted from an image in which indocyanine green (ICG) is passing through the blood vessels of the retina and choroid, and a choroidal blood vessel image CLA may be generated by eliminating blood vessel diameters smaller than a predetermined threshold.

Since the choroidal blood vessel image may include the eyelids, in step 306, the image processing control unit 206 generates a choroidal blood vessel image CLE (see FIG. 14) in which the fundus region is cut out from the choroidal blood vessel image CLA (removing the eyelids, etc.). The choroidal blood vessel image CLA and the choroidal blood vessel image CLE are images in which the choroidal blood vessels are visualized and obtained by image processing the fundus image.

In step 308, the image processing control unit 206 calculates the vascular diameter of each choroidal vessel in the choroidal vessel image CLE using each of the multiple vascular center points obtained along the vascular running direction. The detailed calculation process of the vascular diameter will be described later.

In step 309, the image processing control unit 206 creates a database of choroidal vessel diameters that includes at least the choroidal vessel diameters and pixel coordinates at the vessel center points calculated in step 308. This database of choroidal vessel diameters is associated with the UWF-SLO fundus image that was the subject of analysis and stored in the RAM 266 or the storage device 254. Details of the database structure will be described later.

In step 310, the display control unit 204 creates a display image of the display screen 800 (see also FIG. 28) showing the results of the analysis using the vascular diameter data calculated in step 308. The layout of the display screen will also be described later.

In step 312, the processing unit 208 transmits the display image data of the display image to the image viewer 150 via the communication I/F 258.

Next, the process of calculating the choroidal vessel diameter will be explained using the flowcharts shown in Figures 7, 8, 9, and 10.

Figure 7 is a main flowchart of the choroidal vessel diameter. In step 322, the image processing control unit 206 creates a vascular skeleton image VS (shown in Figure 15) and a vascular network VN (shown in Figure 17), which will be described in detail later. In step 324, the image processing control unit 206 sets multiple vascular center points determined along the vascular running direction, and in step 326 calculates the vascular diameter for each vascular center point, and finally calculates the vascular diameters of all choroidal vessels in the choroidal vessel image CLE.

Figure 8 shows a flowchart of the process of step 322 in Figure 7.

In step 332, the image processing control unit 206 creates a choroidal vessel binary image (not shown) by binarizing the pixel value of each pixel of the choroidal vessel image CLE based on a predetermined threshold value. This choroidal vessel binary image is an image in which the choroidal vessels are visualized (pixels in areas corresponding to the choroidal vessels are white, and pixels in areas other than the choroidal vessels are black).

In step 334, the image processing control unit 206 thins the binary choroidal vessel image to obtain the center line of each choroidal vessel diameter in the binary choroidal vessel image. This center line is a line with a width of 1 pixel. In other words, a blood vessel skeleton image VS is generated by performing thinning processing to convert the binary image into a line image with a width of 1 pixel.
15 shows a vascular skeleton image VS of one choroidal vessel in the choroidal vessel image CLE. As shown in FIG 15, the vascular skeleton image VS is represented by a plurality of pixels 402, 404, 406, 408, ... indicating the center line of the choroid.

In step 334, the image processing control unit 206 assigns pixel numbers to the multiple pixels 402, 404, 406, 408, etc. that represent the choroidal blood vessels, as shown in FIG. 16, and creates a vascular skeleton map VSM in which pixel positions (or pixel coordinates) correspond to pixel numbers. For example, pixel number 1 is assigned as pixel number 402A1 to pixel 402 in the vascular skeleton map that corresponds to pixel 402 in the vascular skeleton image VS shown in FIG. 15. Also, pixel number 4 is assigned as pixel number 404A to pixel 404 in the vascular skeleton map VSM that corresponds to pixel 404 in the vascular skeleton image VS. The vascular skeleton map VSM shown in FIG. 16 is a display of the vascular skeleton map VSM of area A in FIG. 15. It goes without saying that the vascular skeleton map VSM is created for the entire vascular skeleton image VS.

Next, in step 338, the image processing control unit 206 creates a vascular network VN from the vascular skeleton map VSM. Here, the vascular network VN is a graph consisting of nodes (points) and edges (lines) in which the pixel numbers of the vascular skeleton map VSM are used as nodes and the connections between the pixel numbers are indicated by edges.

The image processing control unit 206 forms pixel number 1 of pixel 402 in the vascular skeleton map VSM as node (1) at a predetermined position (see 402B) in the memory space where the vascular network VN is created.

Next, the image processing control unit 206 searches for a pixel with a pixel number assigned in the vicinity of pixel 402 in the vascular skeleton map VSM. Specifically, first, a pixel with a pixel number is searched for among the four neighboring pixels (four pixels vertically and horizontally, i.e., the four pixels above, below, left, and right of pixel 402). In FIG. 16, there is no pixel with a number assigned among the four neighboring pixels of pixel 402A. Therefore, the image processing control unit 206 next searches for a pixel with a pixel number assigned among the eight neighboring pixels (eight pixels vertically, horizontally, and diagonally adjacent, i.e., the four pixels above, below, left, and right of pixel 402 plus the four pixels diagonally to the left and right). In FIG. 16, pixel number 4 is assigned to pixel 404 diagonally to the upper right of pixel 402. Therefore, the image processing control unit 206 sets pixel number 1 assigned to pixel 402 (see 402B in FIG. 17) and pixel number 4 assigned to pixel 404 as a new node (4). Then, nodes (1) and (4) are connected with an edge (see Figure 18). Taking into account vascular branching, the process ends when no pixel with a pixel number is found. By performing this process on the entire vascular skeleton map VSM, a vascular network VN is created.

After completing step 338 of creating the vascular network VN, the image processing control unit 206 ends the subroutine in FIG. 8 and returns to the main flowchart in FIG. 7.

Figures 9A and 9B show flowcharts of the process of step 324 in Figure 7. Figure 9A is a flowchart for setting the vascular center point, and Figure 9B is a subroutine for searching for the vascular center point.

In step 1300, the image processing control unit 206 projects the blood vessel skeleton image VS onto the simulated eyeball surface. The simulated eyeball surface is a spherical surface equivalent to the eyeball reproduced in the memory space of the RAM 266 by the image processing control unit 206 based on the axial length and age of the subject. The blood vessel skeleton image VS is reverse stereo converted and projected onto this simulated eyeball surface. This is because the UWF-SLO image itself is an image of the eyeball stereo projected onto a two-dimensional plane. Although distortion occurs in the peripheral parts of the UWF-SO image and the blood vessel skeleton image VS due to stereo conversion, the influence of the distortion can be removed by reverse stereo conversion and projecting onto the simulated eyeball surface. Since the blood vessel diameter is calculated in a state where the distortion is removed, a value close to the actual blood vessel diameter can be obtained.
Then, in step 1320, the image processing control unit 206 sets an arbitrary end point of the vascular skeleton image VS as the current search point. The current search point in step 1320 means the search start point in the vascular skeleton image NV, and is the starting point for searching for the vascular center point. To explain using the skeleton image shown in Figure 19, 460N1 indicated by an asterisk (*) on the vascular skeleton image is set as the current search point.

In step 1340, the image processing control unit 206 starts processing the subroutine for searching the vascular center point in FIG. 9B. The subroutine in FIG. 9B is a subroutine that is recursively called in step 1340 in FIG. A and step 1500 in FIG. 9B.

In step 1400, the image processing control unit 206 sets the current search point 460N1 as the vascular center point.

In step 1420, the image processing control unit 206 sets a point on the vascular skeleton image VS that is a certain distance (for example, 20 μm) away from the vascular center point as a vascular center point candidate. Here, the number of "vascular center point candidates" is one if there is no vascular branch within a certain distance from the vascular center point and the vascular center point is continuous as one, and if there is a vascular branch (a blood vessel is divided into two), two are set for each branched blood vessel. If the blood vessel itself does not exist, no vascular center point candidate is set. FIG. 18 is a diagram showing a schematic diagram of the relationship between the vascular network NV, the vascular center point, and the vascular center point candidate. 402B, 404B, 406B, and 408B correspond to node (1), node (2), node (3), and node (5) of the vascular network VN, respectively. The vascular center point 452 has already been set between node (1) and node (4). The blood vessel branches into two at node (4), and the next blood vessel center point candidates are set to be blood vessel center point candidate 1 454 located between nodes (4) and (5), and blood vessel center point candidate 2 456 located between nodes (4) and (3).
In step 1440, the image processing control unit 206 determines the blood vessel running direction between the blood vessel center point and the blood vessel center point candidate. Furthermore, the image processing control unit 206 assigns spherical coordinate data of the blood vessel center point on the simulated eyeball surface, the blood vessel running direction, and a unique blood vessel center point number, and stores them in the RAM 266.

Next, proceed from step 1460 to the loop process of step 1500. The number of times this loop process is executed varies depending on the number of vascular center point candidates set in step 1420. In other words, if there are no vascular branches and the blood vessel is continuous as one, it is executed once, and if there are vascular branches (blood vessels that are divided into two), it is executed twice. In Figure 19, point 460N2 is the vascular center point candidate.

In step 1460, if there is no candidate vascular center point, the image processing control unit 206 does not execute the loop process, ends the subroutine in FIG. 9B, and proceeds to step 1360 in FIG. 9A. In step 1360, the image processing control unit 206 stores data of all set vascular center points (data including at least the spherical coordinate data of the vascular center points on the simulated eyeball surface, the vascular running direction, and the unique vascular center point number) in the RAM 266 or the storage device 254. Since the vascular running direction cannot be calculated for a vascular center point located at the end point of a blood vessel, the vascular running direction of the previous vascular center point may be set.

If a blood vessel center point candidate exists, proceed to step 1480. If a blood vessel center point candidate exists, the image processing control unit 206 sets the blood vessel center point candidate as the current search point in step 1480.

Then, the process proceeds to step 1500, where the image processing control unit 206 recursively executes the subroutine in FIG. 9B. When the process of searching for the vascular center points of the branched blood vessels ends in step 1500, the loop process of the branched blood vessels ends in step 1520, and the process returns to step 1460 to repeat the process of searching for the vascular center point of another branched blood vessel. In this way, the presence or absence of vascular branching is determined by the vascular network NV, and by repeating the loop process, the vascular center points can be set for all blood vessels in the vascular skeleton image VS.

Next, the calculation of the vascular diameter for each vascular center point in step 326 in FIG. 7 will be described. FIG. 10 shows a flowchart of the process of step 326 in FIG. 7. The process of FIG. 10 uses the data of the vascular center points set in the process shown in FIG. 9A and FIG. 9B to calculate the vascular diameter of the choroidal blood vessel in the binarized choroidal blood vessel image (FIG. 14) projected onto the simulated ocular surface for each vascular center point.

Specifically, in step 372, the image processing control unit 206 reads out the data related to the blood vessel center points stored in step 1360 of FIG. 9A. Then, one blood vessel center point is selected, and the spherical coordinate data of the selected blood vessel center point on the simulated eyeball surface and the blood vessel running direction are identified.

In step 374, the image processing control unit 206 calculates a great circle perpendicular to the blood vessel running direction of the selected blood vessel center point. Specifically, as shown in Figures 20 and 21, the image processing control unit 206 calculates a great circle 474 (Figure 21) perpendicular to the blood vessel running direction 460ND at the identified blood vessel center point 460Nn in the choroidal blood vessel binarized image projected onto the simulated eyeball surface. Here, the center of the great circle is the same as the center of the simulated eyeball.

In step 376, the image processing control unit 206 calculates the intersection point between the great circle 474 and the blood vessel edge. Specifically, as described above, the great circle 474 is positioned based on the blood vessel center point 460Nn in the choroidal blood vessel binarized image projected onto the simulated ocular surface. The image processing control unit 206 calculates the intersection points 476A and 476B between the positioned great circle 474 and the blood vessel edge of the choroidal blood vessel in the choroidal blood vessel binarized image on the simulated ocular surface.

In step 378, the distance r between intersection 476A and intersection 476B is calculated and stored in correspondence with the vascular center point of the vascular skeleton image. This distance r is part of the great circle 474 and is the shortest distance on the simulated eyeball surface that passes through vascular center point 460Nn. Therefore, it is possible to obtain a vascular diameter that is close to the shape of the eyeball.

As described above, the process of FIG. 10 is performed on all vascular center points in each vascular skeleton image VS. Therefore, as shown in FIG. 22, the vascular diameter at each vascular center point in the binary image of choroidal blood vessels on the simulated eyeball surface is calculated. The vascular diameter at each vascular center point is associated with the vascular center point number and stored in the RAM 266 or the storage device 254.

Figure 11 shows a flowchart of the choroidal vessel diameter database creation process in step 309 in Figure 6. In step 382, the image processing control unit 206 reads out the stored vessel center number, spherical coordinate data on the simulated eyeball surface, vessel running direction, and vessel diameter data.

In step 385, the image processing control unit 206 labels each pixel representing a blood vessel with the number and direction of the blood vessel center point that is closest to the pixel. Specifically, as shown in FIG. 23, the image processing control unit 206 identifies positions P1, P2, ... corresponding to the blood vessel center points in the choroidal blood vessel binarized image (FIG. 14) (that is, converts the spherical coordinate data indicating the blood vessel center points on the simulated eyeball spherical surface into two-dimensional coordinate data indicating the blood vessel center points on the two-dimensional image above). The image processing control unit 206 identifies the blood vessel center point that is closest to each pixel representing a blood vessel in the choroidal blood vessel binarized image. The number of the identified blood vessel center point and the blood vessel diameter are associated with each pixel (labeling). More specifically, as shown in FIG. 24, the display control unit 204 associates the number (#=1) of the blood vessel center point P1 that is closest to each pixel and the blood vessel diameter (r#1) with pixels G1 to G6 that represent blood vessels in the choroidal blood vessel binarized image. Similarly, as shown in FIG. 25, the display control unit 204 associates the number (#=4) of the blood vessel center point P2 that is closest to each pixel and the blood vessel diameter (r#4) with pixels G7 to G12 that represent blood vessels in the choroidal blood vessel binarized image.

In step 386, the image processing control unit 206 creates a database of choroidal vessel diameters. FIG. 26 shows the database of choroidal vessel diameters. As shown in FIG. 26, the database of choroidal vessel diameters includes a memory area 502 that stores the data numbers of choroidal vessel images. The image processing control unit 206 stores data numbers M1, M2, ... that identify each choroidal vessel image in the memory area 502.

The choroidal vessel diameter database includes a memory area 504 that stores skeleton numbers unique to blood vessels in each vascular skeleton image of a choroidal vessel image identified by the data number of the choroidal vessel image. For example, the image processing control unit 206 stores skeleton numbers S1, S2, ... of blood vessels in each vascular skeleton image of a choroidal vessel image identified by the data number M1 of the choroidal vessel image in the memory area 504 in association with the data number M1. Alternatively, blood vessels may be identified by analyzing the vascular network NV and assigned skeleton numbers.

The choroidal vessel diameter database includes a memory area 506 that stores the vessel length of each vessel. The image processing control unit 206 performs the process of step 1300 in FIG. 9A and calculates and stores the vessel length on the simulated eyeball surface for each blood vessel when a vessel skeleton image is attached to the simulated eyeball surface. The calculated vessel length is then stored in the memory area 506 in association with the skeleton number.

The choroidal vessel diameter database includes a memory area 508 that stores the total number of vessel center points for each vessel. The display control unit 204 counts the vessel center points set in each vessel skeleton image (FIG. 19) (or vessel network (FIG. 17)) and stores the total number in the memory area 508 in correspondence with the vessel length.

The blood vessel diameter database includes a memory area 510 that stores numbers that identify each blood vessel center point in the blood vessel skeleton image. The image processing control unit 206 counts the numbers of each blood vessel center point that exists in each blood vessel, and stores the total number of blood vessel center points in the blood vessel in the memory area 510.

The choroidal vessel diameter database includes a memory area 512 that stores the coordinates of each vessel center point on the SLO image (choroidal vessel image (Figure 13)). The image processing control unit 206 identifies each vessel center point of the vessel skeleton image on the SLO image (choroidal vessel image (Figure 13)) and stores the coordinates of each identified center point in the memory area 512 in correspondence with the vessel center point. Note that in addition to storing the coordinates of each vessel center point on the UWF-SLO image, the coordinates on the choroidal vessel binarized image may also be stored.

The choroidal vessel diameter database includes a memory area 514 that stores the coordinates of each pixel of the choroidal vessel binarized image to which the number of the vessel center point is assigned. The image processing control unit 206 identifies each pixel of the choroidal vessel binarized image to which the number of the vessel center point is assigned, and stores the coordinates of each identified pixel on the choroidal vessel binarized image in the memory area 514 in correspondence with the coordinates of the vessel center point on the SLO image. The coordinates of each pixel match the coordinates on the SLO image (choroidal vessel image (Figure 13)).

The choroidal vessel diameter database includes a memory area 516 that stores the vascular running direction of each vascular center point. The image processing control unit 206 reads out the vascular running direction of the vascular center point from the vascular network (Figure 17), and stores the vascular running direction of the read vascular center point in the memory area 516 in correspondence with each of the coordinates of each pixel of the choroidal vessel binarized image to which the number of the vascular center point has been assigned.

The choroidal vessel diameter database includes a memory area 518 that stores the vessel diameter at the vessel center point. The image processing control unit 206 reads out the vessel diameter at the vessel center point stored in correspondence with the vessel center point of the vessel skeleton image, and stores it in the memory area 518 in correspondence with the vessel running direction.

The choroidal vessel diameter database may be divided into two databases as shown in FIG. 27. The first database includes memory areas 502-512 and 518. The second database includes memory areas 514 and 516.

Then, in step 388 of FIG. 11, the image processing control unit 206 stores the database of choroidal vessel diameters created in step 386 in the RAM 266 or the storage device 254 together with the corresponding SLO images.

Next, we will explain the process of creating the display screen data in step 310 of Figure 6.

Here, the display screen 800 shown in FIG. 28 will be described. FIG. 28 shows the display screen that is displayed when the vascular diameter icon 856, which will be described later, is operated. As shown in FIG. 28, the display screen 800 has a personal information display section 802 that displays the patient's personal information, an image display section 804, and a choroid analysis tool display section 806.

The personal information display field 802 has a patient ID display field 812, a patient name display field 814, an age display field 816, a right eye/left eye display field 818, an axial length display field 820, a visual acuity display field 822, an imaging date and time display field 824, and a patient selection icon 314. Each piece of information is displayed in these display fields 812 to 824. When the patient selection icon (not shown) is clicked, a list of patients is displayed on the display 172 of the image viewer 150, and the user is allowed to select a patient to be analyzed.

The image display field 804 has a choroidal vessel image display field 832, a vessel image display field 834, and a diameter range display field 840 that shows the range of vessel diameters of the choroidal vessels to be displayed in the vessel image display field 834. The image display field 804 has icons 844L, 844M, 844H, and icon 842.

Icon 844L is an icon that allows the user to instruct the vascular image display field 834 to display choroidal vessels with a vessel diameter of less than 300 μm in a first color. Icon 344M is an icon that allows the user to instruct the vascular image display field 834 to display choroidal vessels with a vessel diameter of 300 μm or more and less than 600 μm in a second color different from the first color. Icon 344H is an icon that allows the user to instruct the vascular image display field 834 to display choroidal vessels with a vessel diameter of 600 μm or more in a third color different from the first and second colors.

Icon 842 is an icon that allows the user to instruct the display of all choroidal blood vessels in the blood vessel image display area 834. When icon 842 is operated, all choroidal blood vessels are displayed in colors corresponding to the respective blood vessel diameters.

The choroid analysis tool display field 806 is a field in which icons for selecting multiple choroidal analyses are displayed. It includes a vortex vein position icon 852, a symmetry icon 854, a vessel diameter icon 856, a vortex vein-macula/optic disc icon 858, and a choroid analysis report icon 860. The vortex vein position icon 852 indicates that the vortex vein position is to be displayed. The symmetry icon 854 indicates that the symmetry of the analysis point is to be displayed. The vessel diameter icon 856 indicates that the analysis results regarding the diameter of the choroidal vessels are to be displayed. The vortex vein-macula/optic disc icon 858 indicates that the analysis results of the analysis of the positions between the vortex vein, the macula, and the optic disc are to be displayed. The choroid analysis report icon 860 indicates that the choroid analysis report is to be displayed.

Figure 29 shows how icon 844H is operated to display choroidal vessels with a vessel diameter of 600 μm or more in the vessel image display field 834. Pixels with a vessel diameter of 600 μm or more are extracted from the choroidal vessel diameter database memory area 518, and the extracted pixels are displayed in a first color in the vessel image display field 834. Similarly, when icon 844M is operated, pixels with a vessel diameter of 300 μm or more and less than 600 μm are displayed in a second color. When icon 844L is operated, pixels with a vessel diameter of less than 300 μm are displayed in a third color.

Figure 30 shows how the vascular diameter of a choroidal vessel indicated by a cursor is displayed. When a user operates the cursor or the like to specify a choroidal vessel displayed in the vascular image display field 384, the pixel where the cursor is located is identified, and the vascular skeleton number to which the identified pixel belongs is identified from the memory area 504. The vascular diameter of the vascular center point corresponding to the vascular skeleton number is read from the memory area 518, the average value of the read vascular diameters is calculated, and the average value is displayed on the display unit 836.

The blood vessel diameter associated with the pixel designated by the cursor may be read from the memory area 518 and displayed as the blood vessel diameter at the designated blood vessel position. Note that if the cursor is positioned in an area other than the choroidal blood vessels, the blood vessel diameter is not displayed because there is no blood vessel diameter data for the pixel where the cursor is positioned. In other words, the blood vessel diameter is displayed only when the cursor indicates a pixel in the choroidal blood vessel area.

Figure 31 shows how the vascular diameter of a specified choroidal vessel is displayed in graph 838. When the cursor is positioned on a choroidal vessel and then clicked, the vascular center point corresponding to the vascular skeleton number is read from memory area 510, and the vascular diameter at the vascular center point is read from memory area 518. The vascular diameter corresponding to each vascular center point is displayed on graph 838, with the vascular center point on the x-axis and the vascular diameter on the y-axis. This graph 838 allows the user to intuitively grasp the vascular diameter of curved or meandering blood vessels.

32 shows the second display screen 800A. As shown in FIG. 32, the image display field 804 of the display screen 800A has a choroidal vessel image display field 832 that displays a choroidal vessel image, and an enlarged image display field 862 that displays an enlarged image of a predetermined area including a position specified in the choroidal vessel image. As shown in FIG. 32, when a user operates a cursor or the like to position a cursor (cross mark) at a desired point in the choroidal image, an enlarged image of a predetermined area centered on the cursor position is displayed in the enlarged image display field 862. Furthermore, the pixel where the cursor is located is extracted from the memory area 514, and the blood vessel diameter corresponding to the extracted pixel is read from the memory area 518 and displayed as a numerical value in the enlarged image display field 862. Note that when the cursor is located in an area other than the choroidal vessel, the blood vessel diameter is not displayed because there is no blood vessel diameter data in the pixel where the cursor is located. In other words, the blood vessel diameter is displayed only when the cursor indicates a pixel in the choroidal vessel area.

As described above, in this embodiment, multiple vascular center points of choroidal blood vessels are identified, and the vascular diameter is calculated for each of the multiple identified vascular center points, so that the choroidal blood vessel diameter can be accurately calculated. In addition, since the vascular diameter is calculated using an image in which the choroidal blood vessel image is projected onto a pseudo-ocular surface, the vascular diameter at the actual fundus can be obtained. In addition, by creating a choroid database in which the calculated vascular diameter is associated with the pixels of the vascular portion, etc., various GUIs (graphic user interfaces) that support the user's diagnosis become possible.

[Second embodiment]

Next, the second embodiment will be described. The configuration of the second embodiment is similar to that of the first embodiment, so the description will be omitted. The action of the second embodiment is substantially similar to that of the first embodiment, so the differences will be mainly described.

Figure 33 shows a flowchart of the blood vessel diameter calculation process in step 308 in Figure 6 in the second embodiment. In step 902 in Figure 33, the same process as step 322 in Figure 7 is executed. That is, the choroidal vessels included in the choroidal vessel binarized image (Figure 34) are identified, and a blood vessel skeleton image VS and a blood vessel network VN are created for the choroidal vessels 912. In Figure 34, 912 indicates the pixel range of the blood vessel region, and 910, indicated by a dotted line, is a schematic representation of the skeleton image. Also shown is a vortex vein position mark 914 indicating the vortex vein position.

In step 904, the image processing control unit 206 sets the vascular center points 920N1, 920N2, ... as vascular division points for dividing the choroidal vessels 912 into multiple divided regions, as shown in FIG. 35.

In step 906, the image processing control unit 206 calculates the vascular diameter of each choroidal vessel in the choroidal vessel binarized image (FIG. 34). In the second embodiment, the image processing control unit 206 calculates the vascular diameter of the choroidal vessel by calculating the average of the vascular diameters of the choroidal vessels 912 in a plurality of divided regions. The image processing control unit 206 calculates the vascular diameter of each of the choroidal vessels 912 in a plurality of divided regions by dividing the area of each divided region by the length of the blood vessel in the direction of its course.

Specifically, as shown in FIG. 35, at each vascular center point 920N1, 920N2, ..., division lines 922N1, 922N2, ... are set in a direction perpendicular to the vascular running direction of each vascular center point 920N1, 920N2, ... in the region of each choroidal vessel in the choroidal vessel binarized image.

As shown in FIG. 36, division lines 922N1, 922N2, ... may be set in the center of adjacent pairs of vascular center points 920N1, 920N2, ... in a direction perpendicular to the vascular running direction of each vascular center point 920N1, 920N2, ... in the region of each choroidal vessel in the choroidal vessel binarized image.

The image processing control unit 206 extracts image portion 930I (FIG. 37) of each divided area separated by division lines 922N1, 922N2, .... The image processing control unit 206 projects each pixel 932 of image portion 930 of each divided area onto the simulated eyeball surface. The image processing control unit 206 finds the area of area 934 of each pixel of image portion 930 of the divided area projected onto the simulated eyeball surface, and calculates the sum of the areas of all pixels of the divided area.

The image processing control unit 206 calculates the distance between the center points of blood vessels in the blood vessel skeleton image 910 on the simulated eyeball surface. The image processing control unit 206 calculates the blood vessel diameter of the divided region between the blood vessel center points by dividing the area of the divided region by the distance on the eyeball between the blood vessel center points. The image processing control unit 206 then calculates the average of the blood vessel diameters in all divided regions of the choroidal blood vessels. This allows the blood vessel diameter of the choroidal blood vessel to be calculated.

Next, we will explain various variations of the technology disclosed herein.

<First modified example>

In the above embodiment, the management server 140 executes the image processing program shown in FIG. 6 in advance, but the technology of the present disclosure is not limited to this. The image viewer 150 may transmit an image processing command to the management server 140, and the management server 140 may execute the image processing program of FIG. 6 in response.

<Second modified version>

In the above embodiment, an example has been described in which a fundus image with an internal light irradiation angle of approximately 200 degrees is acquired by the ophthalmic device 110. The technology disclosed herein is not limited to this, and the technology disclosed herein may also be applied to a fundus image captured by an ophthalmic device with an internal irradiation angle of 100 degrees or less, or to a montage image in which multiple fundus images are combined.

<Third variant>

In the above embodiment, the fundus image is captured by an ophthalmic device 110 equipped with an SLO imaging unit, but the technology disclosed herein may also be applied to fundus images captured by a fundus camera capable of capturing images of choroidal blood vessels, or images obtained by OCT angiography.

<Fourth variant>

In the above embodiment, the management server 140 executes the image processing program. The technology of the present disclosure is not limited to this. For example, the ophthalmic device 110 or the image viewer 150 may execute the image processing program.

<Fifth variant>

In the above embodiment, the ophthalmic system 100 including the ophthalmic device 110, the axial length measuring device 120, the management server 140, and the image viewer 150 has been described as an example, but the technology of the present disclosure is not limited to this. For example, as a first example, the axial length measuring device 120 may be omitted, and the ophthalmic device 110 may further have the function of the axial length measuring device 120. Also, as a second example, the ophthalmic device 110 may further have at least one of the functions of the management server 140 and the image viewer 150. For example, if the ophthalmic device 110 has the function of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic device 110 or the image viewer 150. Also, if the ophthalmic device 110 has the function of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may perform the function of the management server 140.

＜Other variations＞

The data processing described in the above embodiment is merely an example. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed, without departing from the spirit of the invention.

In addition, in the above embodiment, an example was given of a case where data processing is realized by a software configuration using a computer, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, data processing may be performed only by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). A part of the data processing may be performed by a software configuration, and the remaining processing may be performed by a hardware configuration.

<Additional Notes>
In light of the above, the following note is proposed:
<Appendix 1>
A data structure for data stored in a computer including a control unit and a storage medium, comprising:
The control unit reads the data from the storage medium and uses the data in a process of calculating a vascular diameter of a choroidal blood vessel,
The data is
Data on a plurality of vessel center points of choroidal blood vessels;
Data on the vascular diameter at the center point of each blood vessel,
A data structure including:
<Appendix 2>
A data structure for data stored in a computer including a control unit and a storage medium, comprising:
The control unit reads the data from the storage medium and uses the data in a process of calculating the vascular diameter of each of a plurality of choroidal blood vessels;
The data is
Data on a plurality of vessel center points of each choroidal vessel corresponding to each of a plurality of choroidal vessels;
Data on the vascular diameter of each of the plurality of choroidal blood vessels at their respective vascular center points;
A data structure including:
<Appendix 3>
A data structure for data stored in a computer including a display unit, a control unit, and a storage medium, comprising:
The control unit reads the data from the storage medium and uses the data for processing to display a choroidal vessel image and a vessel diameter of a choroidal vessel at a location indicated in the choroidal vessel image on the display unit;
The data is
Data of each pixel of the choroidal vessel image;
Data on a plurality of vascular center points of choroidal blood vessels in the choroidal blood vessel image;
Data on the vascular diameter at the center point of each blood vessel,
A data structure including:
<Appendix 4>
A storage medium that stores data on a plurality of vascular center points of choroidal blood vessels and data on the vascular diameter of each vascular center point in association with each other.
<Appendix 5>
A storage medium that stores data on a plurality of vascular center points of each of a plurality of choroidal blood vessels and data on the vascular diameter of each vascular center point, corresponding to each of a plurality of choroidal blood vessels.
<Appendix 6>
A storage medium that stores, in a corresponding relationship, data on each pixel of a choroidal vessel image, data on a plurality of vessel center points of choroidal vessels in the choroidal vessel image, and data on the vessel diameter of each vessel center point.

Reference Signs List 100 Ophthalmic system 110 Ophthalmic device 120 Axial length measuring device 140 Management server 150 Image viewer 202 Photography control unit 204 Display control unit 206 Image processing control unit 262 CPU
254 Storage device

Claims (10)

A step of acquiring data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points , and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from the vascular center point;
A step of changing the pixel value of each pixel based on the blood vessel diameter at the blood vessel center point corresponding to each pixel, and changing and displaying the choroidal blood vessel in a different display mode ;
An image processing method comprising the steps of: The display step changes a pixel value of each pixel based on a blood vessel diameter at a blood vessel center point associated with each pixel , and changes a color or a pattern of the choroidal blood vessels to display the blood vessels.
The image processing method according to claim 1 . The method further includes a step of receiving a designation of a vascular diameter of a choroidal blood vessel,
The display step changes the display mode of the choroidal blood vessels based on the designated blood vessel diameter and the blood vessel diameter at the blood vessel center point associated with each pixel.
3. The image processing method according to claim 1. A step of acquiring data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points, and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from the vascular center point;
receiving designation of pixels of choroidal vessels on the choroidal vessel image;
displaying a blood vessel diameter at a blood vessel center point associated with a pixel on a specified choroidal blood vessel;
An image processing method comprising the steps of: The display step displays, as a numerical value, an average of vascular diameters at a plurality of vascular center points in a blood vessel to which a designated pixel belongs.
The image processing method according to claim 4 . the display step displays a graph in which one axis represents a center point of a blood vessel in a blood vessel to which a specified pixel belongs, and the other axis represents a blood vessel diameter at each of the center points of the blood vessel.
6. The image processing method according to claim 4 or 5 . An acquisition unit that acquires data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points , and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from each of the vascular center points;
A display unit that changes the pixel value of each pixel based on the blood vessel diameter at the blood vessel center point corresponding to each pixel, and changes and displays the display mode of the choroidal blood vessel .
An image processing device comprising: On the computer,
An acquisition step of acquiring data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points , and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from the vascular center point;
A display step of changing the pixel value of each pixel based on the blood vessel diameter at the blood vessel center point corresponding to each pixel, and changing and displaying the display mode of the choroidal blood vessel ;
An image processing program for executing a process including the steps of: An acquisition unit that acquires data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points, and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from each of the vascular center points;
A reception unit that receives designation of pixels of choroidal blood vessels on the choroidal blood vessel image;
a display unit that displays a blood vessel diameter at a blood vessel center point associated with a pixel on a specified choroidal blood vessel;
An image processing device comprising: On the computer,
An acquisition step of acquiring data including a plurality of vascular center points identified in a choroidal vessel image along the running direction of the choroidal vessel, a vascular diameter calculated at each of the plurality of vascular center points, and each pixel on the choroidal vessel that is associated with at least one of the vascular center points based on a distance from the vascular center point;
a receiving step of receiving designation of pixels of choroidal blood vessels on the choroidal blood vessel image;
A display step of displaying a blood vessel diameter at a blood vessel center point corresponding to a pixel on a specified choroidal blood vessel;
An image processing program for executing a process including the steps of: