TWI865332B - Method and system for identifying and tracking the features offundus images - Google Patents

Abstract

The invention provides a home-based system and method for monitoring fundus images of a retinopathy individual, wherein the aforementioned system can be applied to a portable mobile device.

Description

Method and system for identifying and tracking fundus image features

The present invention relates to a method and system for identifying and tracking fundus image features, particularly a method and system for identifying and tracking fundus time course image features for an individual, and more particularly a method and system for identifying and tracking fundus time course image features for patients with retinal diseases at home.

In addition to high myopia, the three highs, diabetes, or people with a family history of macular degeneration, the causes of retinal disease are that the tiny blood vessels are prone to disease, causing retinal hemorrhage, edema, or necrosis, which in turn impairs visual function. In the case of diabetic retinopathy caused by diabetes, there may be no symptoms in the early stages. Generally, 30% of patients with diabetes for more than ten years will begin to develop fundus lesions, and the incidence rate is as high as 80% for patients with a history of more than 25 years. The occurrence of diabetic retinopathy is mainly because diabetes is a systemic disease. Long-term high blood sugar environment will damage the retinal vascular endothelium and cause a series of fundus lesions. There may be no symptoms in the early stages of diabetic retinopathy. If blood sugar is not well controlled or if you have insulin-dependent diabetes, high blood pressure, high blood fat, or kidney disease, diabetic retinopathy will develop faster, and fundus disease may appear earlier. It is currently known that almost all patients with type 1 diabetes will develop retinopathy after 15 to 20 years, and 20% to 30% of them will become blind. More than 60% of patients with type 2 diabetes will have retinopathy, which is the main cause of blindness among people aged 20 to 65.

The current diagnosis of retinal diseases, such as diabetic retinopathy, mainly looks at the changes in the retina, so a fundus examination must be performed. The traditional fundus examination must first use mydriatics to dilate the pupil, and then use an ophthalmoscope and slit light to check whether there are vascular proliferation or other lesions in the fundus. Its disadvantages are that it is time-consuming, the examination results are difficult to record, and the conditions before and after treatment cannot be objectively compared, which makes it difficult for patients to accept it, so the examination rate is low.

In view of this, how to develop a device that can provide medical personnel with the inspection or diagnosis of patients with retinal diseases, or that patients with retinal diseases can operate conveniently at home without discomfort during the operation, can regularly identify and track fundus image features for convenient inspection, has standardized retinal images with high-definition image quality, improves retinal image quality, facilitates image archiving, and can be used as changes in images at different time points after a period of time, thereby solving the deficiencies of existing technologies and ensuring the accuracy of diagnosis, is an urgent problem that relevant technical fields currently need to solve.

Therefore, the purpose of the present invention is to provide a device that can be operated collaboratively or operated independently by medical personnel or others, and can be used by users to identify, track and examine the characteristics of binocular fundus images at intervals, and to establish standardized high-definition fundus images for effective comparison, thereby improving the convenience of users and further improving the problems of existing technologies.

Another purpose of the present invention is to provide a method for early warning and monitoring of microbleeds, hard exudates, punctate hemorrhages, neovascularization, cotton wool spots, etc. in the fundus of an individual, thereby effectively solving the problems of the prior art.

The invention discloses a method for identifying and tracking fundus image features, the steps of which include: moving an image capture unit provided with a composite collimator to a first position to be measured, the position to be measured enabling the fundus of an individual to be located within a focal length interval of the image capture unit, wherein the composite collimator further comprises an optic cup retinal blood vessel target and a central fossa target; using the optic cup retinal blood vessel target to track a first reference coordinate of the fundus of the individual, and simultaneously using the central fossa target to coordinately track a second reference coordinate of the fundus of the individual, wherein the first reference coordinate is selected from an optic disc, a branch retinal vein (branch retinal vein/retinal vein) or a branch retinal artery (branch retinal artery/retinal artery) of the fundus of the individual. artery), the second reference coordinate is the fovea of the fundus of the individual; an image of the fundus is captured, and the fundus image is obtained when the composite collimator tracks the first reference coordinate and the second reference coordinate successfully.

In one embodiment of the present invention, the optic disc further includes an optic cup.

In one embodiment of the present invention, the branch retinal vein is selected from the superior temporal retinal vein (venula temporalis retinae superior), the superior nasal retinal vein (venula nasalis retinae superior), the inferior nasal retinal vein (venula nasalis retinae inferior), and the inferior temporal retinal vein (venula temporalis retinae inferior), and the branch retinal artery is selected from the superior temporal retinal artery (arteriola temporalis retinae superior), the superior nasal retinal artery (arteriola nasalis retinae superior), the inferior nasal retinal artery (arteriola nasalis retinae inferior), and the inferior temporal retinal artery (arteriola temporalis retinae inferior).

In one embodiment of the present invention, the retinal vascular target of the optic cup and the central fossa target are in the shape of a circle, a cross circle, an oval, a triangle, an inverted triangle, a square, a rhombus, a trapezoid, an inverted trapezoid, a hexagon, an octagon, or an X.

In one embodiment of the present invention, the optic cup retinal vascular target is further provided with a circular target marker and a positive cross and an inverted cross marker, wherein the positive cross and the inverted cross marker are arranged in an X-shaped manner radiating from the circular target marker.

In one embodiment of the present invention, the circular target marker can track the optic cup.

In one embodiment of the present invention, the positive fork and inverted fork marker is further provided with four branching parts, which respectively track the superior temporal retinal vein or superior temporal retinal artery, the superior nasal retinal vein or superior nasal retinal artery, the inferior nasal retinal vein or inferior nasal retinal artery, and the inferior temporal retinal vein or inferior temporal retinal artery.

In one embodiment of the present invention, if the tracking of the first reference coordinate and the second reference coordinate by the composite collimator fails, an error message is issued, and the image capture unit is moved to a second position to be tested after evaluating the focal length relative to the fundus by calculating the difference with the fundus image. The second position to be tested is an appropriate focal plane position. The repeated steps include: moving an image capture unit provided with a composite collimator to a first position to be tested, the position to be tested enables the fundus of an individual to be located in an image capture unit focal length interval, wherein the composite collimator is further provided with an optic cup retinal blood vessel target and a central fossa target; using the optic cup retinal blood vessel target to track the first reference coordinate of the fundus of the individual, and using the central fossa target to coordinately track the second reference coordinate of the fundus of the individual, thereby capturing another fundus image.

In one embodiment of the present invention, fundus images captured at different time periods are compared, particularly the images of the optic disc, branch retinal vein or branch retinal artery and central fovea of the fundus of an individual are compared.

In one embodiment of the present invention, when capturing the fundus image, an aiming beam may be used to aim at the individual's eyes to assist in positioning and aiming, thereby standardizing the direction of the individual's eyeball.

In one embodiment of the present invention, a fundus margin is defined by the contrast value of the fundus image, a fundus region is selected by the composite center from the fundus image, and a feature identification comparison unit calculates whether the fundus region and the fundus image previously established by the composite center meet a repetitive area standard value, wherein the repetitive area standard value is at least 72%.

The invention discloses a fundus image feature recognition and tracking system, comprising: an image capture unit, wherein the image capture unit is provided with a composite collimator, the composite collimator is further provided with an X-shaped virtual target and a circular target, wherein the image capture unit moves to a first position to be tested, the position to be tested enables the fundus of an individual to be located within a focal length interval of the image capture unit, wherein the optic cup retinal blood vessel target is tracked The first reference coordinate of the fundus of an individual, the central fovea target is a second reference coordinate of the fundus of the individual, wherein the first reference coordinate is selected from the optic disc, branch retinal vein or branch retinal artery of the fundus of the individual, and the second reference coordinate is the central fovea of the fundus of the individual; one fundus image is captured by successfully tracking the first reference coordinate and the second reference coordinate by compound collimation.

In one embodiment of the present invention, the optic cup retinal vascular target is further provided with a circular target marker and a positive fork and inverted fork marker, wherein the positive fork and inverted fork marker is arranged in an X-shaped manner radiating from the circular target marker, wherein the circular target marker tracks the optic cup, wherein the positive fork and inverted fork marker is further provided with four branching parts, wherein the branching parts respectively track the superior temporal retinal vein or superior temporal retinal artery, the superior nasal retinal vein or superior nasal retinal artery, the inferior nasal retinal vein or inferior nasal retinal artery, the inferior temporal retinal vein or inferior temporal retinal artery.

In one embodiment of the present invention, a feature identification and matching unit is further included, wherein the feature identification and matching unit is used to identify and match the optic disc, branch retinal vein or branch retinal artery and central fossa in fundus images captured at different time periods.

In one embodiment of the present invention, a fundus image database is further included to store fundus images captured at different time periods.

In one embodiment of the present invention, a display unit is further included, which is electrically connected to the image capture unit and the fundus image database, and can display the fundus image transmitted from the image capture unit or transmitted from the fundus image database.

In order to make the above features and advantages of the present invention more clearly understood, the following is a detailed description of the embodiments with the accompanying drawings.

100: Feature marker recognition system for fundus images

101: Handheld mobile devices

1: Image capture unit

2: Fundus image database

3: Display unit

4: Feature identification and matching unit

5: Light source

51: Light

C: Circle crosshair

E: Eyes

R1: First reference coordinate

R2: Second reference coordinate

T1a: Optic cup retinal vascular target

T1b: Optic cup retinal vascular target

T11: Circular target calibrator

T12: Fork and fork calibrator

T2a: Central fossa target

T2b: Central fossa target

Figure 1 is a schematic diagram of the operation of the method according to the present invention.

Figure 2 is a schematic diagram of the operation of another method of the present invention.

Figure 3 is a schematic diagram of another operation method of the present invention.

Figure 4 is a schematic diagram of the operation of the tracking method of the present invention.

Figure 5 is a schematic diagram of the operation process of another tracking method of the present invention.

Figure 6 is a schematic diagram showing the successful targeting of the retinal vascular target of the optic cup and the central fossa target by cooperating with the first reference coordinate and the second reference coordinate of the fundus.

FIG. 7 is a schematic diagram of another embodiment of the present invention in which the first reference coordinate and the second reference coordinate are tracked and the targeting fails.

Figure 8 is a block diagram of the system of the present invention.

The following describes the technical content of the present invention through specific implementation forms. People familiar with this technology can understand the advantages and effects of the present invention from the content disclosed in this manual. However, without departing from the spirit of the present invention, the present invention can also be implemented or applied in a variety of different forms.

The "one embodiment" of the present invention described in the specification is used to describe and indicate that a specific function, structure or feature is included in the present invention. The term "one embodiment" appearing in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. That is, some embodiments may describe certain specific features, while other embodiments do not. In addition, structures, components or connections that are well known in the relevant field are not described in detail in order to avoid blurring the novel and unique features of the present invention.

In one embodiment of the present invention, the term "individual" used herein refers to a user, an operator, such as a retinal disease patient, especially a diabetic retinopathy patient. In another embodiment, the present invention can be operated by medical personnel, or by others assisting the retinal disease patient in operating at home, or by the retinal disease patient himself in the home environment. In addition, the individual can perform fundus image capture and confirmation in one of the two eyes after fundus image feature recognition and tracking, and then switch to the other eye for fundus image capture and confirmation.

Please refer to Figures 1 to 8. In one embodiment of a method for identifying and tracking fundus image features of the present invention, the image capture unit 1 can be moved to a position to be tested to capture the fundus of the individual. The aforementioned position to be tested is located in front of the individual's eye. At the same time, the light beam (aiming beam) 51 emitted by the light source 5 can be aimed at the individual to assist in positioning and aiming, wherein the light source 5 can be set in the handheld mobile device 101a. In other words, the individual eye E can look directly at the target light emitted from the image capture unit 1 (such as a lens) of the handheld mobile device 101a (i.e., a handheld smart device, a mobile phone) of the present invention, thereby ensuring that the individual eye can look directly at the aforementioned target light, thereby standardizing the direction of the individual's eyeball, and at the same time, the image of the individual eye and the image of the target light can also be transmitted to the display unit 3. In other words, the target light can standardize the direction of the individual's eyeball in the display unit 3.

The present invention also provides a method for identifying and tracking fundus image features, the steps of which include: moving an image capture unit 1 provided with a composite collimator to a first position to be tested, the first position to be tested enabling the fundus of an individual to be located within the focal length interval of the image capture unit 1, wherein the composite collimator is further provided with an optic cup retinal blood vessel target T1a and a central fossa target T2a; using the optic cup retinal blood vessel target T1a to track a first reference coordinate R1 of the fundus of the individual, and simultaneously using the central fossa target T2a to coordinately track a second reference coordinate R2 of the fundus of the individual. The first reference coordinate R1 is selected from the optic disc, branch retinal vein or branch retinal artery of the fundus of the individual, and the second reference coordinate R2 is the central fossa. The optic disc, the branch retinal vein or the branch retinal artery, and the central fossa can be regarded as three points in the fundus and form a common plane. In addition, the aforementioned tracking can be automatic tracking, semi-automatic tracking, or manual tracking. The aforementioned optic disc further includes an optic cup, the aforementioned branch retinal vein is selected from the retinal superior temporal vein, the retinal superior nasal vein, the retinal inferior nasal vein, and the retinal inferior temporal vein, and the aforementioned branch retinal artery is selected from the retinal superior temporal artery, the retinal superior nasal artery, the retinal inferior nasal artery, and the retinal inferior temporal artery. The first reference coordinate R1 is closer to the nasal side than the second reference coordinate R2, that is, the second reference coordinate R2 is closer to the temporal side than the first reference coordinate R1. Capture a fundus image, and this fundus image is obtained when the composite collimator tracks the first reference coordinate R1 and the second reference coordinate R2 successfully.

In one embodiment of the present invention, the optic cup retinal vascular target T1a and the central fossa target T2a may be, but not limited to, circular, cross-circular, oval, triangle, inverted triangle, square, rhombus, trapezoid, inverted trapezoid, hexagon, octagon, or X-shaped. For example, if the X-shaped optic cup retinal vascular target T1b is taken as an example, the X-shaped optic cup retinal vascular target T1b is further provided with a circular target marker T11 and a positive cross and inverted cross marker T12.

In one embodiment of the present invention, the circular target marker T11 tracks the optic cup, and the positive fork and inverted fork marker T12 further has four branching parts that are radiated from the circular target marker in an X-shape, and the four branching parts track the branch retinal vein or the branch retinal artery. In other words, if the first branch of one of the four branches can track the superior temporal retinal vein or the superior temporal retinal artery, the second branch of two of the four branches can track the superior nasal retinal vein or the superior nasal retinal artery, the third branch of three of the four branches can track the inferior nasal retinal vein or the inferior nasal retinal artery, and the fourth branch of four of the four branches can track the inferior temporal retinal vein or the inferior temporal retinal artery.

In another embodiment of the present invention, if the first reference coordinate R1 and the second reference coordinate R2 are not tracked by the composite collimation, an error message is issued, and the focal length relative to the fundus is evaluated by calculating the difference with the fundus image, and the image capture unit is moved to a second position to be tested, which is an appropriate focal plane position, and the above steps are repeated to capture another fundus image.

Furthermore, another embodiment of the present invention is to compare fundus images captured at different time periods, especially to compare images of the optic disc, branch retinal veins or branch retinal arteries and central fossa of the fundus of an individual. Thus, by comparing fundus images at different time periods, medical personnel/or patients can obtain whether fundus blood vessels at different time periods have mutated by a simple method or simple operation steps without expensive medical testing equipment or complicated operation steps.

In addition, the present invention provides a fundus image feature recognition and tracking system 100, for example, a handheld mobile system that can be used by medical personnel, others, or patients with retinal diseases to recognize and track fundus image features in a simple, convenient and effective manner in any place such as medical institutions or homes, wherein the aforementioned system 100 can be applied to a handheld mobile device 101, which includes, but is not limited to, a smart mobile device. The system 100 at least includes an image capture unit 1, a fundus image database 2, a display unit 3, and a feature identification and comparison unit 4. The feature identification and comparison unit 4 is used to identify and compare the optic disc, branch retinal vein or branch retinal artery and central fossa in the fundus images captured at different time periods, and the fundus image database 2 is used to store the fundus images captured at different time periods. In other words, the display unit 3 can be electrically connected to the image capture unit 1 and the fundus image database 2, and can transmit and display the fundus image from the image capture unit 1 or transmit and display the fundus image from the fundus image database 2. The aforementioned image capture unit 1, fundus image database 2, display unit 3 and feature identification and comparison unit 4 can be set in a handheld mobile device and electrically connected to each other.

Of course, the present invention is not limited thereto, wherein defining a fundus margin is obtained by the contrast value of the fundus image, and selecting a fundus interval is obtained from the fundus image by the circular crosshair C provided in the image capture unit 1, and the feature identification and comparison unit 4 calculates whether the fundus interval and the fundus image previously established by the circular crosshair C meet a repetition area standard value, wherein the repetition area standard value is at least 72%. If not, an error message may be issued by the handheld mobile device, for example, but not limited to, an error message may be provided by the display unit 3, such as an image error message, or the handheld mobile device 101 may also issue a voice error message. In other words, the auxiliary method provided by the present invention is to define the fundus margin by the contrast value of the fundus image and circle the fundus area, and calculate whether the fundus area and the circle center C in the previously established fundus image meet a repetitive area standard value. That is, the coverage rate of the circle center C can show whether the capture of the fundus image has deviated from the center, and on the other hand, it can also show whether an appropriate focal plane exists between the image capture unit 1 and the individual fundus distance.

Implementation Example 1

Please refer to Figure 5 and Figure 6. The retinal vascular target T1b and the central fossa target T2b are used to coordinately track the first reference coordinate R1 and the second reference coordinate R2. After successful tracking, the targeting is completed and the fundus image is obtained.

Implementation Example 2

Please refer to Figure 7. The retinal vascular target T1b of the optic cup and the central fossa target T2b are tracked together with the first reference coordinate R1 and the second reference coordinate R2. The tracking fails and the targeting cannot be completed, so the fundus image cannot be effectively obtained. The above tracking can be automatic tracking, semi-automatic tracking, or manual tracking.

The following describes the technical content of the present invention through specific implementation forms. People familiar with this technology can understand the advantages and effects of the present invention from the content disclosed in this manual. However, without departing from the spirit of the present invention, the present invention can also be implemented or applied in a variety of different forms.

R1: First reference coordinate

R2: Second reference coordinate

T1a: Optic cup retinal vascular target

T2a: Central fossa target

Claims (16)

A method for identifying and tracking fundus image features, the steps of which include: moving an image capture unit having a composite collimator to a first position to be measured, the first position to be measured enabling the fundus of an individual to be located within a focal length interval of the image capture unit, wherein the composite collimator further has an optic cup retinal blood vessel target and a central fovea target; using the optic cup retinal blood vessel target to track the first reference coordinate of the fundus of the individual , and simultaneously using the central fossa target to coordinately track the second reference coordinate of the fundus of the individual, wherein the first reference coordinate is selected from the optic disc, branch retinal vein or branch retinal artery of the fundus of the individual, and the second reference coordinate is the central fossa of the fundus of the individual; capturing a fundus image, which is obtained when the composite collimator successfully tracks the first reference coordinate and the second reference coordinate. The method of claim 1, wherein the viewing plate further comprises a viewing cup. The method of claim 2, wherein the branch retinal vein is selected from the superior temporal retinal vein, the superior nasal retinal vein, the inferior nasal retinal vein, and the inferior temporal retinal vein, and the branch retinal artery is selected from the superior temporal retinal artery, the superior nasal retinal artery, the inferior nasal retinal artery, and the inferior temporal retinal artery. The method of claim 3, wherein the optic cup retinal vessel target and the central fossa target are in the shape of a circle, a cross circle, an oval, a triangle, an inverted triangle, a square, a rhombus, a trapezoid, an inverted trapezoid, a hexagon, an octagon, or an X. As in the method of claim 4, the optic cup retinal vascular target is further provided with a circular target marker and a positive cross and inverted cross marker, and the positive cross and inverted cross marker is arranged in an X-shaped manner radiating from the circular target marker. The method of claim 5, wherein the circular target marker tracks the optic cup. As in the method of claim 6, the positive fork and inverted fork marker is further provided with four branching parts, and the branching parts respectively track the superior temporal retinal vein or superior temporal retinal artery, the superior nasal retinal vein or superior nasal retinal artery, the inferior nasal retinal vein or inferior nasal retinal artery, the inferior temporal retinal vein or inferior temporal retinal artery. As in the method of claim 1, if the tracking of the first reference coordinate and the second reference coordinate by the composite collimator fails, an error message is issued, and the image capture unit is moved to a second position to be tested after evaluating the focal length relative to the fundus by calculating the difference with the fundus image. The second position to be tested is an appropriate focal plane position, and the steps described in claim 1 are repeated to capture another fundus image. The method of claim 8, wherein the fundus images captured at different time periods are compared, particularly the images of the optic disc, branch retinal vein or branch retinal artery and central fovea of the fundus of the individual are compared. As in the method of claim 9, when capturing the fundus image, auxiliary positioning and aiming can be performed by aiming a light at the individual's eyes to standardize the direction of the individual's eyeballs. As in the method of claim 10, defining a fundus margin is obtained by the contrast value of the fundus image, and selecting a fundus interval is obtained from the fundus image by the composite center, and a feature identification comparison unit calculates whether the fundus interval and the fundus image previously established by the composite center meet a repeated area standard value, wherein the repeated area standard value is at least 72%. A fundus image feature recognition and tracking system includes an image capture unit, wherein the image capture unit is provided with a composite collimator, the composite collimator further having an optic cup retinal blood vessel target and a central fossa target, wherein the image capture unit moves to a first position to be measured, the position to be measured enables the fundus of an individual to be located within an image capture unit focal length interval, wherein the optic cup retinal blood vessel target is aimed at the individual The first reference coordinate of the fundus of the individual, the central fossa target is coordinated to track the second reference coordinate of the fundus of the individual, wherein the first reference coordinate is selected from the optic disc, branch retinal vein or branch retinal artery of the fundus of the individual, and the second reference coordinate is the central fossa of the fundus of the individual; wherein a fundus image is captured when the composite collimator tracks the first reference coordinate and the second reference coordinate successfully. As in the system of claim 12, the optic cup retinal vascular target is further provided with a circular target marker and a positive fork and inverted fork marker, the positive fork and inverted fork marker is arranged in an X-like shape radiating from the circular target marker, wherein the circular target marker tracks an optic cup, wherein the positive fork and inverted fork marker is further provided with four branching parts, wherein the branching parts respectively track the superior temporal retinal vein or superior temporal retinal artery, the superior nasal retinal vein or superior nasal retinal artery, the inferior nasal retinal vein or inferior nasal retinal artery, the inferior temporal retinal vein or inferior temporal retinal artery. The system of claim 12 further includes a feature identification and matching unit, wherein the feature identification and matching unit is used to identify and match the optic disc, branch retinal vein or branch retinal artery and central fossa in the fundus images captured at different time periods. The system of claim 14 further includes a fundus image database for storing the fundus images captured at different time periods. The system of claim 15 further includes a display unit electrically connected to the image acquisition unit and the fundus image database, and displays the fundus image transmitted from the image acquisition unit or transmitted from the fundus image database.