WO2021046659A1 - Method for improving the vision of a visually impaired person with a portable vision aid - Google Patents

Abstract

The invention relates to a method for improving the vision of a visually impaired person with an electronic vision aid which is held in front of the eye. The vision aid comprises a camera module, image memory, microprocessor, display unit, and sensors for detecting head movement and for detecting whether the vision aid is located on the eye. The method comprises the steps of detecting whether the vision aid is on the eye, viewing in video mode, creating a modified still image, reproducing the still image or a section thereof, and shifting the section using head movements. The shifting of the section is adjusted such that disruptive effects, such as shaking movements, are reduced. If the vision aid is removed from the head, the shifting is stopped and the image is frozen. When the vision aid is brought to the eye, the shifting can be continued.

Description

Method for improving the eyesight of a visually impaired person with a portable visual aid

The invention relates to a method for improving the vision of a visually impaired person, in which an image or several images or a video are recorded using at least one camera module integrated in the visual aid, cached in at least one image memory integrated in the visual aid and using at least one integrated in the visual aid Modified microprocessor and presented to one or both eyes by means of a display unit integrated in the visual aid. The presentation can take place in real time (live mode) or with a time delay. The visual aid is held on the head with the hands while viewing the performance. The display unit is designed in the form of a "Near To Eye" display. The display is operated close to the eye, similar to binoculars, and also has an external light barrier in the form of an eyecup. The visual aid also has an audio unit for the input and output of speech. Functions can also be carried out with voice support. The visual aid is designed in such a way that, analogously to binoculars, it can be held very quickly with the hands to the eye or eyes and can also be quickly removed from the eye or eyes.

Without limiting the scope of the invention, the device is described in connection with age-related macular degeneration (AMD). AMD is a progressive disease that typically begins around 60 years of age and later. AMD sufferers suffer from a loss of central vision.

The macula affected by the disease is the region on the retina with the highest resolution that normal sighted people use to read and see details. AMD sufferers adapt to the loss of central vision by perceiving incident light rays with neighboring, still functional, peripheral sensory cells on the retina. The resolution in this peripheral region is significantly lower (even for people with normal vision). In order to compensate for the reduced resolution of eyes affected by AMD, it is known that magnification is the most important supporting factor that helps these people to see and read text and object details again. When the magnification increases, however, the useful field of view is proportionally reduced, which in turn makes reading text difficult, for example due to the small number of letters displayed. Various aids are known to support those affected.

Reading devices consisting of a camera, a computer, a large display and a guide table attached under the camera are known. The guide table enables the reading material to be guided line by line. Due to their size, these arrangements can only be used in a stationary manner.

Electronic magnifiers are also known. These contain an integrated camera, memory, processor and an integrated display. Similar to a tablet PC, the display is operated at a normal reading distance (usually around 30 cm) in front of the eye. From a certain enlargement requirement and the associated screen size, the devices for portable use become unwieldy. Another disadvantage is the sensitivity to extraneous light.

Furthermore, arrangements are known which offer a relatively large field of view and can be used portably, in which the image supplied by the camera is fed to a head-worn, home-like or glasses-like image display device (so-called "Head Mounted Display" - HMD). Disadvantages of these systems are that they are rather difficult to tolerate socially due to their size and their constant position on the head, and that the image is smeared, especially when the head is trembling. If the visually impaired person reads a text with such a system and follows the text line with a rotating movement of the head, the superimposed trembling movement has the consequence that the ability to read is significantly restricted.

Furthermore, arrangements are known which display a specific image section on an image display device which is intended to change its position intuitively through the viewing direction or the head movement, for example in the form of so-called "scrolling". Shifting the image section through the line of sight poses major problems, especially for older people with a visual impairment such as AMD, as they have trouble focusing a point on the image with their eyes.

It is known that if there is a very high need for enlargement, a very low speed of displacement of the image section being viewed is required. Otherwise there is a risk of losing the relevant image section. An effect that binocular users are familiar with. Especially for older people who already suffer from a slight tremor (tremor), it is difficult or even impossible to keep the viewed image section steady or to move it in a suitable manner in order to read an enlarged text.

In contrast, the invention is based on the object of developing a method and a visual aid which enable the visually impaired to improve recognition and reading and can be used portable. The visual aid is intended to prevent the section of interest from being "lost" even when there is a high need for magnification and if there is any tremor, and to enable text such as letters, magazines, timetables, display boards, etc. to be read smoothly.

The object is achieved in such a way that the camera outputs the surroundings recorded in video mode in real time on the display. The magnification is set so that the person concerned recognizes enough details to capture the area of interest (e.g. a display board). The device is held with the flanks to the eye, like binoculars. Individual settings in the form of enlargement and image filters are possible at any time for better recognition of the area (triggered by means of appropriate operating elements and / or language support). The enlargement can be implemented by digital and / or optical zooming.

If the area of interest is in the detection area of the camera, a still image is recorded and displayed by means of a manual operating element (typically a button) or by means of a voice command.

If the area of interest is captured and displayed as a still image, it is now a matter of recognizing and reading further details and / or any existing text (for example in the case of a display board).

For this purpose, the recorded and displayed still image is digitally zoomed in using a corresponding operating element (typically a zoom wheel or a rocker switch) or a voice command until a suitable magnification factor is reached and the enlarged and displayed image section can be recognized or read by the user. In addition, individual image filter settings, such as contrast enhancement, can also be made at any time at this stage using appropriate operating elements and / or voice support.

The navigation, ie the shifting of the displayed image section within the still image, takes place via corresponding head movements. For this purpose, the head movements in the horizontal (x) and vertical (y) directions are recorded by the device and the section given by the set magnification is shifted over the still image as a function of these movements and the corresponding image section is displayed. The device is located on the eye during the entire process and thus follows the movement of the head. In this way, the section given by the set enlargement over the entire previous can be moved to allow the entire still image to be viewed. In contrast to the video mode in real time, however, in this operating mode the displacement or the displacement speed of the visible image section in the still image is not directly linked to the head movement. Rather, a displacement speed of the visible section in the still image is determined as a function of head movement and other factors. Essential criteria for determining the optimal speed of movement are the need for enlargement and the tremor situation of those affected. The aim is to reduce or even completely suppress the influence of trembling movements and, even with a large need for enlargement (small displayed image section), a pleasant interplay between head movement and moving displayed image section and thus an optimal reading flow without losing the words / line to be read, to ensure.

The task is solved as follows. The detection of the head movement is carried out at least with a multi-axis gyro sensor. In the case of a tremor, the detected head movements contain superimposed trembling movements of the person concerned. These trembling movements are subjected to digital filtering ("tremor filter") so that the influence of the trembling movements when the image section is shifted is reduced or even completely suppressed. The fact that the tremors are of a higher frequency is used for the filtering. The head movement signal, which corresponds to the movement speed (degrees / second), is filtered using a low-pass filter. This means that, depending on the situation, the tremors can be strongly attenuated or even completely suppressed. The tremor filter works regardless of the set magnification.

In one embodiment, the filtering is carried out as a digital filter.

In a further embodiment, the filtering can be carried out as an analog filter.

In one embodiment, the filtering is carried out separately for the x and y axes.

In a further possible embodiment, the speed vector of the x and y axes is formed and the filtering is carried out on the speed vector. A threshold filter is used in addition to the tremor filter so that the tremor is optimally filtered in the resting position. The threshold value is the value from which the movement of the device on the head triggers a shift in the image section. The image section is not moved below this value. A second threshold is effective of a previous movement, as soon as the value falls below the threshold value, the movement is regarded as stopped and accordingly the image section is no longer shifted. The threshold filter works independently of the set magnification.

In one possible embodiment, the threshold value for a movement from the rest position is identical to the threshold value from the movement into the rest position.

In a further possible embodiment, the threshold values are different and enable a hysteresis to be set between from rest position to movement and from movement in rest position.

In one possible embodiment, a threshold value that can be set separately for the x and y axes is implemented. In this variant, the threshold value filter is connected downstream of the respective tremor signal filter (low pass) and the tremor signal filtering is carried out separately for the x and y axes.

In a further possible embodiment, a threshold value that can be set separately for the x and y axes is implemented. In this variant, the threshold value filter is connected upstream of the respective tremor signal filter (low pass) and the tremor signal filtering is carried out separately for the x and y axes.

In a further possible embodiment, a threshold value that can be set separately for the x and y axes is implemented. In this variant, the threshold value filter is connected upstream of the tremor signal filter (low pass) and the tremor signal filtering is carried out on the speed vector. The speed vector is formed from the x and y signals according to the respective threshold value filter.

The three previously described embodiments can be implemented with a threshold value filter with or without hysteresis on the respective axis.

In a further embodiment, the threshold value filter is connected downstream of the tremor filter of the speed vector (x and y axes) and the threshold value filter thus acts on the filtered (tremor filter) speed vector. This variant can also be implemented in the embodiment with hysteresis of the threshold value and without hysteresis of the threshold value.

Both the threshold value filter and the tremor filter can be deactivated individually (per axis or speed vector).

The output signals present according to the above-described embodiments are used for the formation of the displacement speed of the displayed section in the still image. In the case of an embodiment with an output signal in the form of a speed vector, the vector is returned to its x- and y-components are decomposed. Regardless of whether the filtering was carried out for the individual axes or the vector, a filtered x and y speed signal of the head movement is available for the subsequent formation of the displacement speed of the image section in the still image (degrees / second).

In a further step, based on the filtered x, y speed signal of the head movement, the displacement speed of the displayed section in the still image is to be determined (x and y value, in pixels / second).

In the field of application of the invention, it is precisely in the field of application of the invention that a strong reduction of the head speed in relation to the displacement speed of the image section is necessary (even a large movement of the head should only bring about a small displacement of the image section). Otherwise there is a risk of losing the section of interest. The set magnification for forming the displacement speed is not taken into account, since this has turned out to be disruptive in the practical evaluation in the field of application of the invention.

In a possible embodiment for forming the displacement speed of the displayed section in the still image, the speed is formed separately for both axes independently of the magnification set.

The following applies to the x-axis: the x-displacement speed of the displayed section in the still image is calculated from the multiplication of the filtered x-speed signal of the head movement by an x-axis speed multiplier that can be set individually. The value of the multiplication factor thus corresponds to the reduction factor for the reduction of the filtered x head movement speed to the x displacement speed of the displayed section in the still image.

For the y-axis, the y-displacement speed of the displayed section in the still image is calculated from the multiplication of the filtered y-speed signal of the head movement with a y-axis speed multiplier, which can be set individually. The value of the multiplication factor thus corresponds to the reduction factor for the reduction of the filtered y-head movement speed to the y-displacement speed of the displayed section in the still image.

In a further embodiment, the displacement speed of the displayed section in the still image can be determined as follows. It will be a constant, adjustable basic speed, independent of any other factors, set as the displacement speed for the displayed section in the still image. This can be set separately for the x and y axes. For the x-axis, as long as there is a filtered x-speed signal of the head movement not equal to 0, the direction of movement of the signal is determined, +1 or -1, the basic speed corresponding to the axis is multiplied by the sign and thus forms the x-displacement speed of the displayed section in the still image. For the y-axis, as long as a filtered y-speed signal of the head movement is not equal to 0, the direction of movement of the signal is determined, +1 or -1, the basic speed corresponding to the axis is multiplied by the sign and thus forms the y-displacement speed of the displayed section in the still image. As a rule, a very low displacement speed is also selected in this embodiment. The advantage of this variant is an at least partial decoupling of the head speed from the displacement speed, in that only the filtered directional components (+, -) of the head movement are taken into account to form the displacement speed of the displayed section in the still image.

Regardless of the embodiments for shifting the section or for determining the speed of displacement of the displayed section, the following navigation support is implemented in the still image.

As soon as the edge of the still image is reached in the x or y direction when the displayed section in the still image is moved, a visual and / or audible signal appears and / or sounds. This means that the person knows that if they want to continue looking at something in this direction, they must create a new still image. In addition, the mentioned signal helps to orientate oneself better on the still image.

Furthermore, regardless of the embodiments for the shifting of the detail (or for determining the shifting speed) in the still image, the following problem arises.

Very low speeds of displacement of the displayed section in the still image are associated with the high need for enlargement, which means that large head movements are required for a small displacement of the displayed section in the still image. This means that the physical limits of normal head movement are reached before the displayed section in the still image reaches the limit of the still image. If this physiological limit is reached, that remains The viewer has no choice but to take the device off his head and bring himself physically into a more comfortable starting position. Associated with this is a movement of the device and thus a shift of the displayed section in the still image. If the viewer puts the device to the eye again, the image section is no longer placed where it was when he originally removed the device from the eye. This effect makes a coherent reading almost impossible.

The task of preventing the loss of the displayed section in the still image when the device is set down and the body realigned is achieved in such a way that a sensor (distance sensor) detects whether the device is in the eye of the beholder. As soon as the observer removes the device from the eye, this is recognized by the sensor and the position of the section in the still image that was displayed before the device was removed is saved. From the viewer's point of view, the current section is “frozen”. Now the viewer can physically realign himself with the device removed. If he then puts the device on his head again, this is recognized by the named sensor (distance sensor) and the still image section displayed before the device was set down is restored on the basis of the saved position and the navigation lock is canceled. In this way, the viewer can continue to navigate in the still image based on the section displayed before placing it. Thus, the maximum vertical and horizontal head movement is no longer a restriction when shifting the displayed section in the still image, since the device can simply be removed from the eye, the head can be brought back into a comfortable position and the device can be brought back to the eye without the last viewed Losing the still image section. As an alternative to the sensor, the image can also be frozen using a manual device, such as a button, or voice-assisted.

The sensor setting (distance sensor) from which distance to the eye the device is considered to be remote and thus the navigation lock is activated can be set individually. The distance from which the device is recognized as being on the eye again and the navigation lock is released can also be set individually. This also enables a hysteresis between the points “Device removed from the eye” and “Device on the eye”.

When the device is removed from the eye or when the section is “frozen”, a visual and / or auditory signal is generated. This means that the person using it knows that they can move the device freely without unintentionally moving the currently displayed section in the still image. Even if the device is brought to the head again, a visual and / or auditory signal is generated as soon as the device is recognized as being on the eye. This means that the person using it knows that the navigation lock has been lifted and that it is possible to move the displayed section in the still image again.

The detection of whether the device is in contact with the person's eye is carried out using at least one proximity sensor.

The invention has been shown and described. This can be changed in form and details in various changes and modifications for the person skilled in the art. Accordingly, it is intended that the following claims cover all such changes and modifications as may come within the scope and spirit of the invention.

Claims

Claims 1. A method for improving the vision of a visually impaired person, in which an image or several images or a video are recorded using at least one camera module integrated in the visual aid, buffered in at least one image memory integrated in the visual aid and modified by means of at least one microprocessor integrated in the visual aid and is presented to one or both eyes by means of a display unit integrated in the visual aid, characterized in that the camera in video mode outputs the current environment on the display with little delay, while the magnification and modification for the display output that can be selected by the person is set and executed with little delay, the detection area of interest is searched and a still image is recorded therefrom, after which the still image is modified, the modified still image is digitally zoomed in so that an enlarged still image section is produced which is displayed, then d the section given by the set magnification is shifted over the still image and the corresponding section content is displayed. 2. The method according to claim 1, characterized in that the visual aid is held with the hands. 3. The method according to claim 1 and 2, characterized in that at least one sensor detects whether the visual aid is in contact with the eye. 4. The method according to one or more of claims 1 to 3, characterized in that the creation of a still image and image modifications are triggered by manual controls integrated in the visual aid (button, zoom wheel, rocker switch, etc.) or by voice commands. 5. The method according to one or more of claims 1 to 4, characterized in that the recording displayed in video mode and the still image are modified. 6. The method according to claim 5, characterized in that the modification includes the formation of an image section (digital zooming). 7. The method according to claim 5 and 6, characterized in that the modification includes the digital filtering of the image. 8. The method according to one or more of claims 1 to 7, characterized in that the head movements in the horizontal x- and vertical y-direction are recorded by the visual aid resting on the head or on the eye and the section given by the set magnification as a function of is moved by these movements in the still image and the corresponding image section is displayed. 9. The method according to claim 8, characterized in that the detection of the head movement, that is to say the determination of the x- and y-axis signals, is carried out with at least one multi-axis gyro sensor. 10. The method according to claim 8 and 9, characterized in that the x- and y-axis signals generated by the head movement are filtered via a low-pass filter and the signals filtered in this way are used to further determine the displacement of the image section in order to reduce the influence to reduce the trembling movement to the cutout shift. 11. The method according to claim 10, characterized in that the low-pass filtering is carried out separately for the x- and y-axis signals. 12. The method according to claim 10, characterized in that the speed vector of the x- and y-axis signals is formed and the low-pass filtering is carried out on the speed vector. 13. The method according to one or more of claims 8 to 12, characterized in that the head movement only triggers a shift in the image section from a threshold value and a shift below a threshold value stops. 14. The method according to claim 13, characterized in that the threshold values for the start of a shift and the stop of the shift are set differently. 15. The method according to claim 13 and 14, characterized in that the settings of the threshold values and the determination of whether the threshold values are exceeded or not reached are carried out separately for the x- and y-axis signals. 16. The method according to claim 13 and 14, characterized in that the speed vector of the x- and y-axis signals is formed and the settings of the threshold values and the determination of whether the threshold values are exceeded or undershot are made on the speed vector. 17. The method according to one or more of claims 8 to 16, characterized in that the determination of whether the threshold values from which a head movement triggers or stops a shift of the image section are exceeded or not reached are arranged in front of the low-pass filters. 18. The method according to one or more of claims 8 to 16, characterized in that the determination of whether the threshold values from which a head movement triggers or stops a shift of the image section are exceeded or not reached are arranged after the low-pass filters. 19. The method according to one or more of claims 8 to 18, characterized in that the effects of the low-pass filters and the effects of the threshold values can be deactivated individually. 20. The method according to one or more of claims 8 to 19, characterized in that the x- and y-axis signals present after the low-pass filter and threshold value determination are used to determine the displacement speeds of the displayed image section on the still image in the x and y directions are multiplied by a speed multiplier that can be set separately for the x and y axes and is independent of the set magnification, whereby in the case of a speed vector, the vector is broken down into its x and y components before multiplication. 21. The method according to one or more of claims 8 to 19, characterized in that an adjustable base speed as Shift speed is used for the displayed image section in the still image, the speed for the shift in the x and y directions can be set separately and the image section is shifted at the corresponding basic speed in the x and y directions as long as there is a corresponding direction for the relevant direction Signal after low-pass filter and threshold value determination is available, which indicates that a head movement is being carried out, it is only detected whether a movement is present, the speed of which is not taken into account, the direction of the head movement is also determined from the relevant axis signal and in the case of a negative direction, the relevant basic speed is multiplied by -1, so that the image section is shifted in the negative direction. 22. The method according to one or more of claims 8 to 21, characterized in that as soon as the edge of the still image is reached in the x or y direction when the displayed section in the still image is moved, a visual and / or auditory signal appears and / or sounds. 23. The method according to claim 3 and one or more of claims 8 to 22, characterized in that the shifting of the image section is stopped and the current position of the section is saved as soon as the visual aid is no longer in contact with the eye, whereby the displayed image is “frozen " becomes. 24. The method according to claim 3 and one or more of claims 8 to 23, characterized in that when the visual aid is brought back to the eye, as soon as the visual aid rests on the eye again, the still image section displayed before removing the visual aid based on the stored position is restored and the shifting of the image section via the head movement is released again. 25. The method according to claim 3 and one or more of claims 8 to 24, characterized in that the sensor setting from which distance from the eye the visual aid is considered to be removed and thus the shift of the image section is blocked and the distance from which position the visual aid is is recognized again on the eye and so that the lock is canceled, can be set individually. 26. The method according to claim 3 and one or more of claims 8 to 25, characterized in that when the visual aid is removed from the eye or when the image section is “frozen” and when the visual aid is brought back in, as soon as the visual aid is recognized as being on the eye a visual and / or auditory signal is generated. 27. The method according to claim 3 and one or more of claims 8 to 26, characterized in that the blocking of the shifting of the image section and thus the “freezing of the image section” and the removal of the lock including the restoration of the “frozen image” can also be done manually Facilities such as buttons or by voice commands is possible.