[go: up one dir, main page]

WO2025024138A1 - Auto-étalonnage d'un indicateur d'emplacement de regard - Google Patents

Auto-étalonnage d'un indicateur d'emplacement de regard Download PDF

Info

Publication number
WO2025024138A1
WO2025024138A1 PCT/US2024/037485 US2024037485W WO2025024138A1 WO 2025024138 A1 WO2025024138 A1 WO 2025024138A1 US 2024037485 W US2024037485 W US 2024037485W WO 2025024138 A1 WO2025024138 A1 WO 2025024138A1
Authority
WO
WIPO (PCT)
Prior art keywords
gaze
location
display
region
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/037485
Other languages
English (en)
Inventor
Akanksha SARAN
Jacob A. ALBER
Danielle Karen BRAGG
Cyril ZHANG
John Carol LANGFORD
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US18/382,980 external-priority patent/US12487666B2/en
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Publication of WO2025024138A1 publication Critical patent/WO2025024138A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233—Character input methods
    • G06F3/0237—Character input methods using prediction or retrieval techniques

Definitions

  • a technique for eye tracker calibration can include using dynamic fiducial markers placed on the screen. This calibration technique gathers a large array of calibration points in a fast and unsupervised manner.
  • a self-calibrating approach for eye trackers can be based on a computational model of bottom-up visual saliency.
  • the computational model of bottom-up visual saliency assumes that the user’s gaze fixations always lie inside a small cluster of salient regions in the egocentric view of the user. While this approach is implicitly adaptive and leverages natural junctures of the user’s visual view on a screen or the 3D environment, it is data intensive for accurate autocalibration.
  • Adaptive techniques for gaze typing aim to improve the efficiency and user experience of gaze-based text entry by dynamically adjusting system parameters based on the user’s performance and gaze behavior.
  • a prior technique emphasizes a need to create gaze typing that proactively adapts dwell time instead of retrospectively reacting to user fatigue. This enables users to type short texts at their peak performance and economically use cognitive resources for long texts.
  • Another technique, a cascading dwell technique automatically adjusts a dwell time for gaze-based input based on the user’s performance. This approach has been shown to improve typing speeds and reduce errors in text entry tasks, highlighting the importance of dynamic adjustments in gaze-based input systems.
  • Another technique uses an adaptive gaze typing system that develops a computational model of the control of eye movements in gaze-based selection. This technique formulates the model as an optimal sequential planning problem bounded by the limits of the human visual and motor systems and use reinforcement learning to approximate optimal solutions for a number of fixations and duration required to make a gaze-based selection.
  • Another adaptive learning approach adaptively calibrates a red, green, blue (RGB)- based eye tracker used for display screens. This approach assumes the user can provide click- based feedback in the learning system in the form of backspaces activated on a physical keyboard/device.
  • RGB red, green, blue
  • a device, system, method, and computer-readable medium are configured for autocalibration of a visual eye tracker indicator.
  • a method can include receiving gaze tracker output data that indicates a gaze location on a display at which a user is gazing.
  • the method can further include determining an expected dwell region to which the user is expected to gaze after the gaze location.
  • the method can further include receiving further gaze tracker output data that indicates a subsequent gaze location on the display at which the user is gazing.
  • the method can further include calibrating, based on the subsequent gaze location and the expected dwell region, a location of a visual gaze indicator on the display resulting in a calibrated visual gaze indicator.
  • the expected dwell region can be a range of pixels.
  • the gaze location can be a pixel.
  • the expected dwell region can be in an output display region of the display.
  • the gaze location can be in a virtual keyboard region of the display
  • the method can further include responsive to determining the subsequent gaze location is within the expected dwell region and a typing speed is above a specified threshold characters per unit time decreasing a number of pixels in the range of pixels.
  • the method can further include responsive to determining the subsequent gaze location is not within the expected dwell region and a typing speed is less than a specified threshold characters per unit time increasing the number of pixels in the range of pixels.
  • the method can further include receiving, from a predictive text engine, a predicted next character.
  • the expected dwell region can be at a location of a virtual button corresponding to the predicted next character in a virtual keyboard region of the display.
  • Receiving, from the predictive engine can further include receiving a confidence associated with the predicted next character and the expected dwell region is at the location of the virtual button corresponding to the predicted next character only if the confidence is greater than a specified threshold.
  • the method can further include determining the gaze on the subsequent gaze location is sustained for a threshold amount of time. Calibrating can occur responsive to determining the gaze on the subsequent gaze location is sustained for the threshold amount of time.
  • a system can include a display device and a gaze tracker configured to generate output data that indicates a gaze location on the display at which a user is gazing.
  • the system can include a user interface that causes a virtual keyboard and an output display region to be displayed concurrently on the display device.
  • the system can further include an autocalibration application configured to receive the output data and determine an expected dwell region to which the user is expected to gaze after the gaze location.
  • the autocalibration application can be further configured to receive further output data from the gaze tracker that indicates a subsequent gaze location on the display at which the user is gazing.
  • the autocalibration application can be further configured to calibrate, based on the further output data and the expected dwell region, a location of a visual gaze indicator provided by the user interface on the display resulting in a calibrated visual gaze indicator.
  • the autocalibration application can be further configured to determine the gaze on the subsequent gaze location is sustained for a threshold amount of time. The calibrating can occur responsive to determining the gaze on the subsequent gaze location is sustained for the threshold amount of time.
  • the system can further include a predictive text engine configured to generate a predicted next character.
  • the autocalibration application can be further configured to receive the predicted next character.
  • the expected dwell region can be at a location of a virtual button corresponding to the predicted next character in a virtual keyboard region of the display.
  • the expected dwell region can be in the output display region of the display and the gaze location is in the virtual keyboard region of the display.
  • the autocalibration application can further configured to, responsive to determining the subsequent gaze location is within the expected dwell region, decreasing a number of pixels in a range of pixels covered by the expected dwell region.
  • FIGS.1 and 2 illustrate, by way of example, respective diagrams of an embodiment of a system for autocalibrating a gaze indicator.
  • FIG. 3 illustrates, by way of example, a diagram of an embodiment of an autocalibration system that includes a predictive text engine.
  • FIG. 4 illustrates, by way of example, a diagram of an embodiment of a method for calibration of a gaze indicator.
  • FIG. 1 illustrates, by way of example, a diagram of an embodiment of a method for calibration of a gaze indicator.
  • FIG. 5 illustrates, by way of example, a diagram of an embodiment of states for calibration of a gaze indicator.
  • FIG. 6 illustrates, by way of example, respective graphs that quantify typing efficiency using a prior gaze tracker and the autocalibrated gaze tracker.
  • FIG. 7 illustrates, by way of example, respective graphs the quantify mental workload on a participant.
  • FIG. 8 illustrates, by way of example, a block diagram of an embodiment of a machine (e.g., a computer system) to implement one or more embodiments.
  • DETAILED DESCRIPTION [0025]
  • Eye tracking technology has emerged as a valuable tool in a variety of applications, including accessibility, augmented reality (AR), virtual reality (VR), and even gaming. Eye tracking has been particularly beneficial for individuals with motor impairments who rely on gaze- based input methods for communication and device control.
  • Gaze typing a common use case of eye tracking, enables users to input text by looking at keys of an on-screen keyboard, thereby offering a hands-free and non-vocal method of communication.
  • the accuracy and efficiency of gaze typing depends on the calibration of the eye tracker.
  • Calibration in the context of gaze typing, is a process that establishes a relationship between a target of a gaze and the corresponding screen coordinates. Calibration can be time-consuming, tedious, and in some cases, uncomfortable for the user.
  • Head movements, eye fatigue, changes in lighting and other environmental conditions, and hardware inconsistencies lead to miscalibration over time. Miscalibration can significantly impact the performance of gaze typing.
  • the performance of gaze typing technologies is exacerbated for users with motor impairments.
  • ALS amyotrophic lateral sclerosis
  • miscalibration of gaze tracking devices and the resulting need for repeated calibrations are a significant barrier to use of gaze typing technologies. As gaze typing devices miscalibrate, people tend to auto-correct by gazing at neighboring targets, which makes it difficult to detect miscalibration from eye signals.
  • an eye tracker can be autocalibrated during gaze typing.
  • An autocalibrated gaze tracker can identify a location on a display (by pixel or a range of pixels) at which a typed character is to be displayed.
  • the autocalibrated gaze tracker can determine a location a gaze tracker indicates the user is looking at. Since the user is expected to look to see that the typed character is displayed correctly, a difference between the location provided by the gaze tracker and the location at which the typed character is to be displayed provides an indication as to the calibration of the gaze tracker.
  • the autocalibrated gaze tracker can adjust a location of a gaze indicator (a graphical component visible on the display that indicates a location at which the user is gazing) to account for the miscalibration.
  • a predictive text engine can predict a next character to be typed by the user.
  • the autocalibrated gaze tracker can determine a location of the next character to be typed on a virtual keyboard on the display.
  • the autocalibrated gaze tracker can adjust a location of the gaze indicator to account for the miscalibration.
  • the autocalibrated gaze tracker thus operates as a sort of man-in-the-middle that detects and compensates for miscalibration automatically.
  • Autocalibrated gaze tracking can leverage this insight to track reading behavior and compare their gaze during readying versus typing to the location of typed characters on the screen to estimate the miscalibration amount and direction.
  • An improved technique to gaze typingO aims to improve the gaze typing experience by continuously adjusting the calibration in real-time based on the user’s gaze behavior, thereby reducing the need for manual recalibration and offering a more natural and efficient interaction.
  • Autocalibrated gaze tracking can benefit a wide range of users, including those with motor impairments who use gaze keyboards for everyday communication, with amyotrophic lateral sclerosis (ALS), and others who use gaze typing systems for extended periods, such as gamers and virtual/augmented reality headset users.
  • ALS amyotrophic lateral sclerosis
  • the proposed autocalibrated gaze tracking leverages natural elements of a user interface (UI), such as a text box displaying typed text, which enables implicit calibration (without the user being aware that it is taking place).
  • UI user interface
  • autocalibrated gaze tracking relies on natural elements of the UI (even when the visual display does not have particularly salient regions such as a visual keyboard).
  • FIGS.1 and 2 illustrate, by way of example, respective diagrams of an embodiment of a system 100 for autocalibrating a gaze indicator.
  • the system 100 includes a display 102 and a gaze tracker 104 coupled to a computer 118.
  • the computer 118 executes an application that cause the display 102 to provide a UI for gaze typing.
  • Gaze typing provides functionality for a user to type using just their eyes.
  • the UI for gaze typing includes an output region 106 and a visual keyboard 108.
  • the gaze tracker 104 monitors a gaze (indicated by arrow 112) of a user 110 and provides (x, y) coordinates that indicate respective locations on the display 102 at which the user is predicted to be gazing.
  • the gaze tracker output 120 can also include a time or can otherwise be associated with a time (e.g., a relative time or a computer time).
  • Temporally ordered gaze tracker output 120 forms a series of locations (e.g., pixels of the display 102) at which the user 110 is predicted to be gazing.
  • the visual keyboard 108 contains multiple visible representations of keys that can be selected by user gaze.
  • the user is gazing at the letter “x” on a virtual button 114 of the virtual keyboard 108 in an attempt to type the word “next.”
  • the letter “x” is displayed in the output region 106 of the display 102.
  • the user 110 will type or otherwise alter the output displayed in the output region 106 by dwelling on a virtual button on the virtual keyboard 108. Then the user 110 will gaze at the output region 106 to observe how their gaze altered the output.
  • the user 110 tends to gaze at a next location in the output region 106 affected by dwelling on the virtual button.
  • the location is to the right of the “E” of “NE” in the output region 106 is where the letter “X” (or other inidicia) will appear when typed correctly by the user.
  • the gaze tracker 104 can be quite accurate at conveying the location at which the user 110 is gazing, the gaze tracker output 120 can be inaccurate due to the user moving their head, lighting conditions, or other factors. The gaze tracker output 120 can thus drift or otherwise be inaccurate.
  • a gaze tracker output 120 inaccuracy and amount of the inaccuracy can be determined by comparing the gaze tracker output 120 when the user 110 is gazing at the output region to the 106 to the actual, deterministic location of the output that is expected to be the target of the gaze. Additionally or alternatively, the next location may be selected in a different manner, such as by predicting a next letter the user 110 is likely to select, as some users may desire to select all the letters in a word prior to checking the output 120. [0038] The typical user will look at the output region 106 after dwelling on a virtual button of the virtual keyboard 108. Because the display in the output region 106 is deterministic, an application can know exactly which pixels of the display 102 will be used to provide the visualization caused by dwelling on the virtual button.
  • the exact location of where the user is expected to dwell next is known.
  • an amount of miscalibration can be detected, such as by software.
  • a location of an indicator 116 that provides a visual indication of the gaze tracker output 120 can be adjusted in accord with the amount of miscalibration that is detected. Assume that the gaze tracker output 120 is (x1, y1) for the time during which the user 110 is dwelling on the virtual button 114 and the miscalibration is determined to be ( ⁇ x, ⁇ y) based on the user 110 dwelling in the output region 106 immediately after dwelling on the virtual button 114.
  • the miscalibration can be determined, in some instances, only if a key is successfully typed on the display (in this case the letter “X”).
  • the software of the computer 118 can adjust the indicator 116 to be displayed at a location corresponding to (x1- ⁇ x, y1- ⁇ y). In the example of FIG. 1, the indicator 116 is detected as being off-center (to the right and below the center) of the virtual button 114.
  • the gaze tracker output 120 as discussed is two-dimensional (2D), it is not limited to just 2D.
  • the gaze tracker output 120 can be three-dimensional (3D) such as for AR, VR, or other applications that include depth. [0039] FIG.
  • FIG. 2 illustrates, by way of example, a diagram of the system 100 (without the computer for simple explanation purposes) immediately after the user 110 adjusts their gaze away from the virtual button 114.
  • the user 110 has moved their gaze to the output region 106 and to a location at which the character “x” is expected to be displayed.
  • An expected dwell region 220 indicates an area in which the user 110 is expected to dwell after dwelling on the character “x”.
  • the application can assume that the user 110 intends to gaze at the center of the expected dwell region 220, determine a difference (e.g., in terms of a number of pixels in the x direction and the y direction) between the gaze tracker output 120 and the center (or other location) within the expected dwell region 220.
  • the application can then adjust the location of the indicator 116 on the display 102 to properly reflect where the user 110 was actually gazing.
  • the application can thus automatically calibrate the gaze tracker output 120 to meet user expectations and provide a more seamless gaze typing user experience.
  • the expected dwell region 220, the number of calibrations, or other aspect of the UI can be adjusted based on a speed at which a user is typing. If the user is typing slower, the calibrations can be performed more often than if a user is typing faster (in which case it is likely the system is already well-calibrated).
  • a moving average of the difference between gaze tracker output 120 can be used as the amount of pixels to move the indicator 116.
  • FIG. 3 illustrates, by way of example, a diagram of an embodiment of an autocalibration system that includes a predictive text engine 330.
  • an autocalibration application 334 can execute or receive output from a predictive text engine 332.
  • the predictive text engine 332 can predict, based on recent typed text, a next character or characters to be typed by the gaze of the user 110.
  • the predictive text engine 330 can include T9, iTap, eZiText, LetterWise, WordWise, or the like.
  • the predictive text engine 330 can implement a machine learning (ML) algorithm to determine a next character or word being typed by a user.
  • Output of the predictive text engine 330 can indicate a letter, word, or the like, along with a confidence.
  • the confidence can indicate how likely the letter or word is correct. If the confidence is sufficiently high (e.g., above a specified threshold like 50%, 75%, 80%, 90%, 95%, a greater threshold or some threshold therebetween), the autocalibration application 334 can assume that the user 110 intends to gaze type the next letter indicated by the predictive text engine 330.
  • the expected dwell region 220 can be at a location in the visual keyboard 108 corresponding to a next predicted character.
  • the next predicted character in the example of FIG. 3 is “x”.
  • the calibration can then be performed based on a gaze in the visual keyboard region 108.
  • the application 334 can know which pixels of the display 102 correspond to each of the virtual buttons in the visual keyboard 108.
  • the knowledge of the actual location of the virtual button and the gaze tracker output 120 can be leveraged to determine the offset for calibration.
  • the difference between the center location (or other location) of the virtual button and the gaze tracker output 120 can be used in the running average (or other determination) of the offset.
  • FIG. 4 illustrates, by way of example, a diagram of an embodiment of a method 400 for calibration of a gaze indicator.
  • the method 400 as illustrated includes receiving gaze tracker output data that indicates a gaze location on a display at which a user is gazing, at operation 440; determining an expected gaze location to which the user is expected to gaze after the gaze location, at operation 442; receiving further gaze tracker output data that indicates a subsequent gaze location on the display at which the user is gazing, at operation 444; and adjusting, based on the subsequent gaze location and the expected gaze location, a location of a visual gaze indicator on the display, at operation 446.
  • the expected gaze location can be a range of pixels.
  • the expected gaze location can be in an output display region of the display and the gaze location is in a virtual keyboard region of the display.
  • the method 400 can further include receiving, from a text prediction model, a predicted next character.
  • the expected gaze location can be at a location of a virtual button corresponding to the predicted next character in a virtual keyboard region of the display.
  • the method 400 can further include determining the gaze on the subsequent gaze location was sustained for a threshold amount of time. The adjusting can occur responsive to determining the gaze was sustained for the threshold amount of time.
  • the method 400 can further include responsive to determining the subsequent gaze location is within the expected dwell region (e.g., a specified number of pixels, such as 150 or fewer pixels, note the specified number can be dependent on pixel size and can be less for larger pixels and more for smaller pixels) and a typing speed is above a specified threshold characters per unit time (e.g., 4, 5, 6, 7 or a different number of characters per minute) decreasing a number of pixels in the range of pixels.
  • the method 400 can further include responsive to determining the subsequent gaze location is not within the expected dwell region and the typing speed is below the specified characters per unit time, increasing the number of pixels in the range of pixels.
  • the proposed autocalibrated gaze trackers can improve the gaze typing experience by continuously adjusting the calibration in real-time based on gaze behavior, thereby reducing the need for manual recalibration and offering a more natural and efficient interaction.
  • Autocalibrated gaze trackers benefit a wide range of users, including those with motor impairments who use gaze keyboards for everyday communication and others who use gaze typing systems for extended periods, such as gamers and virtual/augmented reality headset users.
  • Experimental Setup [0050] An experimental setup consists of both hardware and software components to effectively evaluate the performance of the proposed autocalibrated gaze trackers with real users. An eye tracker tracked gaze data directed towards a screen that displayed an on-screen visual keyboard (FIG. 1).
  • the autocalibration application facilitated the capture and processing of x, y coordinates from the eye tracker, providing necessary data for the autocalibration technique.
  • the software for the autocalibration application received the 2D gaze coordinates in real-time. This allowed the autocalibration application to apply the necessary miscalibration corrections which could be displayed back on the visual keyboard application.
  • a user can activate a key (i.e. type a character) on the visual keyboard by dwelling on it for a fixed duration of time.
  • the autocalibration application has functionally similar to visual keyboards, however, it serves as a test bed with more control to process and update the miscalibrated gaze coordinates in real-time. A user’s detected gaze location is displayed as an indicator on the screen.
  • the user can direct their gaze at a desired character for typing. If they fixate on a key for a specified amount of time, a dwell timer is initiated by the system. If the user continues to look at the same key, the indicator can change size and, for example, decrease in area over time, eventually collapsing at the center of the key. Otherwise, the timer can be aborted and the user must fixate and dwell again to type any character subsequently.
  • the dwell timer is a mechanism to ensure that a user is intentionally willing to type that specific character. After the timer is completed, the user receives visual feedback (the key turns red or some other indicator) that the character is typed on the text box at the top of the screen.
  • a gaze location ⁇ ⁇ ⁇ , ⁇ ⁇ ⁇ is determined by the gaze tracker 104.
  • An automatically calibrated gaze location ⁇ ⁇ ⁇ , ⁇ ⁇ ⁇ is determined.
  • the automatically calibrated gaze location in this instance is not exact.
  • a true gaze location that is unobserved is illustrated. It can be a goal of automatic calibration to have the calibrated gaze location overlap with the true gaze location as much as possible.
  • a gaze fixation location ⁇ ⁇ ⁇ , ⁇ ⁇ ⁇ is detected by the gaze tracker 104.
  • a calibrated gaze fixation location is determined by calculating the error in calibration.
  • a difference between the detected gaze fixation location and a true gaze location ⁇ ⁇ ⁇ , ⁇ ⁇ ⁇ provides an estimation of error in the calibration.
  • the true gaze location in the example of FIG.5 is inferred to be a center of a symbol just typed (“N” in the example of FIG.5) in region 106.
  • N a symbol just typed
  • the autocalibrated gaze trackers herein provide an approach that automatically recalibrates while the person is typing without the need for manual recalibration, for a more seamless user experience.
  • the autocalibrated gaze tracker can assume the gaze is directed towards a text box when it falls above the keyboard region and within a certain threshold distance from a center of the text typed.
  • the system can assume that the user reads the last typed character when they look up to read and detect the offset in calibration accordingly.
  • Autocalibrated eye trackers can compute a moving average of the calibration offsets, which enables the calibration to be updated continuously and smoothly in real-time.
  • Autocalibrated eye trackers can continuously update the calibration error based on the user’s gaze behavior while reading the typed text.
  • the correction to the detected miscalibration can continuously applied when the user’s gaze moves away from the text box (i.e. while typing on the visual keyboard).
  • the autocalibrated gaze coordinates can be displayed to the user with the updated location of the red gaze cursor on the screen.
  • An example of the autocalibrated gaze tracker approach is detailed in the Algorithm.
  • User Study To evaluate the effectiveness of the autocalibrated gaze tracker, an in-lab user study was performed. The effectiveness of the autocalibrated gaze tracker was compared to a standard manual calibration approach. Manual calibrations were performed via a standard calibration procedure. Participants typed a set of 5 phrases each in two gaze typing systems, but were not made aware of any differences between the two gaze typing systems. [0059] 19 participants with no prior experience in gaze typing were recruited. All participants were sighted (i.e.
  • FIG. 6 illustrates, by way of example, respective graphs that quantify typing efficiency using a prior gaze tracker and the autocalibrated gaze tracker.
  • the autocalibrated system in the study, exhibited faster typing speeds (characters/minute), lower abort frequency (number of sessions aborted/total number of sessions), and requires lower number of backspaces to be used in comparison to the control.
  • FIG. 7 illustrates, by way of example, respective graphs the quantify mental workload on a participant.
  • participants consistently rated the autocalibration system more favorably than the manual calibration of the control.
  • differences between EyeO and the static control are statistically significant (p ⁇ 0.05).
  • the miscalibration amount for a static system does not change in a session and thus users can learn to consistently compensate for it, albeit under cognitive strain.
  • one user reports that for the autocalibrated system, “I would start off having to recognize the offset and type accordingly, but after a few characters it would adapt and then I could actually look at the intended character, so it got progressively easier”.
  • Another user states “I think [the autocalibrated system] is more comfortable. With [the manually calibrated system], I had to compensate for each word and found it harder to select the letter”.
  • Three primary themes were derived from the semi-structured interview questions and any additional content covered during the interview. Each primary theme was subdivided into multiple secondary themes. A transcript was categorized sentence-by-sentence into one or more secondary codes, if applicable.
  • Participants [0085] 7 participants from support organizations for ALS were recruited. Two of the participants were direct caregivers for family members who have/had ALS, three were members of organizations which support people with ALS, and one was a speech pathologist who treats ALS clients in the states of Washington, New Jersey, and Maryland in the United States. [0086] One of the participants had 20 years of experience in software and hardware development, with a focus on accessibility programming and building eye trackers.
  • Another participant has 40 years of software development experience and got involved in the ALS association 18 years ago, building software for missing pieces of eye tracking to make them do things they were not programmed or capable of doing at the time.
  • a participant with cerebral palsy who used eye trackers for gaze typing in her daily life participated.
  • caregivers are heavily relied on for setting up and using gaze tracking or other assistive devices.
  • support staff who specialize in assistive devices and care for people with ALS have a wealth of experience from working with many individuals, and it wan an aim to learn from their broad experience with many people.
  • Resulting Themes [0089] Findings from the semi-structured interview categorized by the three identified primary themes and their subsequent secondary themes.
  • Eye Function Participants noted that glasses often diminish eye tracking quality as they reflect both external light sources and the screen itself. Thus, many people with ALS do not wear glasses during gaze typing. [0093] Additionally, towards later stages of the disease, one side of the body often tightens, causing the head to drop and rotate to one side and making eye tracking more difficult. As a result, in later phases of the disease, only one eye will be used for tracking. Most modern eye trackers can reliably detect gaze by tracking only one eye. [0094] Changing Positions: Participants shared that people with ALS often move out of their chair or have their screens removed for medical care. Every time they return, they must recalibrate their eye tracker.
  • Recalibration Frequency Participants explained that ALS users often recalibrate as much as 50 times a day, in an attempt to fix poor tracking. This causes frustration and can discourage use of the device at all. “ It’s annoying to them and they need to calibrate again and again’; ‘..they’ll need to have 20 calibrations in a 2 hour period. Well, there’s no solution other than recalibrating. So like this would be amazing, them having the ability to have that autocalibration.”
  • Resources for Eye-Gaze Typing Participants discussed resources that help alleviate obstacles to gaze typing for people with ALS.
  • FIG. 8 illustrates, by way of example, a block diagram of an embodiment of a machine 700 (e.g., a computer system) to implement one or more embodiments.
  • a machine 700 e.g., a computer system
  • the computer 118, gaze tracker 104, or the display 102, autocalibration application 334, predictive text engine 330, or the like can include one or more of the components of the machine 700.
  • One or more of the computer 118, gaze tracker 104, display 102, autocalibration application 334, predictive text engine 330, or method 400 or a component or operations thereof can be implemented, at least in part, using a component of the machine 700.
  • One example machine 700 (in the form of a computer), may include a processing unit 702, memory 703, removable storage 710, and non- removable storage 712.
  • the example computing device is illustrated and described as machine 700, the computing device may be in different forms in different embodiments.
  • the computing device may instead be a smartphone, a tablet, smartwatch, or other computing device including the same or similar elements as illustrated and described regarding FIG.8.
  • Devices such as smartphones, tablets, and smartwatches are generally collectively referred to as mobile devices.
  • the various data storage elements are illustrated as part of the machine 700, the storage may also or alternatively include cloud-based storage accessible via a network, such as the Internet.
  • Memory 703 may include volatile memory 714 and non-volatile memory 708.
  • the machine 700 may include – or have access to a computing environment that includes – a variety of computer-readable media, such as volatile memory 714 and non-volatile memory 708, removable storage 710 and non-removable storage 712.
  • Computer storage includes random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM) & electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD ROM), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices capable of storing computer-readable instructions for execution to perform functions described herein.
  • the machine 700 may include or have access to a computing environment that includes input 706, output 704, and a communication connection 716.
  • Output 704 may include a display device, such as a touchscreen, that also may serve as an input component.
  • the input 706 may include one or more of a touchscreen, touchpad, mouse, keyboard, camera, one or more device-specific buttons, one or more sensors integrated within or coupled via wired or wireless data connections to the machine 700, and other input components.
  • the computer may operate in a networked environment using a communication connection to connect to one or more remote computers, such as database servers, including cloud-based servers and storage.
  • the remote computer may include a personal computer (PC), server, router, network PC, a peer device or other common network node, or the like.
  • the communication connection may include a Local Area Network (LAN), a Wide Area Network (WAN), cellular, Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), Bluetooth, or other networks.
  • LAN Local Area Network
  • WAN Wide Area Network
  • IEEE Institute of Electrical and Electronics Engineers
  • Wi-Fi Wi-Fi
  • Bluetooth or other networks.
  • Computer-readable instructions stored on a computer-readable storage device are executable by the processing unit 702 (sometimes called processing circuitry) of the machine 700.
  • a hard drive, CD-ROM, and RAM are some examples of articles including a non-transitory computer-readable medium such as a storage device.
  • a computer program 418 may be used to cause processing unit 702 to perform one or more methods or algorithms described herein.
  • the operations, functions, or algorithms described herein may be implemented in software in some embodiments.
  • the software may include computer executable instructions stored on computer or other machine-readable media or storage device, such as one or more non- transitory memories (e.g., a non-transitory machine-readable medium) or other type of hardware- based storage devices, either local or networked.
  • machine-readable media or storage device such as one or more non- transitory memories (e.g., a non-transitory machine-readable medium) or other type of hardware- based storage devices, either local or networked.
  • functions may correspond to subsystems, which may be software, hardware, firmware, or a combination thereof. Multiple functions may be performed in one or more subsystems as desired, and the embodiments described are merely examples.
  • the software may be executed on processing circuitry, such as can include a digital signal processor, ASIC, microprocessor, central processing unit (CPU), graphics processing unit (GPU), field programmable gate array (FPGA), or other type of processor operating on a computer system, such as a personal computer, server, or other computer system, turning such computer system into a specifically programmed machine.
  • processing circuitry such as can include a digital signal processor, ASIC, microprocessor, central processing unit (CPU), graphics processing unit (GPU), field programmable gate array (FPGA), or other type of processor operating on a computer system, such as a personal computer, server, or other computer system, turning such computer system into a specifically programmed machine.
  • the processing circuitry can, additionally or alternatively, include electric and/or electronic components (e.g., one or more transistors, resistors, capacitors, inductors, amplifiers, modulators, demodulators, antennas, radios, regulators, diodes, oscillators, multiplexers, logic gates, buffers, caches, memories, GPUs, CPUs, field programmable gate arrays (FPGAs), or the like).
  • electric and/or electronic components e.g., one or more transistors, resistors, capacitors, inductors, amplifiers, modulators, demodulators, antennas, radios, regulators, diodes, oscillators, multiplexers, logic gates, buffers, caches, memories, GPUs, CPUs, field programmable gate arrays (FPGAs), or the like.
  • the terms computer-readable medium, machine readable medium, and storage device do not include carrier waves or signals to the extent carrier waves and signals are deemed too transitory.
  • Example 1 includes a method comprising receiving gaze tracker output data that indicates a gaze location on a display at which a user is gazing, determining an expected dwell region to which the user is expected to gaze after the gaze location, receiving further gaze tracker output data that indicates a subsequent gaze location on the display at which the user is gazing, and calibrating, based on the subsequent gaze location and the expected dwell region, a location of a visual gaze indicator on the display resulting in a calibrated visual gaze indicator.
  • Example 2 further includes, wherein the expected dwell region is a range of pixels.
  • Example 3 further includes responsive to determining the subsequent gaze location is within the expected dwell region and a typing speed is above a specified threshold characters per unit time decreasing a number of pixels in the range of pixels.
  • Example 4 at least one of Examples 2-3 further includes responsive to determining the subsequent gaze location is not within the expected dwell region and a typing speed is less than a specified threshold characters per unit time increasing the number of pixels in the range of pixels.
  • Example 5 at least one of Examples 1-4 further includes, wherein the gaze location is a pixel.
  • Example 6 at least one of Examples 1-5 further includes, wherein the expected dwell region is in an output display region of the display and the gaze location is in a virtual keyboard region of the display.
  • Example 7 further includes receiving, from a predictive text engine, a predicted next character, and wherein the expected dwell region is at a location of a virtual button corresponding to the predicted next character in a virtual keyboard region of the display.
  • Example 8 further includes, wherein receiving, from the predictive engine, further includes receiving a confidence associated with the predicted next character and the expected dwell region is at the location of the virtual button corresponding to the predicted next character only if the confidence is greater than a specified threshold.
  • at least one of Examples 1-8 further includes determining the gaze on the subsequent gaze location is sustained for a threshold amount of time, and wherein the calibrating occurs responsive to determining the gaze on the subsequent gaze location is sustained for the threshold amount of time.
  • Example 10 includes a system configured to perform at least one of Examples 1-9.
  • Example 11 includes a machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations of the method of at least one of Examples 1-10.
  • the logic flows depicted in the figures do not require the order shown, or sequential order, to achieve desirable results.
  • Other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems.
  • Other embodiments may be within the scope of the following claims.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

D'une manière générale, sont discutés ici des dispositifs, des systèmes, et des procédés pour l'étalonnage d'un indicateur d'emplacement de regard. Un procédé peut consister à recevoir des données de sortie d'un oculomètre qui indiquent un emplacement du regard sur un module d'affichage qu'un utilisateur regarde, à déterminer un emplacement de regard attendu que l'utilisateur est censé regarder après l'emplacement du regard, à recevoir d'autres données de sortie de l'oculomètre qui indiquent un emplacement de regard ultérieur sur le module d'affichage que l'utilisateur regarde, et à ajuster, sur la base de l'emplacement de regard ultérieur et de l'emplacement de regard attendu, un emplacement d'un indicateur d'emplacement de regard visuel sur le module d'affichage.
PCT/US2024/037485 2023-07-27 2024-07-11 Auto-étalonnage d'un indicateur d'emplacement de regard Pending WO2025024138A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202363529276P 2023-07-27 2023-07-27
US63/529,276 2023-07-27
US18/382,980 2023-10-23
US18/382,980 US12487666B2 (en) 2023-07-27 2023-10-23 Autocalibration of gaze indicator

Publications (1)

Publication Number Publication Date
WO2025024138A1 true WO2025024138A1 (fr) 2025-01-30

Family

ID=92108336

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/037485 Pending WO2025024138A1 (fr) 2023-07-27 2024-07-11 Auto-étalonnage d'un indicateur d'emplacement de regard

Country Status (1)

Country Link
WO (1) WO2025024138A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120725945A (zh) * 2025-08-29 2025-09-30 北京理工大学 一种基于人眼注意力感知机理的图像增强方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019144196A1 (fr) * 2018-01-25 2019-08-01 Psykinetic Pty Ltd Procédé de saisie par mouvements oculaires, système et support lisible par ordinateur
US20190265788A1 (en) * 2016-06-10 2019-08-29 Volkswagen Aktiengesellschaft Operating Device with Eye Tracker Unit and Method for Calibrating an Eye Tracker Unit of an Operating Device
US20230107040A1 (en) * 2018-09-21 2023-04-06 Neurable Inc. Human-computer interface using high-speed and accurate tracking of user interactions

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190265788A1 (en) * 2016-06-10 2019-08-29 Volkswagen Aktiengesellschaft Operating Device with Eye Tracker Unit and Method for Calibrating an Eye Tracker Unit of an Operating Device
WO2019144196A1 (fr) * 2018-01-25 2019-08-01 Psykinetic Pty Ltd Procédé de saisie par mouvements oculaires, système et support lisible par ordinateur
US20230107040A1 (en) * 2018-09-21 2023-04-06 Neurable Inc. Human-computer interface using high-speed and accurate tracking of user interactions

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120725945A (zh) * 2025-08-29 2025-09-30 北京理工大学 一种基于人眼注意力感知机理的图像增强方法

Similar Documents

Publication Publication Date Title
US12141342B2 (en) Biofeedback method of modulating digital content to invoke greater pupil radius response
US9189095B2 (en) Calibrating eye tracking system by touch input
US12333084B2 (en) Assistant device arbitration using wearable device data
US12380281B2 (en) Injection of user feedback into language model adaptation
US20140354531A1 (en) Graphical user interface
US10438560B2 (en) Reducing visual crowding, increasing attention and improving visual span
Zhao et al. Eyesaycorrect: Eye gaze and voice based hands-free text correction for mobile devices
US12487673B2 (en) Adapting assistant suggestions rendered at computerized glasses according to changes in user gaze and/or other user input
Coelho et al. Developing accessible TV applications
WO2025024138A1 (fr) Auto-étalonnage d'un indicateur d'emplacement de regard
Nicolau et al. Mobile text-entry and visual demands: reusing and optimizing current solutions
US12487666B2 (en) Autocalibration of gaze indicator
Putze et al. Platform for studying self-repairing auto-corrections in mobile text entry based on brain activity, gaze, and context
Ikematsu et al. Investigation of Smartphone Grasping Posture Detection Method Using Corneal Reflection Images Through a Crowdsourced Experiment
Wang et al. A one-and two-phased model of aimed movement with eye-hand incompatibility
Saran et al. EyeO: Autocalibrating Gaze Output with Gaze Input
US12488626B1 (en) Liveness detection
US20250029385A1 (en) Computer vision to determine when video conference participant is off task
JP2025021012A (ja) 情報処理装置および情報処理方法
Schwarz Faceboard: Hands-free Text-Entry based on head movement
Feng Dwell-free input methods for people with motor impairments
WO2023219645A1 (fr) Adaptation de suggestions d'assistant rendues à des lunettes informatisées en fonction de changements de regard d'utilisateur et/ou d'une autre entrée d'utilisateur
Rodrigues Information Systems and Computer Engineering
Guerreiro User-Sensitive Mobile Interfaces: Accounting for Individual Differences amongst the Blind

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24748779

Country of ref document: EP

Kind code of ref document: A1