CN103699210A - Mobile terminal and control method thereof - Google Patents

Abstract

The invention discloses a method for controlling a mobile terminal, which includes the following steps: the mobile terminal detects and tracks the user's eye track; determines whether to trigger a preset operation function according to the tracked eye track; and when the determination result is yes, executes the Describe the operation functions triggered by the eye track. The invention discloses a mobile terminal, comprising: a detection and tracking module for detecting and tracking a user's eye track; a trigger judgment module for determining whether to trigger a preset operation function according to the tracked eye track; and an operation execution module, It is used for executing the operation function triggered by the eye track when the determination result is yes. The mobile terminal control method provided by the present invention enables the user to control the terminal device only by turning the eyeballs.

Description

Mobile terminal and control method thereof

technical field

The present invention relates to the field of mobile device communication applications, in particular to a mobile terminal and the mobile terminal using the method.

Background technique

Existing control methods running on smart mobile terminal devices basically interact with the device through the user actively touching the screen or keyboard input. The control mode requires the user to cooperate manually, including button clicking and touching the screen. On the one hand, it is laborious, and on the other hand, when the user's hands are inconvenient, the device cannot be controlled.

Chinese patent application number 200910252528.7 discloses a mobile communication terminal, including: an electrooculogram signal detection unit, used to collect potential changes generated during the movement of the user's eyeballs, and convert them into electrooculogram signals; The electric oculograph signal obtained by the electric signal detection unit is converted into a control signal for the mobile communication terminal, and the mobile communication terminal is controlled accordingly by using the control signal. The mobile communication terminal and the control method thereof provided by the embodiments of the present invention solve the problem that it is inconvenient for persons with physical disabilities and persons engaged in special work to control the mobile communication terminal by hand. The visual control mobile communication terminal will be an upgrade and evolution of the manual control mobile communication terminal, which has high-tech content and high convenience, and will bring people a more convenient and faster way of personal communication. The disadvantage of this technology is that it needs a special "glasses" as an operating medium or tool to convert the electro-oculogram signal into a communication signal, and the eye track is tracked by means of special glasses to capture the weak electro-oculogram of the human eye, so that It is very inconvenient for the user to achieve the control purpose, and the control efficiency is low due to the need for glasses as an intermediary transmission.

In view of this, the industry urgently needs a method of controlling terminal equipment to improve the intelligence level of the equipment and bring a new interactive experience.

Contents of the invention

In order to solve the above-mentioned technical problems, one of the objects of the present invention is to provide a mobile terminal manipulation method, including the following steps: the mobile terminal detects and tracks the user's eye track; and determines whether to trigger a preset operation function according to the tracked eye track ; and when the determination result is yes, execute the operation function triggered by the eye track.

According to another embodiment of the present invention, the operation function includes: an unlock function; according to the tracked eyeball track, determining to trigger the preset unlock function includes the following steps: when the mobile terminal is in the locked state, if the tracked When the eyeball track falls within a predetermined range of the screen of the mobile terminal and the time of falling within the predetermined range exceeds a first preset time, it is determined to trigger a preset unlocking function.

According to another embodiment of the present invention, the operation function includes: a pattern encryption function; according to the tracked eyeball trajectory, determining to trigger the preset pattern encryption function includes the following steps: when the mobile terminal is in an unlocked state, if When an encrypted pattern is drawn on the screen of the mobile terminal by the tracked eyeball track, it is determined to trigger a preset pattern encryption function.

According to another embodiment of the present invention, the operation function includes: pattern unlocking function; according to the tracked eyeball track, determining to trigger the preset pattern unlocking function includes the following steps: when the mobile terminal is in the locked state, if the tracking When the detected eyeball track falls within a predetermined range of the mobile terminal screen and moves according to a preset sequence of encrypted patterns, it is determined to trigger a preset pattern unlocking function.

According to another embodiment of the present invention, the operation function includes: page turning function; according to the tracked eyeball track, determining to trigger the preset page turning function includes the following steps: when the mobile terminal is in the reading state, if the tracking After the detected eye track falls into the front set line or the back set line of the page currently displayed on the screen of the mobile terminal for more than a second preset time, it is determined to trigger the preset page turning function.

According to another embodiment of the present invention, the operation function includes: a page-turning function; according to the tracked eyeball track, determining to trigger the preset page-turning function includes the following steps: when the mobile terminal is in the reading state , if the tracked eye track covers the page range displayed on the screen of the mobile terminal for more than a third preset time, it is determined to trigger a preset page-turning function.

According to another embodiment of the present invention, the operation function includes: scrolling function; according to the tracked eyeball trajectory, determining to trigger the preset scrolling function includes the following steps: when the mobile terminal is in the reading state, if the tracked When the eye track covers the page range displayed on the screen of the mobile terminal for a fourth preset time, and the eye track continues to move to the lower or upper range of the page within the fourth preset time, a trigger is determined. Preset scrolling functionality.

According to another embodiment of the present invention, executing the preset scrolling function triggered by the eye track includes the following steps: recording the number of lines of the page where the eye focus is located; directly scrolling the line where the eye focus resides to the user-preset The position of the page that starts scrolling; and scrolling the page in a set direction.

According to another embodiment of the present invention, the operation function includes: scrolling function; according to the tracked eyeball trajectory, determining to trigger the preset scrolling function includes the following steps: when the mobile terminal is in the reading state, if the tracked When the eye track stays on the front set line or the back set line of the page displayed on the screen of the mobile terminal for more than a fifth preset time, it is determined to trigger a preset scrolling function.

According to another embodiment of the present invention, the operation function includes: vocabulary translation function; according to the tracked eyeball trajectory, determining to trigger the preset vocabulary translation function includes the following steps: when the mobile terminal is in the reading state, if the tracking When the detected eye track falls within the range of the page displayed on the screen of the mobile terminal and stays on the word on the current page for more than a sixth preset time, it is determined to trigger the preset vocabulary translation function.

According to another embodiment of the present invention, executing the preset vocabulary translation function triggered by the eye track includes the following steps: the mobile terminal outputs the translation in the form of a voice broadcast or a floating small window.

According to another embodiment of the present invention, the operation function includes: a focus function; according to the tracked eyeball track, determining to trigger the preset focus function includes the following steps: when the mobile terminal is in the shooting state, if the tracked When the time during which the eyeball track falls within the screen range of the mobile terminal exceeds the seventh preset time, it is determined to trigger a preset focusing function.

According to another embodiment of the present invention, executing the preset focus function triggered by the eye track includes the following steps: the mobile terminal automatically focuses on the focus position of the current gaze on the screen as the focus of the photograph.

According to another embodiment of the present invention, the preset operation functions include: content screening of data information, screening of webpage topics, or search for interesting words in webpage browsing.

Another object of the present invention is to provide a mobile terminal, including: a detection and tracking module, used to detect and track the user's eye track; a trigger judgment module, used to determine whether to trigger a preset operation function according to the tracked eye track; And an operation execution module, configured to execute the operation function triggered by the eye track when the determination result is yes.

According to another embodiment of the present invention, the trigger judging module includes: an unlocking unit, configured to determine to trigger a preset unlocking function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in a locked state, if the When the eye track tracked by the detection tracking module falls within a predetermined range of the mobile terminal screen, and the time of falling within the predetermined range exceeds a first preset time, the unlocking unit determines to trigger a preset unlocking Function.

According to another embodiment of the present invention, the trigger judgment module includes: a pattern encryption unit, configured to determine to trigger a preset pattern encryption function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in an unlocked state, If the eye track tracked by the detection and tracking module draws an encrypted pattern on the screen of the mobile terminal, the pattern encryption unit determines to trigger a preset pattern encryption function.

According to another embodiment of the present invention, the trigger judging module includes: a pattern unlocking unit, configured to determine to trigger a preset pattern unlocking function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the locked state, if When the eye track tracked by the detection and tracking module falls within a predetermined range of the mobile terminal screen and moves according to a preset sequence of encrypted patterns, the pattern unlocking unit determines to trigger a preset pattern unlocking function.

According to another embodiment of the present invention, the trigger judgment module includes: a page turning unit, configured to determine to trigger a preset page turning function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if After the eye track traced by the detection and tracking module falls into the front set line or the back set line of the page currently displayed on the screen of the mobile terminal for more than a second preset time, the page turning unit determines to trigger a preset The set page turning function.

According to another embodiment of the present invention, the trigger judging module includes: a page turning unit, configured to determine to trigger a preset page turning function according to the tracked eyeball track, wherein: when the mobile terminal is reading state, if the eye track traced by the detection and tracking module covers the page range displayed on the screen of the mobile terminal for more than the third preset time, the page turning unit determines to trigger the preset direction Back page function.

According to another embodiment of the present invention, the trigger judging module includes: a scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if the The time for the eye track tracked by the detection and tracking module to cover the range of the page displayed on the screen of the mobile terminal reaches a fourth preset time, and the eye track continues to the bottom or top of the page within the fourth preset time When the range moves, the scrolling unit determines to trigger a preset scrolling function.

According to another embodiment of the present invention, the operation execution module for executing the preset scrolling function triggered by the eye track includes: a unit configured to record the number of page rows where the gaze focus is located; A unit that scrolls the reserved line directly to the page position preset by the user to start scrolling; and a unit configured to scroll the page in a set direction.

According to another embodiment of the present invention, the trigger judging module includes: a scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if the When the eye track traced by the detection and tracking module stays on the front set line or the back set line of the page displayed on the screen of the mobile terminal for more than the fifth preset time, the scrolling unit determines to trigger the preset scroll function.

According to another embodiment of the present invention, the trigger judging module includes: a vocabulary translation unit, configured to determine to trigger a preset vocabulary translation function according to the tracked eye track, wherein: when the mobile terminal is in the reading state, if When the eye track traced by the detection and tracking module falls within the range of the page displayed on the screen of the mobile terminal and stays on the word on the current page for more than a sixth preset time, the vocabulary translation unit determines to trigger Preset glossary translation function.

According to another embodiment of the present invention, the operation execution module for executing the preset vocabulary translation function triggered by the eye track includes: a unit configured to output the translation in the form of a voice broadcast or a floating small window.

According to another embodiment of the present invention, the trigger judgment module includes: a focus unit, configured to determine to trigger a preset focus function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the shooting state, if the The focus unit determines to trigger a preset focus function when the eye track traced by the detection tracking module falls within the screen range of the mobile terminal for more than a seventh preset time.

According to another embodiment of the present invention, the operation execution module for executing the preset focusing function triggered by the eye track includes: configured to use the focus position of the current gaze on the screen as the focal point for taking pictures to perform automatic focusing unit.

According to another embodiment of the present invention, the trigger judgment module includes: a content screening unit, configured to determine the content screening function that triggers preset data information according to the tracked eyeball trajectory; The trajectory determines to trigger the preset webpage filtering function; and the interest word search unit is used to determine and trigger the preset interest word search function in browsing the webpage according to the tracked eyeball trajectory.

The method for controlling a terminal device provided by the present invention can detect and track a user's eye trajectory, and can determine an operation function for a terminal based on the tracked eye trajectory, and then perform corresponding operations. With this technical solution, users only need to turn their eyes to realize the control of the terminal device, which improves the operation efficiency of the terminal, further improves the intelligence level of the device, and brings a new interactive experience.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

Fig. 1 is a structural diagram according to an exemplary embodiment of the present invention;

Fig. 2 is a structural diagram according to another exemplary embodiment of the present invention;

Fig. 3 is a structural diagram according to another exemplary embodiment of the present invention;

Fig. 4 is a structural diagram according to another exemplary embodiment of the present invention;

Fig. 5 is a structural diagram according to another exemplary embodiment of the present invention;

Fig. 6 is a structural diagram according to another exemplary embodiment of the present invention;

Figure 7 is a flowchart according to another exemplary embodiment of the present invention;

FIG. 8 is a flowchart according to another exemplary embodiment of the present invention;

FIG. 9 is a flowchart according to another exemplary embodiment of the present invention;

Figure 10 is a flowchart according to another exemplary embodiment of the present invention;

Figure 11 is a flowchart according to another exemplary embodiment of the present invention;

FIG. 12 is a flowchart according to another exemplary embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. However, the invention may be embodied in many different forms and should not be considered limited to the specific embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey to those skilled in the art The idea, concept, purpose, conception, reference scheme and protection scope of the present invention. The terminology used in the detailed description of the specific exemplary embodiments illustrated in the drawings is not intended to be limiting of the invention. In the drawings, the same reference numerals refer to the same elements.

Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of said features, integers, steps, manipulations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, manipulations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in commonly used dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless defined as herein explain.

Those skilled in the art can understand that the "terminal" used here includes not only a device with a wireless signal receiver without transmission capability, but also a device with receiving and transmitting hardware capable of bidirectional communication on a bidirectional communication link . Such equipment may include: cellular or other communication equipment with or without a multi-line display; personal communication systems (PCS) that may combine voice and data processing, facsimile and/or data communication capabilities; may include radio frequency receivers and pagers, Internet / Personal Digital Assistant (PDA) with intranet access, web browser, notepad, calendar and/or Global Positioning System (GPS) receiver; and/or conventional laptop and/or palmtop computer including radio frequency receiver or other devices. A "mobile terminal" as used herein may be portable, transportable, mounted in a vehicle (air, sea, and/or land), or adapted and/or configured to operate locally and/or in a distributed manner around the globe and/or any other location in the space to run. The "mobile terminal" used here may also be a communication terminal, an Internet terminal, and a music/video player terminal. The "mobile terminal" used here may also be a PDA, MID and/or a mobile phone with music/video playing function, etc. The "mobile terminal" used here may also be devices such as smart TVs and set-top boxes.

The present invention integrates an eyeball track tracking computing unit in the control mode of the terminal device, so that the user can realize various operation functions of the terminal device, such as unlocking function, pattern encryption function, pattern unlocking function, and vocabulary translation, only by turning the eyeball Functions (such as instant translation of words in foreign languages), reader page turning function, backward page turning function, scrolling function, camera auto-focus function, filtering function of concerned information, filtering function of web page theme, search function of interest words in browsing web pages And other controls, further enhance the intelligent level of the equipment, and bring a new interactive experience. The calculation unit of eye track tracking can perceive the user's attention time and eye movement trajectory on the mobile phone screen, so as to interact with the user more intelligently.

The present invention uses an eye track tracking and analysis unit to analyze the attention time and degree of the user's gaze and the eye track, and use the analysis result to control the intelligent terminal device. Utilizing this device, the present invention designs a system scheme for manipulating a mobile phone through eyes. The present invention includes several application scenarios of gaze control, including the manipulation method of using gaze to unlock the device, the manipulation method of instant translation with gaze, the manipulation method of controlling page turning and scrolling by gaze in reading mode, and the manipulation of gaze controlling camera autofocus Way.

As shown in FIG. 1 , an eye track tracking and analysis unit 102 is integrated in a mobile terminal function module 105 applicable to the present invention as a manipulation mode. The unit includes: a detection and tracking module 103 , a trigger judgment module 104 and an operation execution module 106 . The detection and tracking module 103 uses the front camera of the mobile terminal device to collect images of the user's eyeballs. The trigger judging module 104 is used to analyze the collected images of the user's eye movement, and analyze the user's eye movement track (ie gaze). The operation execution module 106 is configured to execute the operation function triggered by the eye track when the determination result is yes. The eyeball track tracking and analysis unit 102 outputs the dwell time of the user's eyeball track within a certain range (such as the mobile terminal screen) and the movement trajectory within a certain range (such as the mobile terminal screen) to the mobile terminal function module 105, and the mobile terminal function module 105 completes corresponding user operations or mobile phone functions by using the obtained user's eyeball trajectory data.

As shown in FIG. 12 , the embodiment of the present invention discloses a mobile terminal manipulation method, including the following steps: the mobile terminal detects and tracks the user's eye track; determines whether to trigger a preset operation function according to the tracked eye track; and When the determination result is yes, the operation function triggered by the eye track is executed. The eyeball track tracking and analysis module mentioned in the present invention will record the focus position coordinates of the user's eyeballs on the screen in real time, and calculate the running track of the eyeballs from multiple focus coordinate values in a certain continuous time. Through the analysis and judgment of these data, various control modes of smart terminal equipment can be realized through eyesight. Various eye manipulation scenarios are described in detail below.

Further, the preset operation functions that can be triggered by the eye track include: unlocking function; according to the tracked eye track, determining to trigger the preset unlocking function includes the following steps: when the mobile terminal is in the locked state, if the tracked eyeball When the trajectory falls within a predetermined range of the screen of the mobile terminal and the time within the predetermined range exceeds a first preset time, it is determined to trigger a preset unlocking function. Specifically, there are generally two types of application scenarios for eye track tracking to unlock mobile terminals: the first category, when the user does not encrypt the mobile terminal, the user uses his eyes to automatically unlock the mobile terminal; the second category, the user automatically unlocks the mobile terminal Under encryption conditions, the user uses the eye track tracking function to complete the preset encryption pattern to unlock the mobile terminal.

According to the example of the automatic unlocking of the eyes of the present invention, as shown in FIG. 2 , the function of "automatic unlocking of the eyes" is added to the mobile terminal. The user enables this function (as shown in step 201). The mobile terminal is in a dormant state (as shown in step 202), the user wakes up the mobile phone through a hard button on the mobile terminal (as shown in step 203), the eye track tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 204 shown). When the user's eyeball track (that is, gaze) falls within the range of the screen of the mobile terminal for the first time (as shown in step 205), a timer is started (as shown in step 206). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) falls within the range of the mobile terminal screen for more than a preset time (this preset time can be set by the user within 2 to 5 seconds, for example, the user sets 2 seconds ) (as shown in step 207), it is determined that the user wants to unlock and use the device (as shown in step 208), the timer resets to zero (as shown in step 209), and the smart terminal device is automatically unlocked (as shown in step 210 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, the gaze) falls within the screen range of the mobile terminal and does not exceed the preset time and leaves the screen range (this preset time can be set by the user within 2 to 5 seconds, for example The user sets 2 seconds) (as shown in step 207), then it is determined that the user does not want to unlock and use the device (as shown in step 211), the timer is reset to zero (as shown in step 212), and the detection and tracking module continues to detect the user eye movement trajectory (as shown in step 204).

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: a pattern encryption function; according to the tracked eye track, determining to trigger the preset pattern encryption function includes the following steps: In the locked state, if the traced eyeball trajectory draws an encrypted pattern on the screen of the mobile terminal, it is determined to trigger the preset pattern encryption function.

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: a pattern unlock function; according to the tracked eye track, determining to trigger the preset pattern unlock function includes the following steps: when the mobile terminal In the locked state, if the tracked eyeball track falls within a predetermined range of the mobile terminal screen and moves according to a preset sequence of encrypted patterns, it is determined to trigger a preset pattern unlocking function.

Specifically, according to the example of the encryption and unlocking of the special pattern of the eye track according to the present invention, as shown in Figure 3-4, the user can set the special pattern of the eye track to encrypt the smart device. When the user uses the eye track pattern function to unlock , the user needs to draw the preset encrypted patterns sequentially on the screen of the mobile terminal with the eye track (gaze). As shown in the figure below, for example, if the user pre-sets to use the trajectory pattern "Z" to unlock, the user only needs to turn his eyes and draw the trajectory of "Z" in the 9 circles on the screen to unlock. Furthermore, the present invention can also add the function of "eye track pattern unlocking" on the mobile terminal. The specific steps are as follows: the user activates this function (as shown in step 301); the mobile terminal is in a dormant state (as shown in step 302), the user wakes up the mobile phone through a hard button on the mobile terminal (as shown in step 303), and the eye track tracking function starts , turn on the front camera of the mobile terminal to capture the trajectory of the user's eyeballs (as shown in step 304); when the user's eyeball trajectory falls within the screen range of the mobile terminal, and the trajectory of the eyeballs on the 9 preset circles on the screen conforms to the preset When the encrypted pattern passes through the sequence of the nine circles and the final pattern (as shown in step 305), the mobile terminal is unlocked (as shown in step 306), otherwise, continue to detect the user's eye movement trajectory. The nine circles described in this example are for illustration, and actually include any pattern form that can be displayed on the screen of the mobile terminal and used to calculate the trajectory of the user's eyeballs.

According to another embodiment of the present invention, the preset operation functions that can be triggered by the eye track include: page turning function; according to the tracked eye track, determining to trigger the preset page turning function includes the following steps: when the mobile terminal is reading state, if the traced eye track falls into the front set line or the back set line of the page currently displayed on the screen of the mobile terminal after exceeding the second preset time, it is determined to trigger the preset page turning function. Furthermore, automatic page turning and scrolling in reading mode can be controlled by tracking eyeballs.

Specifically, in the reading mode of the present invention, the user can freely set two modes of automatic page turning controlled by gaze and automatic scrolling controlled by gaze. The control of automatic page turning is controlled by monitoring the focus position of the user's eyeball trajectory on the screen; the control of automatic scrolling can be controlled by monitoring the user's reading completion degree of the current page, which can be set by the user. For example, if the range of the user's eyeball track already includes 60% of the current page range, the system will automatically scroll the screen. The present invention adds the function of "automatic page turning controlled by eyesight" on the mobile terminal. At the end of the page, start the page turning; the second is the fast reading (speed reading) page turning mode, which means that the user presets the completion of the reading content as the indicator for starting the page turning operation. %, start the operation of turning the page backwards. 2) The present invention is designed to use color to identify the content of the page that the user has read, so as to visually display the trajectory of the eyeball on this page, and at the same time, highlight and flash the character at the current focus character of the eyeball, so as to identify The user's progress. The design allows users to intuitively understand the content they have read, and at the same time makes the reading service of the present invention more humanized and improves the user's comfort. The specific steps of each operation are described in detail below.

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: a page turning function; according to the tracked eye track, determining to trigger the preset page turning function includes the following steps: When the mobile terminal is in the reading state, if the tracked eyeball track covers the page range displayed on the screen of the mobile terminal for more than the third preset time, it is determined to trigger the preset page-turning function. According to an example of the backward page turning mode of the present invention, as shown in FIG. 5 , the operation steps of this function are as follows: the user enables this function (as shown in step 401 ). If the user chooses the normal page-turning mode, the operation steps are as follows: the user opens the document in the mobile terminal to read (as shown in step 402), the eye track tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 403 shown). When the user detection and tracking module detects that the user's eye track (i.e. gaze) falls into the range of the end of the set line of the page being browsed on the mobile terminal screen (as shown in step 404), start the timer (as shown in step 405) Show). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) always falls within the preset time (this preset time can be set by the user within 2 to 5 seconds, for example, the user sets 2 seconds) If it is within the range of the end of the set row of the browsed page (as shown in step 406), then it is determined that the user wants to perform a page-turning operation (as shown in step 407), and the timer is reset to zero (as shown in step 409). shown), the mobile terminal device automatically turns pages backwards (as shown in step 409). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) stays within the range of the end of the last set line of the browsing page for no more than the above-mentioned preset time (as shown in step 406), then it is determined that the user If you do not want to turn the page backwards (as shown in step 410 ), the timer is reset to zero (as shown in step 411 ), and the detection and tracking module continues to detect the user's eye movement track (as shown in step 403 ). Similar to the design, for turning pages forward, the above-mentioned "back setting row" can be modified to "before setting row".

According to the example of the fast reading (speed reading) page turning mode of the present invention, as shown in Figure 6, if the user sets the fast reading page turning mode, that is, the mode of turning pages according to the degree of completion of the reading content, the operation steps are as follows: the user opens The document in the mobile terminal is read (as shown in step 402 ), the eye tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 403 ). The detection and tracking module records the page range covered by the user's eye track, and calculates the degree of completion of the user's reading of this page. When the user detection and tracking module monitors the range covered by the user's eye track (ie, gaze) (as shown in step 406), that is, the user's reading volume reaches a preset percentage, for example, the reading rate of 60% set by the user, then it is determined that the user If it is desired to turn the page backwards (as shown in step 407), the mobile terminal device automatically turns the page backwards (as shown in step 409). If the eye track tracking and analysis unit detects that the range covered by the user's eye track (that is, gaze) does not reach the above-mentioned preset degree of completion (as shown in step 406), it is determined that the user does not want to turn the page backward (as shown in step 410) , the detection and tracking module continues to detect the user's eye movement trajectory (as shown in step 403). Similar to the design, for turning pages forward, the above-mentioned "back setting row" can be modified to "before setting row".

According to the example of the forward page turning mode of the present invention, as shown in FIG. 7 , the function of "automatic page turning controlled by gaze" is added on the mobile terminal. The user enables this function (as shown in step 501). The user opens the document in the mobile terminal to read (as shown in step 502), the eye tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 503). When the user detection and tracking module detects that the user's eye track (i.e. gaze) falls into the range of the first row of the page being browsed on the screen of the mobile terminal (as shown in step 504), start the timer (as shown in step 505 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) falls within the range of the first line of the first line of the current page on the screen of the mobile terminal for more than a preset time (this preset time can be determined within 2 to 5 seconds) User setting, such as the user setting 2 seconds) (as shown in step 506), then it is determined that the user wants to turn the page forward (as shown in step 507), the timer is reset to zero (as shown in step 509), and the mobile terminal The device automatically turns pages forward (as shown in step 509). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) falls within the range of the first line of the first line of the current page on the screen of the mobile terminal and does not exceed the above-mentioned preset time (as shown in step 506), Then it is determined that the user does not want to turn the page forward (as shown in step 510), the timer is reset to zero (as shown in step 511), and the detection and tracking module continues to detect the user's eye movement track (as shown in step 503). Similar to the design, for turning pages backwards, the above-mentioned "first row" can be modified to "back setting row".

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: scrolling function; according to the tracked eye track, determining to trigger the preset scrolling function includes the following steps: when the mobile terminal is in the reading state If the tracked eye track covers the page range displayed on the screen of the mobile terminal for a fourth preset time, and the eye track continues to move to the lower or upper range of the page within the fourth preset time, it is determined to trigger the preset Scroll function. Further, the preset scrolling function triggered by the eye track provided by the present invention includes the following steps: recording the number of rows of the page where the focus of the gaze is located; directly scrolling the row where the focus of the gaze resides to the user-preset scrolling start point; position of the page; and scroll the page in a set direction. According to an example of the backward scrolling mode of the present invention, as shown in FIG. 8 , the function of "automatic scrolling controlled by gaze" is added to the mobile terminal. The user enables this function (as shown in step 601). The user opens the document in the mobile terminal to read (as shown in step 602), the eye tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 603). The user can preset the activation conditions of automatic backward scrolling. For example, the user can set the user's eyeball trajectory to cover 50% of the current page range, and the automatic backward scrolling operation will be activated. When the detection and tracking module detects that the user's eye track (ie, gaze) has covered a preset percentage of the page, for example, reaching 50% preset by the user (as shown in step 604 ), start a timer (as shown in step 605 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) is within the preset timing time (this preset time can be set by the user within 2 to 6 seconds, for example, the user sets 2 seconds) to continue to the page If the lower range moves (as shown in step 606), it is determined that the user needs to perform a scroll-down operation (as shown in step 607), the timer is reset to zero (as shown in step 609), and the mobile terminal starts an automatic scroll-back operation ( as shown in step 609). Here, the steps of the scrolling operation are as follows: 1. Record the number of rows on the page where the user's gaze is focused; 2. Scroll the row at the focus of the user's gaze directly to the above-mentioned position of the page that the user presets to start scrolling, for example, in 60% of the page; 3. The page scrolls down by a single line. If the eye track tracking and analysis unit detects that the user's eye track (that is, the gaze) does not continue to move down the page (as shown in step 606), for example, the user's gaze looks back at the front of the page again within the preset timing time. If there are several lines, it is determined that the user does not continue to read down, and there is no need to perform a downward scrolling operation (as shown in step 610), the timer is reset to zero (as shown in step 611), and the detection and tracking module continues to detect the user's eye movement trajectory (as shown in step 603).

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: scrolling function; according to the tracked eye track, determining to trigger the preset scrolling function includes the following steps: when the mobile terminal is in the reading state If the tracked eye track stays on the front set line or the back set line of the page displayed on the screen of the mobile terminal for more than the fifth preset time, it is determined to trigger the preset scrolling function. According to an example of the forward scrolling mode of the present invention, as shown in FIG. 9 , the function of "automatic scrolling controlled by gaze" is added to the mobile terminal. The user enables this function (as shown in step 701). The user opens the document in the mobile terminal to read (as shown in step 702), the eye tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 703). When the detection and tracking module monitors the user's eye track (ie gaze), the detection and tracking module detects that the user's gaze is fixed on the head of the first line of the page (as shown in step 704 ), start the timer (as shown in step 705 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) falls within the range of the first line of the first line on the current page of the mobile terminal for more than a preset time (this preset time can be set by the user within 2 to 6 seconds) For example, the user sets 2 seconds) (as shown in step 706), then it is determined that the user wants to scroll forward (as shown in step 707), the timer resets to zero (as shown in step 709), and the mobile terminal device automatically Scroll forward (as shown in step 709). If the eye track tracking and analysis unit detects that the user's eye track (that is, the gaze) falls within the range of the first line of the first line of the browsed page on the screen of the mobile terminal and does not exceed the above-mentioned preset time (as shown in step 706), then determine The user does not need to scroll forward (as shown in step 710 ), the timer is reset to zero (as shown in step 711 ), and the detection and tracking module continues to detect the user's eye movement track (as shown in step 703 ).

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: vocabulary translation function; according to the tracked eye track, determining to trigger the preset vocabulary translation function includes the following steps: when the mobile terminal is in the reading state, if the tracked eye track falls within the range of the page displayed on the screen of the mobile terminal and stays on the word on the current page for more than the sixth preset time, it is determined to trigger the preset vocabulary translation function. Furthermore, the execution of the preset vocabulary translation function triggered by the eye track includes the following steps: the mobile terminal outputs the translation in the form of a voice broadcast or a floating small window. According to an example of the automatic word translation function controlled by the gaze of the present invention, as shown in FIG. 10 , the function of "automatic translation of vocabulary controlled by the gaze" is added on the mobile terminal. The user enables this function (as shown in step 801). The user reads a foreign document on the mobile terminal (as shown in step 802), the eye tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 803). When the detection and tracking module detects that the user's eye track (ie gaze) falls within the screen range displayed on the current document (as shown in step 804 ), start a timer (as shown in step 805 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) stays on a word on the screen for more than a preset time (this preset time can be set by the user within 2 to 6 seconds, for example, the user sets 2 seconds) (as shown in step 806), it is determined that the user needs to start the automatic translation function of the word (as shown in step 807), the timer is reset to zero (as shown in step 809), and the mobile terminal device starts the background translation tool to The word is translated (as shown in step 809), and displayed to the user in the form of a voice broadcast or a floating small window. If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) has not been resident on a certain word within the above-mentioned preset time (as shown in step 806), it is determined that the user does not need to perform automatic translation of the word ( As shown in step 810 ), the timer is reset to zero (as shown in step 811 ), and the detection and tracking module continues to detect the user's eye movement track (as shown in step 803 ).

According to another embodiment of the present invention, the preset operation function that can be triggered by the eye track includes: a focus function; according to the tracked eye track, determining to trigger the preset focus function includes the following steps: when the mobile terminal is in the shooting state If the tracked eye track falls within the screen range of the mobile terminal for more than a seventh preset time, it is determined to trigger a preset focusing function. Furthermore, executing the preset focus function triggered by the eye track includes the following steps: the mobile terminal automatically focuses on the focus position of the current gaze on the screen as the focal point for taking photos. According to the example of using eye track tracking for camera auto-focus according to the present invention, as shown in FIG. 11 , the function of "eye-control camera auto-focus" is added to the mobile terminal. The user enables this function (as shown in step 901). The user turns on the camera shooting mode in the mobile terminal to take pictures (as shown in step 902), the eye track tracking function is activated, and the front camera of the mobile terminal is turned on to capture the user's eye movement track (as shown in step 903). When the user's eyeball track (ie gaze) falls into the range of the camera screen of the mobile terminal for the first time (as shown in step 904 ), start a timer (as shown in step 905 ). If the eye track tracking and analysis unit detects that the user's eye track (that is, gaze) falls into a certain area on the mobile terminal camera screen for more than a preset time (this preset time can be set by the user within 2 to 6 seconds, for example The user sets 2 seconds) (as shown in step 906), then it is determined that the user wants to perform auto-focus (as shown in step 907), the timer is reset to zero (as shown in step 909), and the mobile terminal device starts the auto-focus function ( As shown in step 909 ), that is, the focus position of the current gaze on the screen is used as the focal point for taking pictures. If the eye track tracking and analysis unit detects that the dwell time of the user's eye track (i.e. gaze) in the same area within the range of the camera screen of the mobile terminal does not exceed the above preset time (as shown in step 906), it is determined that the user does not want to Auto focus (as shown in step 910 ), reset the timer to zero (as shown in step 911 ), and the detection and tracking module continues to detect the user's eye movement track (as shown in step 903 ).

According to another embodiment of the present invention, the preset operation functions that can be triggered by the eye track include: content screening of data information, screening of webpage topics, or search for interesting words in webpage browsing. For example: the function of background automatic search for vocabulary of interest in the webpages browsed by users by tracking eyeball trajectories and webpage theme filtering.

The present invention is mainly aimed at the eye track tracking technology, and provides several innovative application scenarios. For example, gaze unlock, gaze-controlled automatic page turning and scrolling, gaze-controlled automatic word translation, and gaze-controlled camera autofocus. There are two types of gaze unlocking: static and dynamic. The former refers to that after the eyes are fixed on the screen area, the mobile phone will detect that the eyeball is gazing at the screen area and unlock it; the latter refers to the need for the eyeball to make a certain trajectory, that is, the eyeball blurs and turns a certain outline trace, such as a zigzag, and the mobile phone is unlocked after recognition . Eye-controlled automatic page turning and scrolling. Including the automatic page turning and scrolling activated by focusing on the end of the page in normal mode. In the fast reading mode, the mobile phone detects the range covered by the eyeball track, and starts automatic page turning and scrolling. Automatic word translation feature. Implementation method: When the eyes focus on a word in the document, the mobile phone will automatically start the word translation function, and the translation or Baidu interpretation of the word will pop up in the form of a small pop window. The camera's autofocus function. When looking at a certain area of the camera screen, the mobile phone will automatically focus on this area, avoiding manual intervention and improving efficiency.

The effect of the invention: it has brought unprecedented control experience to the user - enabling the user to control the unlocking, page turning, automatic vocabulary translation, automatic focusing, and data information content screening of the smart terminal device just by turning the eyeball; making the user It can interact with smart terminal equipment more conveniently and accurately, and provides a very convenient and feasible control method for occasions where hand operation is inconvenient and for some disabled people.

As shown in Figure 1, the present invention also provides a mobile terminal, including: a detection and tracking module 103, used to detect and track the user's eye track; a trigger judgment module 104, used to determine whether to trigger the preset according to the tracked eye track and an operation execution module 106, configured to execute the operation function triggered by the eye track when the determination result is yes.

According to another embodiment of the present invention, the trigger judging module includes: an unlocking unit, configured to determine to trigger a preset unlocking function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in a locked state, if the When the eye track tracked by the detection tracking module falls within a predetermined range of the mobile terminal screen, and the time of falling within the predetermined range exceeds a first preset time, the unlocking unit determines to trigger a preset unlocking Function.

According to another embodiment of the present invention, the trigger judgment module includes: a pattern encryption unit, configured to determine to trigger a preset pattern encryption function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in an unlocked state, If the eye track tracked by the detection and tracking module draws an encrypted pattern on the screen of the mobile terminal, the pattern encryption unit determines to trigger a preset pattern encryption function.

According to another embodiment of the present invention, the trigger judging module includes: a pattern unlocking unit, configured to determine to trigger a preset pattern unlocking function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the locked state, if When the eye track tracked by the detection and tracking module falls within a predetermined range of the mobile terminal screen and moves according to a preset sequence of encrypted patterns, the pattern unlocking unit determines to trigger a preset pattern unlocking function.

According to another embodiment of the present invention, the trigger judgment module includes: a page turning unit, configured to determine to trigger a preset page turning function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if After the eye track traced by the detection and tracking module falls into the front set line or the back set line of the page currently displayed on the screen of the mobile terminal for more than a second preset time, the page turning unit determines to trigger a preset The set page turning function.

According to another embodiment of the present invention, the trigger judging module includes: a page-turning unit, configured to determine to trigger a preset page-turning function according to the tracked eyeball track, wherein: when the mobile terminal is in the reading state, if When the eye track traced by the detection and tracking module covers the page range displayed on the screen of the mobile terminal for more than a third preset time, the page turning unit determines to trigger a preset page turning function.

According to another embodiment of the present invention, the trigger judging module includes: a scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if the detection tracking module tracks When the eye track covers the page range displayed on the screen of the mobile terminal for a fourth preset time, and the eye track continues to move to the lower or upper range of the page within the fourth preset time, the scrolling unit determines to trigger the preset scrolling function.

According to another embodiment of the present invention, the operation execution module for executing the preset scrolling function triggered by the eye track includes: a unit configured to record the number of lines on the page where the eye focus is located; A unit that scrolls directly to a user-preset page position that initiates scrolling; and a unit that is configured to scroll the page in a set direction.

According to another embodiment of the present invention, the trigger judging module includes: a scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the reading state, if the detection tracking module tracks When the eye track stays on the front set line or the back set line of the page displayed on the screen of the mobile terminal for more than the fifth preset time, the scrolling unit determines to trigger the preset scrolling function.

According to another embodiment of the present invention, the trigger judging module includes: a vocabulary translation unit, configured to determine to trigger a preset vocabulary translation function according to the tracked eye track, wherein: when the mobile terminal is in the reading state, if When the eye track traced by the detection and tracking module falls within the range of the page displayed on the screen of the mobile terminal and stays on the word on the current page for more than a sixth preset time, the vocabulary translation unit determines to trigger Preset glossary translation function.

According to another embodiment of the present invention, the operation execution module for executing the preset vocabulary translation function triggered by the eye track includes: a unit configured to output the translation in the form of a voice broadcast or a floating small window.

According to another embodiment of the present invention, the trigger judgment module includes: a focus unit, configured to determine to trigger a preset focus function according to the tracked eyeball trajectory, wherein: when the mobile terminal is in the shooting state, if the The focus unit determines to trigger a preset focus function when the eye track traced by the detection tracking module falls within the screen range of the mobile terminal for more than a seventh preset time.

According to another embodiment of the present invention, the operation execution module for executing the preset focusing function triggered by the eye track includes: configured to use the focus position of the current gaze on the screen as the focal point for taking pictures to perform automatic focusing unit.

According to another embodiment of the present invention, the trigger judgment module includes: a content screening unit, configured to determine a content screening function that triggers preset data information according to the tracked eye track; a webpage screening unit, configured to, according to the tracked eye track, determining to trigger a preset web page filtering function; and an interest word search unit, configured to determine to trigger a preset interest word search function in browsing web pages according to the tracked eyeball track.

The method for controlling terminal equipment provided by the present invention is based on the eyeball recognition technology that the camera tracks human eyeballs, so that the user can control the terminal equipment only by turning the eyeballs, so as to further improve the intelligence level of the equipment and bring A brand new interactive experience.

In the above embodiments of the present invention, the preset time involved, such as the first preset time, the second preset time, the third preset time, etc., can be flexibly set according to the corresponding operation function, and can be the default time of the terminal device setting, and can also be set by the user.

Those skilled in the art can understand that certain parts of the above specific implementations may be presented in the form of algorithms, programs, or software modules, and these presentation forms include manipulation of data stored in computer memory. These representations generally consist of an instructional sequence of manipulations to accomplish a desired result. These manipulations require or involve physical manipulations or physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It will be understood by those skilled in the art that these signals are at times, principally for reasons of common usage, referred to as bits, values, elements, symbols, characters, terms, numbers, or other expressions. It should be understood, however, that these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless clearly stated otherwise below, descriptions in the rectification specification using terms such as "processing," "computing," "determining," or "displaying" may refer to data processing systems or similar electronic devices. Actions and processes that manipulate data represented in physical (e.g., electronic) quantities within a computer's registers and memory and convert it into the system's memory, registers, or other such information for storage, transmission, or display Other data similarly expressed in the form of physical quantities within the device.

Those skilled in the art should understand that the present invention may be implemented as a method, a circuit or a communication system. Accordingly, the invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects, all of which are referred to herein collectively as "circuitry". The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or known in the technical field any other form of storage medium.

Those skilled in the art can understand that all or part of the steps carried out in the above embodiments or methods can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method implementation is included. In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, each unit may exist separately physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. If the integrated modules are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, and the like.

Those skilled in the art will appreciate that the present invention may relate to an apparatus for performing one or more of the manipulations described in this application. Said apparatus may be specially designed and fabricated for the required purposes, or it may comprise known apparatus in a general purpose computer selectively activated or reconfigured by a program stored in it. Such a computer program can be stored in a device (e.g., computer) readable medium, including but not limited to any type of medium suitable for storing electronic instructions and respectively coupled to a bus. Types of disks (including floppy disks, hard disks, compact disks, CD-ROMs, and magneto-optical disks), random access memory (RAM), read-only memory (ROM), electrically programmable ROM, electrically erasable ROM (EPROM), electrically erasable Programmable ROM (EEPROM), flash memory, magnetic card or optical card. Readable media include any mechanism for storing or transmitting information in a form readable by a device (eg, a computer). Readable media include, for example, random access memory (RAM), read only memory (ROM), magnetic disk storage media, optical storage media, flash memory devices, signals transmitted in electrical, optical, acoustic or other forms (such as carrier waves, Infrared signal, digital signal), etc.

Those skilled in the art will appreciate that embodiments of the invention described herein may be part of other types of data processing systems, such as entertainment systems or personal digital assistants (PDAs), general purpose computer systems, specialized computer systems, embedded Embedded devices within other devices, cellular phones that do not include a media player, multi-touch tablet devices, other multi-touch devices, or devices that combine aspects or functions of these devices.

Those skilled in the art can understand that such as An object-oriented programming language such as Smalltalk or C++, a conventional procedural programming language such as the "C" programming language, or low-level code such as assembly language and/or microcode to write the computer program code for performing the manipulations of the present invention. The program code may execute entirely on a single processor as a stand-alone software package and/or on multiple processors as part of another software package.

Those skilled in the art can understand that the present invention has been described above with reference to the structural diagrams and/or block diagrams and/or flow diagrams of the methods, methods, systems and computer program products according to the embodiments of the present invention. It should be understood that each block in these structural diagrams and/or block diagrams and/or flow diagrams and combinations of blocks in these structural diagrams and/or block diagrams and/or flow diagrams can be implemented by computer program instructions. These computer program instructions may be provided to a general-purpose computer, specialized computer, or other programmable data-processing processor to create a machine, whereby the instructions executed by the computer or other programmable data-processing processor create a structure for implementing A method specified in a box or boxes of a diagram and/or a block diagram and/or a flow diagram.

Those skilled in the art can understand that these computer program instructions can also be stored in a computer-readable memory, and the computer-readable memory can direct a computer or other programmable data processing method to run in a specific manner, thereby storing in the computer-readable The instructions in the memory generate an article of manufacture comprising instruction methods for the functions specified in the block or blocks of the structural diagrams and/or block diagrams and/or flow diagrams.

Those skilled in the art can understand that these computer program instructions can also be loaded into a computer or other programmable data processing method, so that a series of manipulation steps can be executed on the computer or other programmable data processing method to generate a computer-implemented Processing whereby the instructions executed on a computer or other programmable data processing method provide steps for implementing the functions specified in the block or blocks of the structural diagrams and/or block diagrams and/or flow diagrams.

Those skilled in the art can understand that the various operations, methods, and steps, measures, and solutions in the processes discussed in the present invention can be replaced, changed, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes that have been discussed in the present invention may also be replaced, changed, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and schemes in various operations, methods, and processes disclosed in the present invention may also be alternated, changed, rearranged, decomposed, combined, or deleted in the prior art. Furthermore, relevant steps in the method provided by the present invention can be omitted.

Exemplary embodiments of the present invention are disclosed in the drawings and specification. Although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. It should be pointed out that those skilled in the art can make some improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. The protection scope of the present invention should be defined by the claims of the present invention.

Claims (28)

1. A method for operating a mobile terminal, comprising the following steps: The mobile terminal detects and tracks the user's eye trajectory; Determine whether to trigger a preset operation function according to the tracked eye track; and When the determination result is yes, execute the operation function triggered by the eye track. 2. The mobile terminal control method according to claim 1, wherein the operation function comprises: an unlock function; According to the tracked eyeball track, determine to trigger the preset unlocking function, including the following steps: When the mobile terminal is in the locked state, if the tracked eye track falls within a predetermined range of the screen of the mobile terminal, and the time of falling within the predetermined range exceeds a first preset time, determine a trigger Preset unlock function. 3. The mobile terminal control method according to claim 1, wherein the operation function comprises: a pattern encryption function; According to the tracked eye track, it is determined to trigger the preset pattern encryption function, including the following steps: When the mobile terminal is in an unlocked state, if an encrypted pattern is drawn on the screen of the mobile terminal by the traced eyeball trajectory, it is determined to trigger a preset pattern encryption function. 4. The mobile terminal control method according to claim 1, wherein the operation function comprises: a pattern unlock function; According to the tracked eyeball track, determine to trigger the preset pattern unlocking function, including the following steps: When the mobile terminal is in the locked state, if the traced eyeball track falls within a predetermined range of the screen of the mobile terminal and moves according to a preset sequence of encrypted patterns, it is determined to trigger a preset pattern unlocking function. 5. The mobile terminal control method according to claim 1, wherein the operation function comprises: a page turning function; According to the tracked eye track, determine to trigger the preset page turning function, including the following steps: When the mobile terminal is in the reading state, if the traced eye track falls into the front set line or the back set line of the page currently displayed on the screen of the mobile terminal after exceeding the second preset time, determine the trigger Preset page turning function. 6. The mobile terminal control method according to claim 1, wherein the operation function comprises: a page turning function; According to the tracked eye track, determine to trigger the preset page-back function, including the following steps: When the mobile terminal is in the reading state, if the tracked eye track covers the range of pages displayed on the screen of the mobile terminal for more than a third preset time, it is determined to trigger a preset page-turning function. 7. The mobile terminal control method according to claim 1, wherein the operation function comprises: a scrolling function; According to the tracked eyeball track, determine to trigger the preset scrolling function, including the following steps: When the mobile terminal is in the reading state, if the tracked eye track covers the page range displayed on the screen of the mobile terminal for a fourth preset time, and the eye track is within the fourth preset When the time continues to move to the lower or upper range of the page, it is determined to trigger the preset scrolling function. 8. The mobile terminal control method according to claim 7, wherein executing the preset scrolling function triggered by the eye track comprises the following steps: Record the number of lines on the page where the focus of the eye is located; Scroll the line where the eye focus resides directly to the user-preset position on the page that initiates the scrolling; and Scroll the page according to the set direction. 9. The mobile terminal control method according to claim 1, wherein the operation function comprises: a scrolling function; According to the tracked eyeball track, determine to trigger the preset scrolling function, including the following steps: When the mobile terminal is in the reading state, if the tracked eye track stays on the front set line or the back set line of the page displayed on the screen of the mobile terminal for more than the fifth preset time, Determines to trigger the preset scrolling function. 10. The mobile terminal control method according to claim 1, wherein the operation function comprises: vocabulary translation function; According to the tracked eye track, it is determined to trigger the preset vocabulary translation function, including the following steps: When the mobile terminal is in the reading state, if the tracked eye track falls within the range of the page displayed on the screen of the mobile terminal and stays on the word of the current page for more than the sixth preset time, determine Trigger the preset word translation function. 11. The mobile terminal control method according to claim 10, wherein executing the preset vocabulary translation function triggered by the eye track comprises the following steps: The mobile terminal outputs the translation in the form of voice broadcast or a floating small window. 12. The mobile terminal control method according to claim 1, wherein the operation function comprises: a focusing function; According to the tracked eye track, determine to trigger the preset focus function, including the following steps: When the mobile terminal is in the shooting state, if the tracked eye track falls within the screen range of the mobile terminal for more than a seventh preset time, it is determined to trigger a preset focusing function. 13. The mobile terminal control method according to claim 12, wherein executing the preset focus function triggered by the eye track comprises the following steps: The mobile terminal uses the focus position of the current eyesight on the screen as the focal point for taking pictures to perform automatic focusing. 14. The mobile terminal control method according to claim 1, characterized in that, the preset operation functions include: content filtering function of data information, filtering function of webpage theme, or interest word search in browsing webpage Function. 15. A mobile terminal, characterized in that it comprises: The detection and tracking module is used to detect and track the user's eye track; A trigger judgment module, configured to determine whether to trigger a preset operation function according to the tracked eye track; and An operation executing module, configured to execute the operation function triggered by the eye track when the determination result is yes. 16. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The unlocking unit is configured to determine to trigger a preset unlocking function according to the tracked eye track, wherein: when the mobile terminal is in a locked state, if the eye track tracked by the detection and tracking module falls into the mobile terminal When the screen is within a predetermined range and the time within the predetermined range exceeds a first preset time, the unlocking unit determines to trigger a preset unlocking function. 17. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The pattern encryption unit is configured to determine to trigger a preset pattern encryption function according to the tracked eye track, wherein: when the mobile terminal is in an unlocked state, if the eye track tracked by the detection and tracking module is within the When an encryption pattern is drawn on the screen of the mobile terminal, the pattern encryption unit determines to trigger a preset pattern encryption function. 18. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The pattern unlocking unit is configured to determine to trigger a preset pattern unlocking function according to the tracked eye track, wherein: when the mobile terminal is in a locked state, if the eye track tracked by the detection and tracking module falls within the When the mobile terminal screen moves within a predetermined range and in accordance with a preset sequence of encrypted patterns, the pattern unlocking unit determines to trigger a preset pattern unlocking function. 19. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The page turning unit is configured to determine to trigger a preset page turning function according to the tracked eye track, wherein: when the mobile terminal is in a reading state, if the eye track tracked by the detection and tracking module falls within the The page turning unit determines to trigger a preset page turning function after the front set line or the back set line of the page currently displayed on the screen of the mobile terminal exceeds a second preset time. 20. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The page-turning unit is used to determine and trigger the preset page-turning function according to the tracked eye track, wherein: when the mobile terminal is in the reading state, if the eye track tracked by the detection and tracking module When the time covering the range of pages displayed on the screen of the mobile terminal exceeds a third preset time, the page turning unit determines to trigger a preset page turning function. 21. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: A scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eye track, wherein: when the mobile terminal is in a reading state, if the eye track tracked by the detection and tracking module covers the screen of the mobile terminal When the time of the displayed page range reaches a fourth preset time and the eye track continues to move to the lower or upper range of the page within the fourth preset time, the scrolling unit determines to trigger a preset scrolling function. 22. The mobile terminal according to claim 21, wherein the operation executing module for executing the preset scrolling function triggered by the eye track includes: A unit configured to record the number of page lines on which the gaze is focused; A cell configured to scroll the line where the eye focus resides directly to a user-preset page position that initiates scrolling; and A unit configured to scroll the page in a set direction. 23. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: A scrolling unit, configured to determine to trigger a preset scrolling function according to the tracked eye track, wherein: when the mobile terminal is in a reading state, if the eye track tracked by the detection and tracking module resides in the mobile terminal When the time of the front set line or the back set line of the page displayed on the terminal screen exceeds the fifth preset time, the scrolling unit determines to trigger a preset scrolling function. 24. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The vocabulary translation unit is configured to determine to trigger a preset vocabulary translation function according to the tracked eye track, wherein: when the mobile terminal is in the reading state, if the eye track tracked by the detection and tracking module falls into the When the word on the current page is within the range of the page displayed on the screen of the mobile terminal and resides on the word for more than a sixth preset time, the vocabulary translation unit determines to trigger a preset vocabulary translation function. 25. The mobile terminal according to claim 24, wherein the operation execution module for executing the preset vocabulary translation function triggered by the eye track includes: A unit configured to output translations in the form of voice broadcasts or floating small windows. 26. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: The focus unit is configured to determine to trigger a preset focus function according to the tracked eye track, wherein: when the mobile terminal is in the shooting state, if the eye track tracked by the detection and tracking module falls into the mobile terminal When the time of the screen range exceeds the seventh preset time, the focus unit determines to trigger a preset focus function. 27. The mobile terminal according to claim 26, wherein the operation executing module for executing the preset focus function triggered by the eye track includes: A unit configured to automatically focus by using the focus position of the current gaze on the screen as the focus of taking photos. 28. The mobile terminal according to claim 15, wherein the trigger judgment module comprises: A content screening unit, configured to determine a content screening function that triggers preset data information according to the tracked eyeball trajectory; A webpage filtering unit, configured to trigger a preset webpage filtering function according to the tracked eyeball trajectory; and The word-of-interest search unit is configured to determine and trigger the word-of-interest search function in the preset browsing webpage according to the tracked eyeball trajectories.

Images