KR102448108B1 - Method, system and non-transitory computer-readable recording medium for supporting object control - Google Patents

Abstract

According to an aspect of the present invention, there is provided a method for supporting object control, referring to at least one of whether a trigger event related to movement of a control unit occurs and a distance between a motion coordinate of the control unit and a control target area is referred to. and when the position of the control means is changed, a virtual reference point specified based on the extension line of the indication vector before the position is changed and the motion coordinates of the control means after the position is changed are connected The control position in the control target region is determined by referring to a vector of There is provided a method comprising determining a control position in the control target region with reference to the pointing vector corrected based on a relationship between a second guide vector specified based on the coordinates.
The present invention is a technology developed through the 2020 Post-Corona Specialized Artificial Intelligence Technology Commercialization Support Project (CY201054) "Development of control intention recognition technology of non-contact remote virtual touch panel that prevents contact infection".

Description

Methods, systems and non-transitory computer-readable recording media supporting object control

The present invention relates to a method, system, and non-transitory computer-readable recording medium for supporting object control.

In the process of touching the display, pressing the button, and turning the switch on and off, unavoidable contact occurs. Contact infection occurs when you touch your own mucous membranes. In particular, in the case of a touch display, a button, a switch, etc. in a public place where an unspecified number of people come into contact with each other, the risk increases as the number of people in contact increases.

In recent years, in order to solve this problem, various methods for controlling an object in a non-contact manner have been introduced.

As an example of the related art, the technology disclosed in Korean Patent Application Laid-Open No. 2016-97410 may be cited as an example. According to this, in an input interface for receiving input information from a user, a display unit and one or more sensing devices are provided. a sensing unit for detecting a user's non-contact hand motion through the sensing device, a motion recognition unit generating non-contact motion recognition information by analyzing the non-contact sensing signals sensed by the sensing unit, and the generated non-contact motion recognition information A non-contact input interface through gesture recognition including a control unit for displaying a screen controlled according to the display unit has been introduced.

However, according to the techniques introduced so far, including the prior art as described above, when the position of the control means (eg, hand, fingertip) is changed, the control position on the control target area (eg, For example, a variable control position on the control target area is specified by an instruction vector used for specifying a cursor and a pointer). In the case of this method, the control position in the control target area varies more or less than the user's intention according to the positional relationship between the user and the control target area, so that intuitive control by the user is difficult.

Accordingly, the inventor(s) specifies a vector for specifying the changed control position in the control target region based on the indication vector when the position of the control means is changed, and based on this, the vector to be changed in the control target region By specifying the control position, we propose a new and advanced technology that enables intuitive control by a user.
The present invention is a technology developed through the 2020 Post-Corona Specialized Artificial Intelligence Technology Commercialization Support Project (CY201054) "Development of control intention recognition technology of non-contact remote virtual touch panel that prevents contact infection".

An object of the present invention is to solve all the problems of the prior art described above.

Another object of the present invention is to support a user to intuitively and precisely control a control position in a control target area.

In addition, it is another object of the present invention to dynamically determine an indication vector for specifying a control position conforming to a user's intention.

A representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, there is provided a method for supporting object control, referring to at least one of whether a trigger event related to movement of a control unit occurs and a distance between a motion coordinate of the control unit and a control target area is referred to. and when the position of the control means is changed, a virtual reference point specified based on the extension line of the indication vector before the position is changed and the motion coordinates of the control means after the position is changed are connected There is provided a method comprising determining a control position in the control target region by referring to a vector of

According to another aspect of the present invention, there is provided a method for supporting object control by referring to at least one of whether a trigger event related to movement of a control means occurs and a distance between a motion coordinate of the control means and a control target area. and, when the position of the control unit is changed, a first guide vector specified with reference to the motion coordinates of the control unit before the position is changed and the motion coordinates of the control unit after the position is changed There is provided a method comprising determining a control position in the control target region with reference to the corrected pointing vector based on a relationship between a second guide vector specified by reference.

According to another aspect of the present invention, there is provided a system for supporting object control, indicating at least one of whether a trigger event related to movement of a control means occurs and a distance between a motion coordinate of the control means and a control target area. a pointer vector determining unit for determining a vector, and when the position of the control means is changed, an imaginary reference point specified on the basis of the extension line of the instruction vector before the position is changed and the control means after the position is changed There is provided a system including a control position manager that determines a control position in the control target region by referring to a vector connecting motion coordinates.

According to another aspect of the present invention, there is provided a system for supporting object control, indicating at least one of whether a trigger event related to movement of a control means occurs and a distance between a motion coordinate of the control means and a control target area. an instruction vector determining unit for determining a vector, and when the position of the control unit is changed, a first guide vector specified with reference to the motion coordinates of the control unit before the position is changed and the control unit after the position is changed There is provided a system including a control position manager that determines a control position in the control target region with reference to the corrected reference vector based on a relationship between a second guide vector specified with reference to the motion coordinates of .

In addition to this, another method for implementing the present invention, another system, and a non-transitory computer-readable recording medium for recording a computer program for executing the method are further provided.

According to the present invention, the user can intuitively and precisely control the control position in the control target area.

In addition, according to the present invention, an indication vector for specifying a control position conforming to a user's intention can be dynamically determined.

1 is a diagram illustrating in detail an internal configuration of an object control support system according to an embodiment of the present invention.
2 is a diagram exemplarily illustrating a process in which an indication vector is dynamically determined according to a distance between a motion coordinate of a control means and a control target area according to an embodiment of the present invention.
3 to 6 are diagrams exemplarily illustrating a process of changing a control position in a control target area according to an embodiment of the present invention.
7 to 12 are diagrams exemplarily illustrating a process in which a motion vector is determined according to an embodiment of the present invention.
13 and 14 are diagrams exemplarily illustrating a process of specifying a control position in a control target region by an indication vector according to an embodiment of the present invention.
15 is a diagram exemplarily illustrating a control position that is changed in a control target area according to an indication vector according to an embodiment of the present invention.
16 is a diagram exemplarily illustrating a process in which a vertical vector is determined according to an embodiment of the present invention.
17 to 20 are diagrams exemplarily illustrating a process of specifying a virtual reference point according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [0016] Reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the present invention. In addition, it should be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be taken as encompassing the scope of the claims and all equivalents thereto. In the drawings, like reference numerals refer to the same or similar elements throughout the various aspects.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those of ordinary skill in the art to easily practice the present invention.

Configuration of object control support system

Hereinafter, the internal configuration of the object control support system 100 that performs an important function for the implementation of the present invention and the function of each component will be described.

1 is a diagram illustrating in detail an internal configuration of an object control support system 100 according to an embodiment of the present invention.

Referring to FIG. 1 , the object control support system 100 according to an embodiment of the present invention may include an indication vector determiner 110 , a control location management unit 120 , a communication unit 130 , and a control unit 140 . have. In addition, according to an embodiment of the present invention, the indication vector determiner 110, the control position management unit 120, the communication unit 130, and the control unit 140 at least some of them communicate with an external system (not shown) It can be program modules that do. These program modules may be included in the object control support system 100 in the form of an operating system, an application program module, and other program modules, and may be physically stored in various known storage devices. In addition, these program modules may be stored in a remote storage device capable of communicating with the object control support system 100 . Meanwhile, these program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types according to the present invention.

On the other hand, although described above with respect to the object control support system 100, this description is exemplary, and at least some of the components or functions of the object control support system 100 may be realized in a device to be described later or such It will be apparent to those skilled in the art that it may be included within the device. Also, in some cases, all functions and all components of the object control support system 100 may be entirely executed in the device or all included in the device 200 .

A device according to an embodiment of the present invention is a digital device equipped with a memory means and equipped with a microprocessor to have computing power, and includes smart glasses, smart watches, smart bands, smart rings, smart necklaces, smart earphones, smart earphones, It may be a wearable device such as a smart earring, or it may include a more or less traditional device such as a smartphone, smart pad, desktop computer, server, notebook computer, workstation, personal digital assistant (PDA), web pad, mobile phone, remote controller, etc. Not only one example, but may be changed as much as possible within the scope that can achieve the object of the present invention. In addition, the device according to an embodiment of the present invention includes a camera module (not shown) for photographing a control means (eg, a pointer, a user's eye or fingertip, etc.) or a camera module through a known communication network. (or another device including a camera module) may be able to communicate.

Meanwhile, the above device according to an embodiment of the present invention may include an application for supporting object control according to the present invention. Such an application may be downloaded from an external application distribution server (not shown). On the other hand, the characteristics of such a program module may be generally similar to the instruction vector determiner 110 , the control location manager 120 , the communication unit 130 , and the controller 140 of the object control support system 100 to be described later. have. Here, at least a part of the application may be replaced with a hardware device or a firmware device capable of performing substantially the same or equivalent function as the application, if necessary.

First, the indication vector determiner 110 according to an embodiment of the present invention refers to at least one of whether a trigger event related to the movement of the control means occurs and the distance between the motion coordinates of the control means and the control target area. can perform the function of determining The trigger event related to the movement of the control means according to an embodiment of the present invention may include a change of the movement direction of the control means, a stop of movement of the control means, and the like. More specifically, such a trigger event may include a control means moving backward after moving forward or stopping after moving forward. Such forward and backward directions may be specified based on a control target region, a specific body part of the user (eg, an eye), or a virtual reference point to be described later. Here, the control target area according to an embodiment of the present invention may mean an area in which at least one object controllable by a user is displayed.

For example, when a trigger event related to the movement of the control means is generated, the indication vector determiner 110 coordinates at a time point at which the trigger event related to the movement of the control means occurs among the motion coordinates of the control means. The first coordinate can be determined by referring to as the trigger coordinate, the distance between the trigger coordinate and the motion coordinate, the straight line section specified by the trigger coordinate and the motion coordinate, the distance between the first coordinate and the motion coordinate, and the first coordinate and the motion The second coordinate may be determined with reference to at least one of the straight line sections specified by the coordinates, and a vector determined based on the first coordinate and the second coordinate (hereinafter referred to as a “motion vector”) may be determined as an indication vector. can More specifically, the pointer vector determiner 110 may determine a motion vector having the second coordinate as the starting point and the first coordinate as the end point. Details related to the method of determining such a motion vector will be described later.

For another example, the indication vector determiner 110 determines that the distance between the motion coordinates of the control means and the control target area is less than or equal to a predetermined level (for example, 2.5 cm or less), or a trigger event related to the movement of the control means occurs. and the above motion vector is invalid, a vector (hereinafter, referred to as a “vertical vector”) passing through the above trigger coordinates or the first coordinates and perpendicular to the control target area may be determined as an indication vector.

More specifically, the indication vector determiner 110 generates a trigger event in which the control means stops after advancing, and the first coordinate (or the motion coordinate of the control means) has a distance from the control target area within a predetermined distance (more specifically, . It can be determined as a vector having a predetermined point of the control target region existing in the . In addition, the indication vector determiner 110 generates a trigger event in which the control means moves forward and then backward, and the first coordinate (or the motion coordinate of the control means) has a distance from the control target area within a predetermined distance (more specifically, 1 cm). to 2.5 cm) and the above motion vector is invalid, a vector passing through the trigger coordinates and perpendicular to the control target area (eg, a control target having the trigger coordinates as a starting point and the shortest distance from the first coordinates) A vector having a predetermined point of the region as an end point) can be determined as an indication vector. In addition, the instruction vector determiner 110 determines that the first coordinate (or the motion coordinate of the control means) is within a predetermined distance (more specifically, 0 cm to 1 cm) and the above motion vector is invalid. In this case, a vector passing through the first coordinate and perpendicular to the control target region is an indication vector (for example, a vector having the first coordinate as the starting point and a predetermined point of the control target region located at the shortest distance from the first coordinate as the ending point) ) can be determined as

As another example, when the distance between the motion coordinates of the control means and the control target area is at least a predetermined level (eg, 8 cm or more), the indication vector determiner 110 determines the motion coordinates of the control means and the user's A vector (hereinafter, referred to as a "reference vector") specified based on body coordinates can be determined as a reference vector (for example, a vector having the user's body coordinates as the starting point and the motion coordinates of the control means as the endpoint). have. The user's body coordinates according to an embodiment of the present invention may include body coordinates for various body parts such as the user's eyes (eg, dominant eyes, both eyes), head, hands, and fingertips. On the other hand, when the control means according to the present invention is the user's body part, the indication vector may be determined based on the corresponding body part and other body parts of the user that are not identical thereto.

More specifically, when the control means is the user's fingertip and the distance between the motion coordinate of the user's fingertip and the control target area is 8 cm or more, the instruction vector determiner 110 controls the user's eye (specifically, A vector having the coordinates of the dominant eye as the starting point and the motion coordinates of the fingertip as the ending point can be determined as the pointing vector.

2 is a diagram exemplarily illustrating a process in which an indication vector is dynamically determined according to a distance between a motion coordinate of a control means and a control target area according to an embodiment of the present invention.

Referring to FIG. 2 , first, the motion coordinates of the control means according to an embodiment of the present invention are the coordinates of the user's fingertip (ie, the first body coordinates), and the motion coordinates of the control means are the second coordinates from the control target area. A vector that is specified based on the coordinates of the user's fingertips (i.e., first body coordinates) and the user's eye coordinates (i.e., second body coordinates) when present at a distance of 2 or more (e.g., 8 cm) 201 and 202) may be determined as an indication vector. For example, when the motion coordinates of the control means exist at a first distance (eg, 30 cm) or more from the control target area, the coordinates of the user's fingertip (ie, the first body coordinates) and the coordinates of the user's dominant eye A vector 201 connecting (that is, second body coordinates) may be determined as a pointing vector, and the motion coordinates of the control means are greater than or equal to a second distance (eg, 8 cm) and less than the first distance (eg, 8 cm) from the control target area ( For example, 30 cm), (i) a vector 201 connecting the coordinates of the user's fingertip (ie, the first body coordinates) and the coordinates of the user's dominant eye (ie, the second body coordinates) or (ii) A vector 202 connecting the coordinates of the user's fingertips (ie, first body coordinates) and the center coordinates of both eyes of the user (ie, second body coordinates) may be determined as the pointing vector.

Then, according to an embodiment of the present invention, when the motion coordinates of the control means are greater than or equal to a third distance (eg, 2.5 cm) and less than a second distance (eg, 8 cm) from the area to be controlled is any one of the vectors 201 and 202 and the motion vector 203 specified based on the coordinates of the user's fingertips (ie, the first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates). A vector may be determined as an indicator vector. For example, if the vectors 201 and 202 specified based on the coordinates of the user's fingertips (ie, first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) can be determined, the user The vectors 201 and 202 specified based on the coordinates of the fingertips (ie, first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) are determined as the pointing vectors, and The motion vector 203 is indicated when the vectors 201 and 202 specified based on the coordinates (ie, the first body coordinates) and the coordinates of the user's eyes (ie the second body coordinates) cannot be determined or are not valid. It can be determined as a vector.

Then, according to an embodiment of the present invention, when the motion coordinates of the control means are less than a third distance (eg, 2.5 cm) from the control target area, the coordinates of the user's fingertip (ie, the first Any one of the vectors 201 and 202, the motion vector 203, and the vertical vector 204 specified based on the body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) may be determined as the pointing vector. can For example, if the vectors 201 and 202 specified based on the coordinates of the user's fingertips (ie, first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) can be determined, the user The vectors 201 and 202 specified based on the coordinates of the fingertips (ie, first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) are determined as the pointing vectors, and If the vectors 201 and 202 specified based on the coordinates (ie, the first body coordinates) and the coordinates of the user's eyes (ie the second body coordinates) cannot be determined or are invalid, the motion vector 203 is indicated It can be determined as a vector. In addition, the vectors 201 and 202 and the motion vector 203 that are specified based on the coordinates of the user's fingertip (ie, the first body coordinates) and the coordinates of the user's eyes (ie, the second body coordinates) may be determined. In the case of missing or invalid, a vertical vector (eg, a vector passing through a trigger coordinate or a first coordinate) 204 that is a vector perpendicular to the control target region may be determined as an indication vector.

Next, when the position of the control means is changed, the control position management unit 120 according to an embodiment of the present invention has a virtual reference point and a position above that are specified based on the extension line of the indication vector before the position is changed. After the change, a function of determining a control position in a control target region may be performed by referring to a vector (hereinafter, referred to as a “connection vector”) connecting the motion coordinates of the control means.

For example, when the position of the control means is changed, the control position management unit 120 specifies, as a virtual reference point, a predetermined point existing on an extension line in the opposite direction to the direction in which the pointing vector is directed before the position is changed. After the virtual reference point and the above position are changed, a vector connecting the motion coordinates of the control means can be determined, and the point (or area) where the extension line of the above connection vector and the control target area intersect is determined as the control position. can

More specifically, referring to FIG. 3 , when the indication vector is a motion vector and the position of the control means is changed, the control position management unit 120 determines the direction in which the motion vector 310 is directed before the position is changed (301). A point 40 to 50 cm away from the motion vector (for example, the start or end point of the motion vector) among predetermined points existing on the extension line in the opposite direction to the virtual reference point 302 is specified as the virtual reference point ( 302) and a vector 320 (that is, a connection vector) connecting the motion coordinates 303 of the control means after the position is changed, and the vector 320 or an extension line of the vector 320 and A point where the control target region intersects may be determined as the control position 330 .

In addition, referring to FIG. 4 , when the indication vector is a vertical vector and the position of the control means is changed, the control position management unit 120 controls the direction opposite to the direction in which the vertical vector 410 is directed before the position is changed 401 ( 401 ). A point 40 to 50 cm away from a vertical vector (for example, a starting point or an end point of a vertical vector) among predetermined points existing on the extension line of the direction is specified as a virtual reference point 402, and the virtual reference point 402 After the position and the above are changed, a vector 420 (ie, a connection vector) connecting the motion coordinates 403 of the control means can be determined, and the vector 420 or an extension line of the vector 420 and the control object A point where the regions intersect may be determined as the control position 430 .

In addition, when the position of the control means is changed, the control position management unit 120 includes a first guide vector specified with reference to the motion coordinates of the control means before the position is changed and the motion coordinates of the control means after the position is changed. It is possible to perform a function of determining a control position in the control target region with reference to the corrected reference vector based on the relationship (eg, positional relationship) between the second guide vectors specified with reference to .

For example, when the position of the control means is changed, the control position management unit 120 includes a first guide vector that is specified with reference to the motion coordinates of the control means before the position is changed and the control means after the position is changed. Translate the pointing vector based on the degree of change in the position or direction between the second guide vectors specified with reference to the motion coordinates (eg, the position between the first guide vector and the second guide vector (more specifically, each The end point of the guide vector or the position between each start point) is changed as much as the translation) or the direction the pointing vector is directed (for example, the direction or the direction between the first guide vector and the second guide vector is changed as the angle is changed). and a point (or region) at which the corrected pointer vector or an extension line of the corrected pointer vector and the control target region intersect may be determined as the control position. This guide vector is a vector set to specify a direction associated with the control means (or the user's control intention), for example, by connecting the motion coordinates of the control means and other coordinates of the control means associated with the motion coordinates. It may be a specified vector. More specifically, when the motion coordinates of the control means are the coordinates of the index finger tip of the hand, a vector (for example, a finger vector) having the index finger tip as the end point and the joint (or joint) coordinates of the index finger as the starting point is a guide When specified by a vector, or when the motion coordinates of the control means are palm-center coordinates, a vector having the tip of the index finger as the end point and the palm-center coordinate as the starting point may be specified as the guide vector. On the other hand, it should be noted that the above other coordinates according to the present invention are not necessarily limited to other coordinates of the control means, and may be changed to coordinates of other objects adjacent to the control means other than the control means.

More specifically, referring to FIG. 5 , when the indication vector is a motion vector 510 and the position of the control unit is changed, the control position management unit 120 controls the motion coordinates 501 of the control unit before the position is changed. (For example, the first finger vector 511 specified with reference to (for example, the coordinates of the index finger tip of the hand) (for example, a vector having the coordinates of the tip of the index finger as the end point and the coordinates of the node of the index finger as the starting point) (i.e. , guide vector) and the degree of change 551 of the position and the degree of change of direction 552 between the second finger vector 521 specified with reference to the motion coordinates 503 of the control means after the position is changed Referring to, the motion vector 510 is translated (eg, translated by its position change) and corrected by correcting the direction the motion vector 510 faces (eg, rotational transformation by its direction change). A motion vector can be determined. Then, a point at which the corrected motion vector or an extension line of the corrected motion vector and the control target region intersect may be determined as the control position 530 .

In addition, referring to FIG. 6 , when the indication vector is a vertical vector 610 and the position of the control means is changed, the control position management unit 120 controls the motion coordinates 601 of the control means before the position is changed (eg, For example, the first finger vector 611 specified with reference to the coordinates of the index finger tip of the hand (for example, a vector having the coordinates of the index finger tip as the end point and the coordinates of the index finger node as the starting point) (that is, the guide vector) and the second finger vector 621 specified with reference to the motion coordinate 603 of the control means after the position is changed, with reference to the degree of change in the position and the degree of change in the direction, the vertical vector 610 The corrected vertical vector may be determined by translating (eg, translating by the position change) and correcting the direction in which the vertical vector 610 is directed (eg, rotational transformation by the direction change). Thereafter, a point at which the corrected vertical vector 620 or an extension line of the corrected vertical vector 620 and the control target region intersect may be determined as the control position 630 .

Meanwhile, a case in which the direction of the control means is further changed according to an embodiment of the present invention may be considered. Here, the direction of the control means means a direction associated with the user's control intention, and is specified based on the motion coordinates of the control means and other coordinates of the control means associated with the motion coordinates or is specified based on the shape of the control means. it could be For example, when the motion coordinates of the control means are coordinates of the tip of the index finger of the hand, the direction indicated by the index finger may be the direction of the control means. The direction of the control means may be the same as the direction of the guide vector described above, but may be set differently as needed.

For example, according to an embodiment of the present invention, when the position and direction of the control unit is changed, the control position management unit 120 refers to the motion coordinates of the control unit and the direction of the control unit before the position and direction is changed. Specific (for example, the motion coordinate of the control means before the position and direction is changed as the starting point is specified as the end point, which extends in the same or parallel direction to the direction of the control means and meets the control target area as the end point, or The first guide vector (specified by extending a predetermined length in the same or parallel direction to the direction of the control means as the starting point of the motion coordinates of the control means before the position and direction of With reference to the motion coordinates of the means and the direction of the control means, a specific (for example, the motion coordinates of the control means after the above position and direction are changed, as a starting point, extend in the same or parallel direction as the direction of the control means to be controlled A second guide that is specified using the point that meets the region as the end point, or extended by a predetermined length in the same or parallel direction to the direction of the control means from the starting point of the motion coordinates of the control means before the position and direction are changed) A function of determining a control position in a control target region may be performed with reference to a reference vector corrected based on a relationship between vectors (eg, a relationship between positions or directions).

In addition, the control location management unit 120 may determine a vector to be used for object control of the user from among the above connection vectors and the above corrected instruction vectors with reference to the user's object control pattern (or the user's preferred control pattern), and , a point where the extension line of the vector and the control target area intersect may be determined as the control position.

For example, the control position management unit 120 may include a vector (eg, accuracy) that is specified by referring to a vector frequently used by the user at a predetermined level or more among the above connection vector and the above corrected instruction vector, and the user's feedback. Alternatively, a vector specified as having high satisfaction) or a vector specified by analyzing or learning a user's usage pattern may be determined as a vector to be used for object control of the corresponding user.

Next, according to an embodiment of the present invention, the communication unit 130 may perform a function of enabling data transmission/reception to/from the indication vector determiner 110 and the control position management unit 120 .

Finally, according to an embodiment of the present invention, the control unit 140 may perform a function of controlling the flow of data between the indication vector determiner 110 , the control location management unit 120 , and the communication unit 130 . That is, the control unit 140 according to the present invention controls the data flow to/from the outside of the object control support system 100 or the data flow between each component of the object control support system 100, so that the indication vector determining unit ( 110), the control location management unit 120, and the communication unit 130 may be controlled to perform their own functions, respectively.

Determination of motion vectors

Hereinafter, the process of determining the motion vector discussed above will be described in detail.

The indication vector determiner 110 according to an embodiment of the present invention determines the first coordinates by referring to the coordinates at the time point at which the trigger event related to the movement of the control means occurs among the motion coordinates of the control means as the trigger coordinates. It can perform a decision-making function.

For example, the indication vector determiner 110 may determine the coordinates at the point in time at which a trigger event in which the control means advances toward the control target area and then stops occurs among the motion coordinates of the control means, that is, the trigger coordinates as the first coordinates. have.

For another example, when a trigger event in which the control means moves forward and backward toward the control target area occurs, the indication vector determiner 110 may control the motion coordinates of the control means at the time when the trigger event occurs, that is, the trigger. Based on the coordinates, the motion coordinates of the control means at a predetermined previous time point (eg, the immediately preceding time point) may be determined as the first coordinates. Here, the predetermined previous time point may be specified based on a shooting interval or a frame rate of a shooting module (eg, a camera) for shooting the control means.

In addition, shaking of the control means may be generated at the time when the trigger event is generated. In order to correct this, the indication vector determiner 110 refers to at least one motion coordinate of the control means specified based on the trigger coordinates. 1 Coordinates can be determined.

For example, the indication vector determiner 110 may determine the first coordinates by statistically analyzing a plurality of motion coordinates of the control means specified for a predetermined time based on the timing at which the trigger coordinates are specified. Statistical analysis according to an embodiment of the present invention may include analysis based on average, weighted average, variance, standard deviation, and the like for a plurality of motion coordinates. More specifically, the pointer vector determiner 110 may determine the average coordinates of the plurality of motion coordinates specified for 0.01 seconds to 0.1 seconds based on the timing at which the trigger coordinates are specified as the first coordinates.

As another example, the indication vector determiner 110 may determine the first coordinates by statistically analyzing a plurality of motion coordinates of the control means existing within a predetermined distance from the trigger coordinates. More specifically, the indication vector determiner 110 may determine the average coordinates of a plurality of motion coordinates of the control means specified within 5 to 10 mm from the trigger coordinates as the first coordinates.

Meanwhile, the indication vector determiner 110 may exclude at least one of the trigger coordinates and motion coordinates within a predetermined distance from the trigger coordinates from the target of statistical analysis.

For example, when the control means stops after moving forward, or when a trigger event in which the control means moves backward after moving forward occurs, the indication vector determiner 110 determines the trigger coordinates and the trigger coordinates specified based on the time at which the trigger event occurs. Since motion coordinates within 5 mm from the trigger coordinates may have a large shake, the trigger coordinates and motion coordinates within 5 mm from the trigger coordinates may be excluded from statistical analysis.

In addition, the indication vector determiner 110 may determine the second coordinate with reference to the trigger coordinate or the distance between the first coordinate and the motion coordinate of the control means.

For example, the indication vector determiner 110 may determine the motion coordinates of the control means as the second coordinates when the trigger coordinates or the distance between the first coordinates and the motion coordinates of the control means is equal to or greater than a predetermined level. More specifically, referring to FIG. 7 , the indication vector determining unit 110 is the motion coordinates 702 of the control means when the distance between the trigger coordinates 701 and the motion coordinates of the control means is 40 to 50 mm or more. may be determined as the second coordinate.

For another example, the indication vector determiner 110 uses the motion coordinates of the closest point with respect to the point at which the first coordinate is specified among the motion coordinates of the control means at which the distance from the trigger coordinate is equal to or greater than a predetermined level as the second coordinate. can decide More specifically, referring to FIG. 7 , the indication vector determiner 110 is the closest point based on the point at which the first coordinate 703 is specified among the motion coordinates of the control means having a distance of 50 mm or more from the trigger coordinate 701 . The motion coordinates 702 of may be determined as the second coordinates.

In addition, the indication vector determiner 110 may determine the second coordinate with reference to the trigger coordinate or a straight line section specified by the first coordinate and the motion coordinate of the control means.

For example, the indication vector determiner 110 determines the first coordinates (or trigger coordinates) above the first coordinates (or trigger coordinates) for each of the motion coordinates of the control means at other temporally adjacent time points based on the point at which the first coordinates (or trigger coordinates) are specified. ) and the motion coordinates present at the farthest distance from the above first coordinates (or trigger coordinates) among motion coordinates in which a straight line section is specified may be determined as the second coordinates. Here, all the temporally adjacent motion coordinates from the time point at which the first coordinate (or trigger coordinate) is specified to the other time point above are the first coordinate (or trigger coordinate) and the motion coordinate of the control means at the other time point above. If it exists within a predetermined distance from a straight line connecting the , the straight line section may be specified.

More specifically, in the indication vector determiner 110, the time point at which the first coordinate (or trigger coordinate) is specified is the first time point, and the motion coordinate of the control means at the second time point temporally adjacent to the first time point is the first time point. is the second motion coordinate, the motion coordinates of the control means at a third time point temporally adjacent to the second time point are the third motion coordinates, and control at a fourth time point temporally adjacent to the third time point When the motion coordinates of the means are the fourth motion coordinates, when the second motion coordinates are within a predetermined distance from the straight line connecting the first coordinates (or trigger coordinates) and the third motion coordinates, the pointing vector determining unit 110 ) may be specified that the first coordinates (or trigger coordinates), the second motion coordinates, and the third motion coordinates are straight sections. In addition, when both the second motion coordinates and the third motion coordinates are within a predetermined distance from the straight line connecting the first coordinates (or trigger coordinates) and the fourth motion coordinates, the pointing vector determiner 110 determines the first coordinates (or trigger coordinates), the second motion coordinates, the third motion coordinates, and the fourth motion coordinates may be specified as straight sections.

In addition, the indication vector determiner 110 may be specified by connecting the motion coordinates of the control means at other temporally adjacent times based on the timing at which the first coordinates (or trigger coordinates) are specified in temporally adjacent order, respectively. The longest straight line section among the straight line sections is specified, and the motion coordinates located at the farthest distance from the first coordinates (or trigger coordinates) among a plurality of motion coordinates of the control means existing in the longest straight line section are determined as the second coordinates. can Meanwhile, an interval between a plurality of motion coordinates of the control means existing in a straight line section may be within a predetermined distance.

In addition, referring to FIG. 8 , in the indication vector determiner 110 , the first time point at which the first coordinates 821 are specified is the first time point, and the motion of the control means at the second time point temporally adjacent to the first time point. The coordinates are the second motion coordinates 822 , the motion coordinates of the control means at the third time point temporally adjacent to the second time point are the third motion coordinates 823 , and the third motion coordinates are temporally adjacent to the third point point. When the motion coordinates 824 of the control means at the 4th time point are the fourth motion coordinates, the first straight line section connecting the first coordinates 821 and the second motion coordinates 822, the first coordinates 821 and A second straight line section connecting the third motion coordinates 823 and a third straight line section connecting the first coordinates 821 and the fourth motion coordinates 824 are determined as a specific possible straight line section, and the plurality of straight line sections (ie, the first straight section, the second straight section, and the third straight section) of the plurality of motion coordinates within the third straight section existing in the longest straight section, the motion that exists at the farthest distance from the first coordinate 821 The coordinates 824 may be determined as the second coordinates.

On the other hand, the indication vector determiner 110 is a coordinate determined with reference to the distance between the trigger coordinates (or first coordinates) and the motion coordinates of the control means and the first coordinates (or trigger coordinates) and the motion coordinates of the control means. A coordinate close to the first coordinate (or trigger coordinate) among coordinates determined with reference to the specified straight line section may be determined as the second coordinate.

For example, referring to FIG. 9 , the indication vector determining unit 110 (i) the motion coordinates 910 and 912 of the control means at the point in time when the distance between the trigger coordinates and the motion coordinates of the control means becomes greater than or equal to a predetermined level. , 914, 916, 920, 922) and (ii) a straight line section is specified by connecting each of the motion coordinates of the control means at other temporally adjacent time points with the first coordinates based on the point at which the first coordinates are specified. Among the possible motion coordinates, the motion coordinates (911, 913, 915, 917, 921, 923) located at the farthest distance from the above first coordinates are determined, and a plurality of motion coordinates (910, 911, 912, 913, 914) are determined. , 915 , 916 , 917 , 920 , 921 , 922 , 923 of motion coordinates 911 , 913 , 915 , 917 , 920 , and 922 close to the first coordinate may be determined as the second coordinate.

Also, the pointer vector determiner 110 may determine a motion vector determined based on the first coordinate and the second coordinate.

For example, the pointer vector determiner 110 may determine a vector having the second coordinate as the starting point and the first coordinate as the end point as the motion vector.

On the other hand, the indication vector determiner 110 may determine that the motion vector cannot be specified when there is no motion coordinate in which the distance between the trigger coordinate or the first coordinate and the motion coordinate of the control means is equal to or greater than a predetermined level. .

For example, referring to FIG. 10 , the indication vector determining unit 110 includes the trigger coordinates or the second coordinates 1001 and 1002 determined with reference to a straight line section specified by the first coordinate and the motion coordinate of the control means. Although it may be specified, if the second coordinate determined with reference to the trigger coordinate or the distance between the first coordinate and the motion coordinate of the control means cannot be specified, it may be determined that the motion vector cannot be specified.

In addition, the indication vector determiner 110 may verify the validity of the motion vector by referring to at least one of the length, speed, direction, and position of the first coordinate of the motion vector.

For example, when a length obtained by scaling a length of a motion vector by a predetermined multiple is longer than a length between the control target region and the first coordinates, the pointer vector determiner 110 may determine the corresponding motion vector as a valid motion vector.

For another example, the indication vector determiner 110 may specify an effective region based on the length of the motion vector, and when a control target region exists within the effective region, the indication vector determiner 110 may determine the corresponding motion vector as a valid motion vector. More specifically, referring to FIG. 11 , the indication vector determiner 110 scales the length of the motion vector by a predetermined level to specify an area (or extended area) specified as the effective area 1102 , and the effective area When a common area exists between 1102 and the control target area, the corresponding vector may be determined as a valid motion vector.

As another example, when the length of the motion vector is longer than a predetermined length (eg, 10 to 20 mm), the indication vector determiner 110 may determine the corresponding motion vector as a valid motion vector.

As another example, when the speed of the motion vector is greater than or equal to a predetermined speed (eg, 10 mm/s to 20 mm/s), the indication vector determiner 110 may determine the corresponding motion vector as a valid motion vector.

For another example, referring to FIG. 12 , the indication vector determiner 110 determines that the angle formed by the motion vector and the control target region (specifically, the angle formed by the normal vector of the control target region) is within a predetermined angle (eg, For example, when -45 degrees ≤ θ _x ≤ 45 degrees and 30 degrees ≤ θ _y ≤ 60 degrees), the corresponding motion vector may be determined as a valid motion vector.

As another example, when the first coordinate (ie, the end point of the motion vector) above is within a predetermined distance (eg, within 100 mm) of the control target area, the indication vector determiner 110 may determine the corresponding motion. A vector can be determined as a valid motion vector.

Meanwhile, referring to FIGS. 13 and 14 , the validity of the motion vector may be verified using the gaze vector specified by the user's gaze or head pose.

For example, between a first control position specified in the control target region using the motion vector according to the present invention and a second control position specified in the control target region using a gaze vector specified by the user's gaze or head pose. When the error of is less than or equal to a predetermined level, the corresponding motion vector may be determined as a valid motion vector.

first embodiment

Hereinafter, a situation in which object control according to the present invention is supported to a user using a device including the object control support system 100 according to an embodiment of the present invention will be described. The control means according to an embodiment of the present invention may be a user's fingertip (eg, index finger tip).

First, in the device according to an embodiment of the present invention, when the distance between the motion coordinate of the control means and the control target area is less than a predetermined level (for example, between 8 cm) and a trigger event related to the movement of the control means occurs , among the motion coordinates of the user's fingertip, the first coordinates may be determined by referring to the coordinates at the point in time at which the trigger event that stops after advancing in the direction advancing toward the control target area occurs as the trigger coordinates.

For example, when a trigger event in which the user's fingertip moves forward in the direction toward the control target area and stops occurs occurs, the coordinates at the time the trigger event occurs among the motion coordinates of the user's fingertip, that is, the trigger The coordinates may be determined as the first coordinates.

Then, the second coordinate may be determined with reference to the distance between the trigger coordinates above and the motion coordinates of the user's fingertip and the straight line section specified by the above first coordinates and the motion coordinates of the user's fingertips.

Specifically, the coordinates determined by referring to the distance between the trigger coordinates above and the motion coordinates of the user's fingertips and the first coordinates (or trigger coordinates) and the straight line section specified by the motion coordinates of the user's fingertips Among the determined coordinates, a coordinate close to the above first coordinate may be determined as the second coordinate.

Then, referring to at least one of the length, velocity, direction and position of the first coordinate of a vector (i.e., motion vector) specified with the above second coordinate as the starting point and the above first coordinate as the end point Thus, the validity of the above motion vector can be verified.

Then, when it is determined that the above motion vector is valid, the motion vector may be determined as an indication vector, and a region of the control target region that meets the extension line of the motion vector may be determined as a control position intended by the user.

On the other hand, according to an embodiment of the present invention, the direction associated with the occurrence of the trigger event described above (for example, a direction that is a reference for whether the trigger event occurs) is the type of the indication vector (or the indication vector is specified). reference) or it may be determined according to the distance between the motion coordinates of the control means and the control target area.

For example, in the case of a reference vector among the indication vectors according to an embodiment of the present invention, its direction (ie, a direction as a reference for whether a trigger event occurs or not) based on a specific body part (eg, eye) of the user. ) can be specified, and in the case of a motion vector among the indication vectors, its direction (that is, a reference direction of whether a trigger event occurs) based on a virtual reference point specified based on the motion vector (or an extension line thereof). can be specified, and in the case of a vertical vector among the indication vectors, its direction (ie, a direction that is a criterion for whether a trigger event occurs) is specified based on the control target area, or is specified based on a vertical vector (or an extension line thereof) A direction (ie, a direction as a reference for whether or not a trigger event occurs) may be specified based on a virtual reference point.

For another example, when the distance between the motion coordinates of the control means and the control target area according to an embodiment of the present invention is equal to or greater than the first distance (eg, 8 cm), the user's specific body part (eg, eyes ) may be specified based on the direction (that is, a direction that is a criterion for whether a trigger event occurs or not), and the distance between the motion coordinates of the control means and the control target area is less than the first distance and the second distance (eg, For example, 2.5 cm) or more, the direction (that is, the criterion of whether a trigger event occurs) is direction) can be specified, and when the distance between the motion coordinates of the control means and the control target area is less than the second distance (for example, 0 cm or more and less than the second distance), the direction ( That is, the direction (i.e., the direction that is the basis of whether the trigger event occurs) is specified or the direction (ie, the virtual reference point specified based on a vertical vector (or an extension line thereof)) is specified. direction) as a criterion for whether a trigger event occurs or not may be specified. On the other hand, the method of dividing the distance according to the present invention (eg, the number of sections) is not necessarily limited to those listed above, and various modifications such as more subdivision within the range that can achieve the object of the present invention make it clear that it can be

Then, referring to FIG. 15 , when the position of the user's fingertip is changed, the virtual reference point specified based on the extension line of the motion vector before the position is changed and after the position is changed It becomes the control position to be changed in the control target area by referring to the vector (ie, the connection vector) that connects the motion coordinates of the user's fingertips, or is specified by referring to the motion coordinates of the user's fingertips before the above position is changed Refer to the corrected upper motion vector (ie, the corrected motion vector) based on the relationship between the first guide vector and the second guide vector specified with reference to the motion coordinates of the user's fingertip after the position of the stomach is changed Thus, the control position to be changed in the control target area may be determined. Meanwhile, a vector to be used for object control of a corresponding user among the above connection vectors and the above corrected motion vectors may be determined with reference to the user's object control pattern (or an object control method preset to the user).

second embodiment

Hereinafter, another situation in which object control according to the present invention is supported to a user using a device including the object control support system 100 according to an embodiment of the present invention will be described. The control means according to an embodiment of the present invention may be a user's fingertip (eg, index finger tip).

First, in the device according to an embodiment of the present invention, when the distance between the motion coordinates of the control means and the control target area is less than or equal to a predetermined level (for example, 2.5 cm or less) and a trigger event related to movement of the control means occurs For example, the first coordinates may be determined by referring to coordinates at a time point at which a trigger event that stops after moving forward in a direction advancing toward the control target area occurs among motion coordinates of the user's fingertip as the trigger coordinates.

Then, the second coordinate may be determined with reference to the distance between the trigger coordinates above and the motion coordinates of the user's fingertip and the straight line section specified by the above first coordinates and the motion coordinates of the user's fingertips.

Then, with reference to at least one of a length, a velocity, a direction of a vector (ie, a motion vector) determined based on the first coordinates and the second coordinates, and a position of the first coordinates above, Validity can be verified.

Then, when a vector (ie, a motion vector) determined based on the above first coordinates and second coordinates is invalid, the above trigger coordinates or a vector passing through the first coordinates and perpendicular to the control target area 1601 (ie, a vertical vector) may be determined as the pointer vector.

Then, referring back to FIG. 15 , when the position of the user's fingertip is changed above, the virtual reference point specified based on the extension line of the vertical vector before the position is changed and the position is changed. Then, the control position to be changed in the control target area is determined by referring to the vector (that is, the connection vector) that connects the motion coordinates of the user's fingertips, or by referring to the motion coordinates of the user's fingertips before the above position is changed. The upper vertical vector corrected based on the relationship between the specified first guide vector and the second guide vector specified with reference to the motion coordinates of the user's fingertip after the position of the stomach is changed (ie, the corrected vertical vector) A control position to be changed in the control target region may be determined with reference to . Meanwhile, with reference to the user's object control pattern (or an object control method preset to the user), a vector to be used for object control of the corresponding user among the above connection vector and the above corrected vertical vector may be determined.

Extensive embodiments associated with the identification of virtual reference points

In the process where the control means according to an embodiment of the present invention approaches or moves away from the control target area, the indication vector is determined for each time point according to whether a trigger event occurs or the distance between the motion coordinates of the control means and the control target area. It may be determined (ie, a plurality of indication vectors may be determined). Hereinafter, a detailed situation in which a virtual reference point and a control position are determined in this case will be described.

For example, referring to FIG. 17 , according to an embodiment of the present invention, the motion coordinate of the control means at time t is P _FTt , the virtual reference point at time t is RP _t , and control at time t When the control position in the target region is C _t , the motion vector is not determined effectively at the time t=-5, so a virtual reference point based thereon is not specified, and the control position C _-5 in the control region (1721) may not be determined. Then, when the motion vector is effectively determined at t=-4 according to an embodiment of the present invention, a virtual reference point RP -4 1702 may be specified based on the motion vector, and the virtual reference point RP _-4 1702 may be specified. Based on the point RP _-4 1702 and the motion coordinates P _FT-4 1712 of the control means, the control position C _-4 1722 in the control target area can be determined. Then, when the motion vector is effectively determined at the time t=-3 according to an embodiment of the present invention, a virtual reference point RP _-3 1703 may be specified based on the motion vector, and the virtual reference Based on the point RP _-3 (1703) and the motion coordinates P _FT-3 (1713) of the control means, the control position C _-3 (1723) in the control target region can be determined. Then, when the motion vector is effectively determined at the time t=-2 according to an embodiment of the present invention, a virtual reference point RP _-2 1704 may be specified based on the motion vector, and the virtual reference Based on the point RP _-2 (1704) and the motion coordinates P _FT-2 (1714) of the control means, the control position C _-2 (1724) in the control target region can be determined. Then, when the motion vector is effectively determined at t=-1 according to an embodiment of the present invention, a virtual reference point RP _-1 (1705) may be specified based on the motion vector, and the virtual reference Based on the point RP _-1 (1705) and the motion coordinates P _FT-1 (1715) of the control means, the control position C _-1 (1725) in the control target region can be determined. Then, if the motion vector is not effectively determined at time t=0 according to an embodiment of the present invention, a virtual reference point based thereon is not specified, and a valid virtual reference point at the previous time point (specifically, immediately before Alternatively, the control position C ₀ 1726 in the control target region may be determined based on RP _-1 1705 (a valid virtual reference point at the latest time) and the motion coordinate P _FT0 1716 of the control means.

In addition, a process in which the current virtual reference point is corrected based on an effective virtual reference point at a previous time point (specifically, an effective virtual reference point at the immediately preceding or recent time point) according to an embodiment of the present invention will be described. let's see

Referring to FIG. 18 , according to an embodiment of the present invention, first, if a motion vector is not effectively determined at time t=-5, a virtual reference point based on this is not specified and a control position C _in the control target region is not specified. ₅ (1821) may also not be determined. Then, when the motion vector is effectively determined at the time t=-4 according to an embodiment of the present invention, a virtual reference point RP _-4 1802 may be specified based on the motion vector, and the virtual reference Based on the point RP _-4 (1802) and the motion coordinates P _FT-4 (1812) of the control means, the control position C _-4 (1822) in the area to be controlled can be determined. Then, when the motion vector is effectively determined at t = -3 time point according to an embodiment of the present invention, a virtual reference point RP _-3 1803 may be specified based on the motion vector, and the previous time point ( Or based on the valid virtual reference point RP _-4 (1802) and the current virtual reference point RP _-3 (1803) at the immediately preceding time point (for example, specified through statistical analysis of the coordinates of two points) In this statistical analysis, an imaginary reference point RP' _-3 (1806) and the motion coordinate P _FT-3 (1813) of the control means (mean, weighted average, variance, standard deviation, etc. may be used) The control position C' _-3 (1823) in the control target region may be determined based on . Then, when the motion vector is effectively determined at the time t=-2 according to an embodiment of the present invention, a virtual reference point RP _-2 1804 may be specified based on the motion vector, and at the previous time point Based on the valid virtual reference point RP' _-3 (1806) and the current virtual reference point RP _-2 (1804) of (for example, it can be specified through statistical analysis of the coordinates of the two points, In this statistical analysis, a mean, weighted mean, variance, standard deviation, etc. may be used) and control based on the virtual reference point RP' _-2 (1807) and the motion coordinate P _FT-2 (1814) of the control means. Control position C' _-2 (1824) in the target area can be determined. Then, when the motion vector is effectively determined at the time t = -1 according to an embodiment of the present invention, a virtual reference point RP _-1 1805 may be specified based on the motion vector, and at the previous time point Based on the valid virtual reference point RP' _-2 (1807) and the current virtual reference point RP _-1 (1805) of (for example, it can be specified through statistical analysis of the coordinates of the two points, In this statistical analysis, a mean, weighted mean, variance, standard deviation, etc. may be used) and control based on the virtual reference point RP' _-1 (1808) and the motion coordinate P _FT-1 (1815) of the control means. A control position C' _-1 (1825) in the target area can be determined. Then, if the motion vector is not effectively determined at time t=0 according to an embodiment of the present invention, a virtual reference point based thereon is not specified, and a valid virtual reference point (specifically, The control position C' ₀ (1826) in the control target area can be determined based on RP' _-1 (1808), which is a valid virtual reference point at the immediately preceding or recent time, and the motion coordinate P _FT0 (1816) of the control means. have.

Also, a process in which a virtual reference point is specified using only a valid motion vector according to an embodiment of the present invention will be described.

Referring to FIG. 19 , according to an embodiment of the present invention, if a motion vector is not effectively determined at time points t=-5 to -2, a virtual reference point based thereon is not specified and a control position in the control target region is not specified. C _-5 (1921), C _-4 (1922), C _-3 (1922) and C _-2 (1924) may also be undetermined. Then, when the motion vector is effectively determined at t = -1 according to an embodiment of the present invention, a virtual reference point RP -1 (1905) may be specified based on the motion vector, and the virtual reference point RP _-1 (1905) may be specified. Based on the reference point RP _-1 (1905) and the motion coordinate P _FT-1 (1915) of the control means, the control position C _-1 (1925) in the control target region can be determined. Next, if the motion vector is not effectively determined at time t=0 according to an embodiment of the present invention, a virtual reference point based thereon is not specified, and a valid virtual reference point at the previous time point (specifically, immediately before Alternatively, the control position C ₀ (1926) in the control target region can be determined based on RP _-1 (1905), which is a valid virtual reference point at the latest point in time) and the motion coordinate P _FT0 (1916) of the control means.

For another example, referring to FIG. 20 , according to an embodiment of the present invention, the motion coordinates of the control means at time t are P _FTt , and the virtual reference point at time t is RP _t , and at time t When the control position in the control target region is C _t , first, the motion vector and the vertical vector (eg, the distance between the control target region and the motion coordinates of the control means is a predetermined distance at t=-5 to -2 time points) Invalid due to the above) is not determined validly, so the virtual reference point based on this is not specified, and the control positions C _-5 (2021), C _-4 (2022), C _-3 ( 2023) and C _-2 (2024) may also be undetermined. Then, the motion vector is invalid and the vertical vector is valid (for example, if the distance between the control target area and the motion coordinates of the control means is less than or equal to a predetermined distance If the corresponding) is determined, a virtual reference point RP _-1 (2005) can be specified based on the vertical vector, and the virtual reference point RP _-1 (2005) and the motion coordinates of the control means P _FT-1 (2015) ), the position C ₋₁ ( 2025 ) in the control target region may be determined. Next, the motion vector and the vertical vector (for example, the distance between the control target area and the motion coordinates of the control means is invalid because it corresponds to a predetermined distance or more) at the time t = 0 according to an embodiment of the present invention If not determined to be valid, the virtual reference point based thereon is not specified, and RP _-1 (2005), which is a valid virtual reference point at a previous time (specifically, a valid virtual reference point at the immediately preceding or recent time) and control means Based on the motion coordinate P _FT0 (2016) of the control position C ₀ (2026) in the control target region can be determined.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a non-transitory computer-readable recording medium. The non-transitory computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the non-transitory computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the non-transitory computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks ( magneto-optical media), and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , those of ordinary skill in the art to which the present invention pertains can devise various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below, but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

100: object control support system
110: instruction vector determining unit
120: control position management unit
130: communication department
140: control unit

Claims (12)

A method for supporting object control, comprising:
determining an indication vector by referring to at least one of whether a trigger event related to movement of the control means occurs and a distance between a motion coordinate of the control means and a control target area; and
When the position of the control means is changed, a vector connecting the virtual reference point specified based on the extension line of the indication vector before the position is changed and the motion coordinates of the control means after the position is changed is referred to. determining a control position in the control target area;
Way. A method for supporting object control, comprising:
determining an indication vector by referring to at least one of whether a trigger event related to movement of the control means occurs and a distance between a motion coordinate of the control means and a control target area; and
When the position of the control means is changed, a first guide vector specified with reference to the motion coordinates of the control means before the position is changed and a second guide vector specified with reference to the motion coordinates of the control means after the position is changed Determining a control position in the control target region with reference to the pointer vector corrected based on a relationship between guide vectors
Way. 3. The method of any one of claims 1 and 2,
In the indication vector determining step, when the distance between the motion coordinates of the control means and the control target area is less than or equal to a predetermined level, a time point at which a trigger event related to movement of the control means is generated among the motion coordinates of the control means. ) by referring to the coordinates in the trigger coordinates to determine the first coordinates, the distance between the trigger coordinates and the motion coordinates, the straight line section specified by the trigger coordinates and the motion coordinates, the first coordinates and the motion coordinates A second coordinate is determined with reference to at least one of a distance between the first coordinates and a straight line section specified by the motion coordinates, and a vector determined based on the first coordinates and the second coordinates is used as the pointing vector to decide as
Way. 4. The method of claim 3,
In the indication vector determining step, when the distance between the motion coordinates of the control means and the control target region is less than a predetermined level or the vectors determined based on the first coordinates and the second coordinates are invalid, the trigger coordinates or passing through the first coordinate and determining a vector perpendicular to the control target area as the indication vector
Way. 3. The method of any one of claims 1 and 2,
In the indication vector determining step, when the distance between the motion coordinates of the control means and the control target area is equal to or greater than a predetermined level, a vector specified based on the motion coordinates of the control means and the user's body coordinates is used as the indication vector to decide
Way. 3. The method of any one of claims 1 and 2,
In the indication vector determining step, a direction for specifying whether a trigger event related to the movement of the control means occurs is determined according to the type of the indication vector
Way. 3. The method of any one of claims 1 and 2,
In the indication vector determining step, the direction for specifying whether a trigger event related to the movement of the control means occurs is determined with reference to the distance between the motion coordinates of the control means and the control target area
Way. According to claim 1,
In the control position determining step, before the position is changed, a predetermined point existing on an extension line opposite to the direction in which the pointing vector is directed is specified as the virtual reference point.
Way. 3. The method of claim 2,
In the control position determining step, a first finger vector specified based on the motion coordinates of the control means before the position is changed and a second finger vector specified based on the motion coordinates of the control means after the position is changed Correcting the indication vector based on the degree of change in position and direction between
Way. A non-transitory computer-readable recording medium storing a computer program for executing the method according to any one of claims 1 to 2. A system for supporting object control, comprising:
An indication vector determining unit that determines an indication vector by referring to at least one of whether a trigger event related to the movement of the control means occurs and a distance between a motion coordinate of the control means and a control target area, and
When the position of the control means is changed, a vector connecting the virtual reference point specified based on the extension line of the indication vector before the position is changed and the motion coordinates of the control means after the position is changed is referred to. and a control position management unit that determines a control position in the control target area.
system. A system for supporting object control, comprising:
An indication vector determining unit that determines an indication vector by referring to at least one of whether a trigger event related to the movement of the control means occurs and a distance between a motion coordinate of the control means and a control target area, and
When the position of the control means is changed, a first guide vector specified with reference to the motion coordinates of the control means before the position is changed and a second guide vector specified with reference to the motion coordinates of the control means after the position is changed A control position management unit for determining a control position in the control target region with reference to the instruction vector corrected based on a relationship between guide vectors
system.

Images