WO2022257870A1 - Virtual scale display method and related device - Google Patents

Abstract

The present application provides a virtual scale display method, comprising: detecting a contact gesture with a touch screen, indicating contact between the hand side surface of a user and the touch screen on the basis of the contact gesture, and displaying a virtual scale on the touch screen. According to the present application, when it is detected that the contact gesture indicates the contact between the hand side surface of the user and the touch screen, the display of the virtual scale can be triggered, on one hand, only one hand of the user is needed for operation, so that the operation difficulty and the cost are very low; and on the other hand, when the hand of the user is unfolded, a contact region between the hand side surface and the touch screen is a flat elongated region and is approximate to a straight line, and the display of the virtual scale triggered on the basis of the operation mode better conforms to the operation habit of the user.

Description

A virtual ruler display method and related equipment

This application claims the priority of the Chinese patent application with the application number 202110644928.3 and the title of the invention "a virtual scale display method and related equipment" submitted to the China Patent Office on June 9, 2021, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the field of terminal software, in particular to a method for displaying a virtual scale and related equipment.

Background technique

With the rapid development of Internet and communication technology, office informatization has increasingly become an important part of the entire IT service system. Among them, interactive touch display devices (such as interactive electronic whiteboards) that are closest to users are popular and have become standard accessories for digital conferences. As shown in FIG. 1 , in order to achieve a better writing experience, an interactive touch display device is generally equipped with a stylus to facilitate users to write on the screen.

Electronic whiteboards are generally suitable for multi-person discussion scenarios and play a role similar to blackboards. In conference rooms or teaching scenarios, sometimes there is a need for drawing, table drawing, and pattern measurement, and a virtual ruler needs to be called out through specific operations. The existing gestures for triggering and calling the virtual ruler are relatively complicated, and the operation is difficult and costly, and does not conform to the user's operating habits.

Contents of the invention

The present application detects a contact gesture with the touch screen, and based on the contact gesture indicates the contact of the side of the user's hand with the touch screen, and displays a virtual scale on the touch screen according to the contact area of the contact gesture. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is unfolded, the contact area between the side of the hand and the touch screen is a flat strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operation habits.

An embodiment of the present application provides a method for displaying a virtual ruler, which is applied to an electronic device, the electronic device includes a touch screen, and the method includes:

detecting a contact gesture with the touch screen;

Wherein, the touch screen may include a touch-sensitive surface, a sensor or a sensor group that receives input from the user based on tactile contact, and "detecting a contact gesture with the touch screen" may be understood as detecting a contact gesture with the touch-sensitive surface on the touch screen ;

Based on the contact gesture indicating that the side of the user's hand is in contact with the touch screen, displaying a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is a strip-shaped area, The virtual ruler is attached to the long side of the strip-shaped area.

Among them, "contact area" can be understood as a detectable area in contact with the touch-sensitive surface, which can be a continuous area or a discrete area composed of dense touch points (for example, a contact area can be a group of densely distributed The long strip area formed by the contact points);

Wherein, the virtual scale is attached to the long side of the strip-shaped area, that is to say, in the display position, the virtual scale and the strip-shaped area need to be very close, and in the direction, the virtual scale and the strip-shaped area need to be very close. The long sides of the shaped region are parallel or close to parallel;

In order to reduce the operational difficulty and cost for the user to trigger the virtual ruler display function, in the embodiment of the present application, when a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual ruler is triggered. On the one hand, only the user needs to On the other hand, when the user's hand is extended, the contact area between the side of the hand and the touch screen is a flat strip area, which is close to a straight line. Based on this operation mode Triggering the display of the virtual ruler is more in line with the user's operating habits.

In a possible implementation, the side of the hand is the side of the user's hand that is located on the side of the little finger in an unfolded state.

The side of the hand in the embodiment of the present application can be understood as the side of the user's hand located on the side of the little finger in the unfolded state (for example, including at least one of the surface of the hypothenar part of the user's hand or the surface of the little finger).

Wherein, the unfolded state of the hand may be a state where the user's fingers (or only the little finger) are on the same plane (or close to the same plane) as the palm.

In a possible implementation, the contact area includes: a contact area between a hypothenar part on the side of the user's hand and the touch screen; or, a contact area between a little finger on the side of the user's hand and the touch screen.

The side of the hand may include the area of the hypothenar area and the area of the little finger. When the side of the user's hand is in contact with the touch screen, at least one of the following contact situations may occur due to the posture of the hand or the characteristics of the user's hand shape:

The area of the hypothenar is in contact with the touch screen;

The area of the little finger is in contact with the touch screen;

The area of the hypothenar and the area of the little finger together make contact with the touch screen.

In a possible implementation, the method further includes: based on the contact area of the contact gesture meeting a preset condition, determining that the contact gesture indicates that the side of the user's hand is in contact with the touch screen; the preset condition includes : The contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar part and/or the little finger part of the side of the user's hand is in contact with the touch screen.

Among them, the shape can be understood as the outline feature of the boundary line of the contact area. When the outline shape of the boundary line of the contact area is flat and long, it can be considered that the shape of the contact area conforms to the hypothenar position and/or little finger on the side of the user's hand. The shape characteristics of the contact area when the part is in contact with the touch screen;

Wherein, the shape can also be understood as the distribution characteristics of the contact points included in the contact area, and whether the above shape characteristics are satisfied can be judged by detecting the distribution shape of the touch points. (For example, if a group of densely distributed elongated touch points are detected, it can be considered that the shape of the contact area conforms to the shape characteristics of the contact area when the hypothenar position on the side of the user's hand and/or the little finger position is in contact with the touch screen) ;

Wherein, when the area size of the contact area is within the preset range (the maximum value in this range cannot be too large, and the minimum value in this range cannot be too small, it can be set based on the characteristics of the side of the hand of the human hand. ), then it can be considered that the size of the contact area conforms to the characteristics of the area size of the contact area when the hypothenar part and/or little finger part of the side of the user's hand is in contact with the touch screen.

In a possible implementation, the method also includes:

acquiring gesture data of the contact gesture;

According to the gesture data, through a neural network, it is determined that the contact gesture indicates that the side of the user's hand is in contact with the touch screen.

In a possible implementation, the virtual ruler is attached to the long side of the strip-shaped region, including:

The acute angle between the direction of the virtual ruler and the direction indicated by the long side of the strip-shaped area is smaller than a preset value, and the preset value can be 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 10 degrees, etc.; and the virtual ruler and the strip-shaped area meet one of the following conditions:

There is overlap between the virtual scale and the strip-shaped area; or, the virtual scale is tangent to the strip-shaped area; or, the distance between the virtual scale and the strip-shaped area is less than a preset The preset value can be 1cm, 2cm, 3cm, 4cm, 5cm, 10cm, 15cm, etc.

In a possible implementation, before displaying the virtual ruler on the touch screen according to the contact area of the contact gesture, the method further includes: detecting that the contact gesture is maintained for a time longer than a preset time.

Wherein, the maintenance time of the contact gesture can start counting when the contact gesture is detected, or after detecting that the contact intensity between the contact gesture and the touch screen is greater than a certain contact intensity threshold (threshold greater than 0), the maintenance time can be The timing ends when it is detected that the contact intensity of the contact gesture is 0, or the timing ends when it is detected that the contact intensity of the contact gesture is less than a certain contact intensity threshold (threshold greater than 0).

Wherein, the maintenance time of the contact gesture may be the maintenance time when the contact gesture remains in a static state (or the movement is less than a certain range), for example, the maintenance time of the contact gesture may be the maintenance time when the pressing gesture is at rest (or the movement is less than a certain range) . It should be understood that the above static state may be understood as a static state of the contact area of the touch gesture;

In a possible implementation, the method further includes: detecting movement of a contact area of the contact gesture; and adjusting a display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

Wherein, the so-called movement can be understood as a change in position and/or a change in direction;

Among them, the so-called following can be understood as always fitting the display position of the virtual ruler with the contact area of the touch gesture. Since the contact area of the touch gesture moves, the display position and direction of the virtual ruler will also change, and in the virtual The display area of the ruler is always in close contact with the long side of the contact area.

In a possible implementation, the method further includes: detecting a hand-raising gesture of the user; in response to the hand-raising gesture, hiding the display of the virtual ruler on the touch screen.

Wherein, the gesture of raising the hand can be understood as the side of the hand leaving the touch screen, or the contact intensity between the side of the hand and the touch screen is less than a certain threshold, or the contact area between the side of the hand and the touch screen is smaller than a preset value.

In a possible implementation, the instructing that the side of the user's hand contacts the touch screen based on the contact gesture includes: instructing that the side of the user's hand and the touch screen perform a preset number of taps based on the contact gesture After the virtual ruler is displayed on the touch screen, the method further includes: detecting a hand-raising gesture of the user; in response to the hand-raising gesture, maintaining the display of the virtual ruler on the touch screen.

In some scenarios, the virtual ruler still needs to be used after the user raises his hand, so it is necessary to trigger the fixed display of the virtual ruler on the touch screen based on a certain gesture;

Optionally, the preset number of taps needs to occur within a preset time, and the preset time is a short time (for example, 0.1S, 0.2S, 0.3S, 0.4S, 0.5S, etc.);

Wherein, the tapping positions of the preset number of taps on the touch screen need to be kept consistent, or slight deviations are allowed.

Wherein, the preset number of times can be 2 times, 3 times, etc. For example, when the side of the hand is tapped twice continuously, the virtual ruler can be fixedly displayed on the touch screen.

In a possible implementation, the method further includes: detecting a user's selection instruction for the virtual scale; in response to the selection instruction, displaying a trigger control, where the trigger control is used to indicate the selection of the virtual scale At least one of the following operations is performed: a deletion operation, a position adjustment operation, and a rotation operation.

For example, the user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the virtual ruler is selected, the touch screen will pop up a trigger control. For example, the trigger control can be a prompt for indicating the deletion operation of the virtual ruler. Click the "Delete" prompt or drag the virtual ruler to the trash icon) to delete the virtual ruler.

For example, the user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the virtual ruler is selected, the touch screen will pop up a trigger control. For example, the trigger control can be a prompt for indicating the position adjustment operation for the virtual ruler. The user can By clicking the "posture adjustment" control, you can enter the posture adjustment mode for the virtual ruler. For example, in this mode, the user can adjust the display posture of the virtual ruler by touching the side of the hand and the touch screen. The adjustment method can refer to the above The relevant descriptions about the control of the display position of the virtual ruler in the embodiment will not be repeated here. For another example, if you need to rotate the virtual ruler, you can use two fingers (on the virtual ruler or within the preset distance of the virtual ruler) to place two Click at different positions, and the two touch points rotate clockwise or counterclockwise at the same time to turn the virtual ruler. Optionally, when operating the virtual scale to rotate, the included angle between the virtual scale before and after the rotation can be displayed in real time.

In the embodiment of the present application, by rotating or moving the side of the hand to give corresponding rotation or movement operations to the virtual ruler, tapping and fixed display of the virtual ruler and other gestures are in line with the user's operating habits and can greatly improve the operating efficiency.

In a possible implementation, the virtual ruler is used to measure the length of the drawing line segment displayed on the touch screen; or, the virtual ruler is used to measure the length of the drawing points displayed on the touch screen distance measurement.

In a possible implementation, the method further includes: detecting that there is a drawing line segment associated with the virtual ruler in a gesture on the touch screen, and displaying the length value of the drawing line segment; the existing gesture association includes : the direction difference from the virtual ruler is smaller than a preset value; and/or, the distance value from the virtual ruler is smaller than a preset value.

Among them, the so-called posture association can be understood as a high degree of fit between the virtual ruler and the drawing straight line segment. For example, the posture association can be expressed by the direction difference and the distance value. When the direction difference between the drawing straight line segment and the virtual ruler is less than the preset value ( Such as less than 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees, 45 degrees, etc.); and/or, the distance between the drawn straight line segment and the virtual ruler is less than a preset value (such as 1cm, 2cm, 3cm etc.), it can be considered that there is a drawing line segment associated with the gesture of the virtual ruler on the touch screen.

Among them, the distance can be understood as the distance between the nearest point on the virtual ruler and the drawing straight line segment, or the average value of the distance between the point on the virtual ruler and the drawing straight line segment, or can be used to characterize the virtual ruler and drawing The method for the distance between straight line segments.

It should be understood that when the detection module detects that there are multiple drawing straight line segments associated with the virtual ruler in gestures on the touch screen, the length value of the drawing straight line segment with the highest degree of gesture-association may be acquired and displayed.

In a possible implementation, the method further includes: adjusting the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler fits with the drawn straight line. That is to say, a display effect similar to magnetic attraction is realized. Wherein, adjusting the display position of the virtual ruler may include adjusting the position and display direction of the virtual ruler.

In a possible implementation, after the virtual ruler is displayed on the touch screen, the method further includes: detecting that there is a first intersection point and a second intersection point between the drawn line segment on the touch screen and the virtual ruler, and displaying A distance value between the first intersection point and the second intersection point.

In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

In a possible implementation, the method further includes: detecting a line-drawing gesture on the touch screen; based on the fact that the virtual ruler is displayed on the touch screen, and the position of the line-drawing gesture of the The distance between the virtual rulers is within a preset distance (for example, 1cm, 2cm, 3cm, 4cm, 5cm, etc.), according to the position of the line drawn by the gesture of drawing a line, a straight line segment for drawing is displayed, wherein the straight line segment for drawing parallel to the virtual ruler.

On the one hand, when writing within a certain range of the virtual ruler (without raising your hand), you can restrict the range of writing and drawing not to cross the virtual ruler, that is, you can only write on one side of the virtual ruler. When the drawing range exceeds the virtual ruler , displaying the straight line that fits with the virtual ruler (the side that can be used for writing and drawing), which is equivalent to that the virtual ruler has a function similar to that of a ruler in physical space.

On the other hand, in response to the drawing and drawing behavior within the predefined range of the virtual ruler, the drawing trajectory of the drawing and painting can be corrected to a straight line. When the user draws a line along the virtual ruler in the above way, a straight line will be drawn automatically.

In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes: displaying a line drawing length based on the real-time line drawing position, where the line drawing length is the The distance value between the starting point position and the real-time drawing line position.

And in order to get a line of expected length, when drawing along the ruler, the drawing line between the first and last two drawing points can be calculated and displayed, and the length of the drawing line can be displayed. In a possible implementation, the line drawing position includes a starting point position and an ending point position, the drawing straight line segment is a line segment between the starting point position and the ending point position, and the detection module may be based on the The real-time line drawing position displays the line drawing length, and the line drawing length is the distance value between the starting point position and the real-time line drawing position.

That is to say, if the length of the line segment drawn exceeds the expected value, you can draw it back, which will have a deletion effect. As long as the distance between the end point and the start point is the expected value, the generated line segment is the expected value, and the drawn Curves (the distance between the curve and the virtual ruler is within the preset range) will be automatically corrected to a straight line.

In the embodiment of this application, the two functions of the ruler: measuring and drawing a straight line are converted into intelligent operations suitable for the virtual interface. The measurement is to automatically display the measured value, and the length value is displayed when drawing a straight line, which removes the limitations of the physical world and saves It takes less time for users to take readings, greatly improving user efficiency.

In a second aspect, the present application provides an object copy method, the method comprising: detecting a drag gesture for a target object displayed on a touch screen; in response to the drag gesture, displaying the target object on the touch screen The mirror image of the target object, and according to the drag gesture, update the display position of the mirror image in real time, so that the mirror image moves with the drag gesture; detect the hand-raising gesture; respond to the hand-raising gesture, move the The mirror image is fixedly displayed at the display position where the mirror image is located.

The method of copying and pasting by dragging gestures is simple and intuitive, which reduces the number of steps required for existing copy and paste, and shortens the process of copying and pasting.

In a possible implementation, before the detection of the drag gesture for the target object displayed on the touch screen, the method further includes:

When at least one of the following gestures is detected, the copy function for the target object is enabled:

detecting a long press gesture directed at the target object; or,

A click gesture is detected for the target object.

In a possible implementation, the detecting the long press gesture for the target object includes:

A long press gesture is detected in which the contact area with the touch screen covers the target object or is within a preset distance around the target object.

In a possible implementation, the dragging gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

In a possible implementation, the long press gesture is a two-finger long press gesture.

In a possible implementation, the detecting the click gesture for the target object includes:

A click gesture in which a contact area with the touch screen covers the target object is detected.

In the third aspect, the embodiment of the present application provides a virtual ruler display device, which is applied to electronic equipment, and the electronic equipment includes a touch screen, and the device includes:

A detection module, configured to detect a contact gesture with the touch screen;

A display module, configured to indicate the contact of the side of the user's hand with the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is In the strip-shaped area, the virtual ruler is attached to the long side of the strip-shaped area.

In a possible implementation, the side of the hand is the side of the user's hand that is located on the side of the little finger in an unfolded state.

In a possible implementation, the contact area includes:

the contact area of the hypothenar portion of the side of the user's hand with the touch screen; or,

The contact area between the little finger on the side of the user's hand and the touch screen.

In a possible implementation, the device also includes:

A determining module, configured to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on that the contact area of the contact gesture satisfies a preset condition; the preset condition includes:

The contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar part and/or the little finger part of the side of the user's hand is in contact with the touch screen.

In a possible implementation, the device also includes:

An acquisition module, configured to acquire gesture data of the contact gesture;

The determining module is further configured to determine, according to the gesture data, through a neural network, that the contact gesture indicates that the side of the user's hand is in contact with the touch screen.

In a possible implementation, the virtual ruler is attached to the long side of the strip-shaped region, including:

The acute angle between the direction of the virtual scale and the direction indicated by the long side of the strip-shaped area is smaller than a preset value; and the virtual scale and the strip-shaped area meet one of the following conditions:

There is overlap between the virtual scale and the strip-shaped area; or, the virtual scale is tangent to the strip-shaped area; or, the distance between the virtual scale and the strip-shaped area is less than a preset value.

In a possible implementation, the detection module is configured to detect that the duration of the contact gesture is greater than a preset time before displaying a virtual ruler on the touch screen in the contact area according to the contact gesture .

In a possible implementation, the detection module is configured to detect movement of a contact area of the contact gesture;

The display module is configured to adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

In a possible implementation, the detection module is configured to detect a user's hand-raising gesture;

The display module is configured to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

In a possible implementation, the instructing the contact of the side of the user's hand with the touch screen based on the contact gesture includes:

Instructing the side of the user's hand to tap the touch screen for a preset number of times based on the contact gesture;

After the virtual ruler is displayed on the touch screen, the method further includes:

The user's hand gesture is detected;

In response to the raising hand gesture, maintaining display of the virtual ruler on the touch screen.

In a possible implementation, the detection module is configured to detect a user's selection instruction for the virtual scale;

The display module is configured to display a trigger control in response to the selection instruction, and the trigger control is used to indicate at least one of the following operations on the virtual scale:

Delete operations, reposition operations, and rotation operations.

In a possible implementation, the virtual ruler is used to measure the length of a drawn straight line segment displayed on the touch screen; or,

The virtual ruler is used for distance measurement between drawing points displayed on the touch screen.

In a possible implementation, the detection module is configured to detect that there is a drawing line segment associated with the gesture of the virtual ruler on the touch screen;

The display module is used to display the length value of the straight line segment of the drawing; the existence gesture association includes:

The direction difference from the virtual scale is smaller than a preset value; and/or,

The distance from the virtual ruler is smaller than a preset value.

In a possible implementation, the display module is configured to adjust the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler fits with the drawn straight line.

In a possible implementation, the detection module is configured to detect that there is a first intersection point and a second intersection point between the drawn line segment on the touch screen and the virtual scale after the virtual scale is displayed on the touch screen;

The display module is configured to display a distance value between the first intersection point and the second intersection point.

In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

In a possible implementation, the detection module is configured to detect a line-drawing gesture on the touch screen;

The display module is configured to display the virtual ruler on the touch screen, and the distance between the line-drawing position of the line-drawing gesture and the virtual ruler is within a preset distance, according to the line-drawing gesture The position of the drawn line, displaying the drawn straight line segment, wherein the drawn straight line segment is parallel to the virtual ruler.

In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes:

Based on the real-time line-drawing position, display the line-drawing length, where the line-drawing length is a distance value between the starting point position and the real-time line-drawing position.

In a fourth aspect, the present application provides an object replication device, the device comprising:

A detection module, configured to detect a drag gesture for a target object displayed on the touch screen; the detection module is also configured to detect a hand-raising gesture after the display module displays a mirror image of the target object;

A display module, configured to display a mirror image of the target object on the touch screen in response to the drag gesture, and update the display position of the mirror image in real time according to the drag gesture, so that the mirror image follows the The drag gesture moves; the display module is further configured to, in response to the hand-raising gesture, fix and display the mirror image at the display position where the mirror image is located.

In a possible implementation, the device also includes:

An enabling module, configured to enable the copy function for the target object when at least one of the following gestures is detected before the detection of the drag gesture for the target object displayed on the touch screen:

detecting a long press gesture directed at the target object; or,

A click gesture is detected for the target object.

In a possible implementation, the detecting the long press gesture for the target object includes:

A long press gesture is detected in which the contact area with the touch screen covers the target object or is within a preset distance around the target object.

In a possible implementation, the dragging gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

In a possible implementation, the long press gesture is a two-finger long press gesture.

In a possible implementation, the detecting the click gesture for the target object includes:

A click gesture in which a contact area with the touch screen covers the target object is detected.

In a fifth aspect, the present application provides an electronic device, including: a processor, a memory, a touch screen, and a bus, wherein: the processor, the memory, and the touch screen are connected through the bus;

The memory is used to store computer programs or instructions;

The processor is used to call or execute the programs or instructions stored in the memory, and is also used to control the touch screen, so as to implement the steps described in the first aspect and any possible implementation manner of the first aspect, and Any one of the second aspect and the steps described in the possible implementation manners of the second aspect.

In a sixth aspect, the present application provides a computer storage medium, including computer instructions. When the computer instructions are run on an electronic device or a server, the steps described in any one of the above-mentioned first aspect and possible implementation modes of the first aspect are executed. , and the second aspect and the steps described in any one possible implementation manner of the second aspect.

In a seventh aspect, the present application provides a computer program product. When the computer program product is run on an electronic device or a server, it executes the steps described in any one of the above-mentioned first aspect and possible implementation modes of the first aspect, and the first aspect. The steps described in the possible implementation manners of any one of the second aspect and the second aspect.

In an eighth aspect, the present application provides a chip system, which includes a processor, configured to support an execution device or a training device to implement the functions involved in the above aspect, for example, send or process the data involved in the above method; or, information. In a possible design, the chip system further includes a memory, and the memory is used for storing necessary program instructions and data of the execution device or the training device. The system-on-a-chip may consist of chips, or may include chips and other discrete devices.

An embodiment of the present application provides a method for displaying a virtual scale, including: detecting a contact gesture with the touch screen; indicating the contact of the user's hand side with the touch screen based on the contact gesture, and according to the contact area of the contact gesture, A virtual scale is displayed on the touch screen, wherein the contact area of the contact gesture is a strip-shaped area, and the virtual scale is attached to a long side of the strip-shaped area. When a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual scale is triggered. On the one hand, only the user's single hand is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand When the upper part is unfolded, the contact area between the side of the hand and the touch screen is a flat strip area, which is close to a straight line. Based on this operation mode, triggering the display of the virtual scale is more in line with the user's operation habits.

Description of drawings

Fig. 1 is the product structure schematic diagram that the embodiment of the present application provides;

FIG. 2 is a structural block diagram of an electronic device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an embodiment of a method for displaying a virtual ruler provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a gesture in the embodiment of the present application;

Fig. 5 is a schematic diagram of a gesture in the embodiment of the present application;

Fig. 6 is a schematic diagram of a gesture in the embodiment of the present application;

FIG. 7 is a schematic diagram of a gesture in the embodiment of the present application;

Fig. 8 is a schematic diagram of a gesture in the embodiment of the present application;

FIG. 9 is a schematic diagram of a gesture contact in the embodiment of the present application;

FIG. 10 is a schematic diagram of a gesture contact in the embodiment of the present application;

FIG. 11 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 12 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 13 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 14 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 15 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 16 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 17 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 18 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 19 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 20 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 21 is a schematic diagram of a terminal interface in an embodiment of the present application;

Fig. 22a is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 22b is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 22c is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 22d is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 23 is a schematic diagram of establishing an index table in the embodiment of the present application;

FIG. 24 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 25 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 26 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 27 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 28 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 29 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 30 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 31 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 32 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 33 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 34 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 35 is a schematic diagram of a terminal interface in the embodiment of the present application;

FIG. 36 is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 37a is a schematic diagram of an embodiment of an object copy method provided by the embodiment of the present application;

Fig. 37b is a schematic diagram of a terminal interface in the embodiment of the present application;

FIG. 38 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 39 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 40 is a schematic diagram of a terminal interface in an embodiment of the present application;

FIG. 41 is a schematic diagram of a terminal interface in an embodiment of the present application;

Fig. 42 is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 43 is a schematic diagram of a terminal interface in the embodiment of the present application;

FIG. 44 is a schematic diagram of a terminal interface in an embodiment of the present application;

Fig. 45 is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 46 is a schematic diagram of a terminal interface in the embodiment of the present application;

Fig. 47 is a schematic diagram of a terminal interface in the embodiment of the present application;

FIG. 48 is a schematic diagram of a terminal interface in an embodiment of the present application;

Fig. 49 is a schematic structural diagram of a virtual ruler display device provided by an embodiment of the present application;

FIG. 50 is a schematic structural diagram of an object copying device provided by an embodiment of the present application;

FIG. 51 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

Detailed ways

Embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention. The terms used in the embodiments of the present invention are only used to explain specific examples of the present invention, and are not intended to limit the present invention.

Embodiments of the present application are described below in conjunction with the accompanying drawings. Those of ordinary skill in the art know that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is merely a description of the manner in which objects with the same attribute are described in the embodiments of the present application. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus comprising a series of elements is not necessarily limited to those elements, but may include elements not expressly included. Other elements listed explicitly or inherent to the process, method, product, or apparatus.

First introduce the exemplary operating environment of this application

The embodiment of the present application may be applied in the system 100 including the touch screen 103 .

Referring to FIG. 1 , FIG. 1 shows a system 100 to which the embodiment of the present application is applied, wherein the system 100 may include an electronic device 101 and a pen 102 associated with the electronic device 101 .

The electronic device 101 may be an electronic whiteboard (or called an electronic interactive smart board) shown in FIG. 1 , and the electronic device 101 includes a touch screen 103 . It should be understood that the electronic device 101 may also be a portable mobile device including a touch screen, such as but not limited to a mobile or portable computing device (such as a smart phone), a personal computer, a server computer, a handheld device (such as a tablet), or a laptop. devices, multiprocessor systems, game consoles or controllers, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, wearable or accessory form factors (e.g., watches, glasses, headsets) or earbuds), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

The touch screen 103 can recognize a user's contact gesture.

In one implementation, the touch screen 103 can be an infrared touch screen (infrared touch screen). The infrared touch screen is composed of infrared emitting and receiving sensing elements mounted on the outer frame of the touch screen. On the surface of the screen, an infrared detection network is formed. Any touching object The touch screen operation can be realized by changing the infrared rays on the contacts. The realization principle of the infrared touch screen is similar to that of the surface acoustic wave touch screen, which uses infrared emitting and receiving sensing elements. These elements form an infrared detection network on the surface of the screen, and the touch-operated object (such as the contact between the side of the hand and the touch screen 103 in the embodiment of the present application) can change the infrared rays of the touch point, and then be converted into the coordinate position of the touch, thereby realizing Recognition of touch gestures.

In one implementation, the touch screen 103 may be a capacitive touch screen, and the capacitive touch screen works by utilizing the current induction of the human body. Exemplarily, the capacitive touch screen can be a four-layer composite glass screen, the inner surface and the interlayer of the glass screen are each coated with a layer of ITO (nano indium tin oxide), and the outermost layer is a thin layer of silica glass protective layer, The interlayer ITO coating is used as the working surface, four electrodes are drawn from the four corners, and the inner ITO layer is the shielding layer to ensure a good working environment. When the user's hand touches the metal layer (such as the contact between the side of the hand and the touch screen 103 in the embodiment of the present application), due to the electric field of the human body, a coupling capacitance is formed between the user and the surface of the touch screen. For high-frequency currents, the capacitance is direct conductor, so the finger draws a small current from the point of contact. This current flows out of the electrodes on the four corners of the touch screen respectively, and the current flowing through these four electrodes is proportional to the distance from the finger to the four corners. The controller obtains the position of the touch point through accurate calculation of the four current ratios. , and then the recognition of touch gestures can be realized.

It should be understood that the touch screen 103 can also be other types of touch screens capable of recognizing gestures of contact, or be replaced with a touch screen that only has a display function but can cooperate with other external devices (such as sensors) to recognize gestures of contact. limited.

In one example, the pen 102 may also provide input to the electronic device 101 by contacting or otherwise interacting with the touch screen 103 .

The touch screen 103 can display patterns and characters, and can also provide a drawing interface for users to freely draw and draw, such as a whiteboard interface provided by an electronic whiteboard, a drawing board provided by an application (APP), and the like.

In the embodiment of the present application, the user can touch the touch screen 103 through the side of the hand, and the electronic device 101 can trigger the display of the virtual scale after detecting the user's contact gesture.

The exemplary operating environment of the present application is introduced above, and the internal structure of the electronic device 101 is described below with reference to an example.

Referring to FIG. 2 , FIG. 2 is a schematic structural diagram of an electronic device 101 provided by an embodiment of the present application.

It can be understood that the structure shown in FIG. 2 does not constitute a specific limitation on the electronic device 101 . In other embodiments of the present application, the electronic device 101 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The electronic device 101 can include an input/output controller 218 that can output information to one or more output devices 222 (eg, a touch screen or a speaker) that are separate or integrated with the electronic device 101 . The input/output controller 218 may also be used to receive input from one or more input devices 220 (eg, a keyboard, a microphone, or a touch screen). In one embodiment, output device 222 may also serve as input device 220 . An example of such a device would be a touch screen. In some embodiments, a user may provide input to input device 220 and/or receive output from output device 222 .

In the embodiment of the present application, the input device 220 may be a touch screen, and the user provides gesture input to the input/output controller 218 by touching the side of the hand with the touch screen, and the input/output controller 218 may transmit the gesture input to the processor, and the processor 204 for processing.

The electronic device 101 may include one or more processors 204, and these processors may include one or more processing units, for example: the processor 204 may include an application processor (application processor, AP), a modem processor, a graphics processing unit Graphics processing unit (GPU), image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network Processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller of the processor 204 can generate an operation control signal according to the instruction operation code and the timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 204 for storing instructions and data. In some embodiments, the memory in processor 204 is a cache memory. The memory may hold instructions or data that the processor 204 has just used or recycled. If the processor 204 needs to use the instruction or data again, it can be directly recalled from the memory. Repeated access is avoided, and the waiting time of the processor 204 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 204 may include one or more interfaces. For example, the interface may include, but not limited to, a mobile industry processor interface (mobile industry processor interface, MIPI), an external memory interface, and/or a universal serial bus (universal serial bus, USB) interface and the like.

The MIPI interface can be used to connect the processor 204 with peripheral devices such as a touch screen. The MIPI interface may include a display serial interface (display serial interface, DSI) and the like. The processor 204 communicates with the touch screen through the DSI interface to realize the display function of the touch screen.

It can be understood that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration, and does not constitute a structural limitation of the electronic device 101 . In other embodiments of the present application, the electronic device 101 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The electronic device 101 may implement a display function through a GPU, a touch screen, an application processor, and the like. The GPU is a microprocessor for image processing, which connects the touch screen and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 204 may include one or more GPUs that execute program instructions to generate or alter display information. Specifically, one or more GPUs in the processor 204 can implement image rendering tasks (such as rendering tasks related to drawing virtual rulers, distance values, length values, etc. in this application, and deliver the rendering results to the application processor or Other display drivers, the application processor or other display drivers trigger the display to display virtual rulers, distance values, length values, etc.).

A touch screen may include a display screen and associated sensors (eg, pressure sensors and touch sensors).

Displays are used to display images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc.

The pressure sensor is used to sense the pressure signal and convert the pressure signal into an electrical signal. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 101 can determine the strength of the pressure according to the change of the capacitance. When a touch operation acts on the touch screen, the electronic device 101 may detect the intensity of the touch operation according to the pressure sensor. The electronic device 101 may also calculate the touched position according to the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions.

Touch sensor, also known as "touch device". The touch sensor can be arranged on the touch screen, and the touch screen is composed of the touch sensor and the display screen. The touch sensor is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation may be provided through the display screen. In some other embodiments, the touch sensor may also be disposed on the surface of the electronic device 101, which is different from the position of the display screen.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 101 can be realized through the NPU, for example, a touch gesture recognition task can be realized based on the NPU.

The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 101. The external memory card communicates with the processor 204 through the external memory interface to realize the data storage function. For example, you can save music, video and other files in the external memory card.

Memory 214 may be used to store computer-executable program code, which includes instructions. The memory 214 may include an area for storing programs and an area for storing data. Wherein, the storage program area can store the operating system 206, the application software 208 required by at least one function (such as image playing function, etc.) and the like. The storage data area can store data (such as image data, etc.) created during the use of the electronic device 101 . In addition, the memory 214 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 204 executes various functional applications and data processing of the electronic device 101 by executing instructions stored in the memory 214 and/or instructions stored in the memory provided in the processor.

Wherein, the method described in the embodiment of the present application may be a code stored in the memory 214 (or an external memory), and the processor 110 may acquire the code in the memory to implement the method provided in the embodiment of the present application.

The electronic device 101 can also interact with other electronic devices through the communication device 216 .

It should be understood that alternatively or additionally, the methods described herein may be performed at least in part by one or more hardware logic components. By way of example and not limitation, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts, ASSP), system on chip (system on chip, SOC) and programmable logic device (programable logic device, PLD), etc. For example, functions such as detection of contact gestures and determination of gesture categories can be implemented based on hardware logic components.

Furthermore, at least some of the functionality of the various elements in the figures may be performed by other elements in the figures or by entities not shown in the figures (eg, processors, web services, servers, applications, computing devices, etc.) Distributed execution is not limited here.

For ease of understanding, the method for displaying a virtual ruler provided in the embodiment of the present application is specifically described in conjunction with the drawings and application scenarios.

Referring to Fig. 3, Fig. 3 is a schematic flowchart of a virtual scale display method provided by the embodiment of the present application. As shown in Fig. 3, the virtual scale display method provided by the embodiment of the present application includes:

301. Detect a contact gesture with the touch screen.

In product usage scenarios such as meeting rooms or teaching, sometimes there is a need for drawing, table drawing, and pattern measurement, so it is necessary to use specific gesture operations (such as the touch gesture in the embodiment of this application) to call out virtual gestures on the touch screen. ruler.

In this embodiment of the present application, the electronic device 101 may detect a contact gesture with the touch screen.

Wherein, the touch screen may include a touch-sensitive surface, a sensor or a sensor group that receives input from the user based on tactile contact, and "detecting a contact gesture with the touch screen" may be understood as detecting a contact gesture with the touch-sensitive surface on the touch screen .

Firstly, the devices and modules for detecting contact gestures related to the touch screen are described.

In a possible implementation, the touch screen can cooperate with the detection module to detect a contact gesture on the touch screen. Wherein, the detection module can be a program, a data structure or a subset thereof stored in the memory related to the detection of the contact gesture, or a part exists in the memory in the form of a program, a data structure or a subset thereof, and a part is in the form of a hardware logic module . The touch screen can capture contact data, and the detection module can perform various actions related to contact gesture detection according to the contact data.

In a possible implementation, the touch screen can capture the contact data (such as electrical signals) between the user and the touch-sensitive surface in real time, and transmit the contact data to the detection module. various actions.

Next, the detection module is described to realize the detection of the contact gesture on the touch screen.

In one implementation, the detection module can determine the intensity and/or its change when the user is in contact with the touch screen based on the contact data, as well as the size and/or its change of the contact area, and then determine the gesture of the contact gesture based on the above information type.

Herein, "strength" can be understood as the force or pressure (force per unit area) of a contact (eg, hand side contact) on the touch-sensitive surface of the touch screen. The intensity of contact can be determined using various methods and various sensors or combinations of sensors. For example, below the touch-sensitive surface, the force at different points on the touch-sensitive surface on the touch screen is measured, eg, using one or more sensors adjacent to the touch-sensitive surface. In some implementations, force measurements from multiple sensors may be combined (eg, weighted averaged) to determine the strength of the contact. In addition, the size of the contact area detected on the touch-sensitive surface and/or its change, the capacitance of the touch-sensitive surface in the vicinity of the touch and/or its change and/or the resistance of the touch-sensitive surface in the vicinity of the touch and/or It changes to indirectly represent the force or pressure of a contact on a touch-sensitive surface.

Among them, "contact area" can be understood as a detectable area in contact with the touch-sensitive surface, which can be a continuous area or a discrete area composed of dense touch points (for example, a contact area can be a group of densely distributed The long strip area formed by the contact points).

The detection module can perform various actions related to the contact gesture detection based on the contact data, and then determine the gesture type of the contact gesture, the movement of the contact gesture, the stop of the contact gesture, and the like.

For example, the detection module can determine whether contact has occurred and the type of contact gesture (e.g., detect a finger press event, or a contact event from the side of the hand), determine whether there is movement of the contact, and track movement on the touch-sensitive surface (e.g., Detect one or more finger drag events, or a drag event from the side of the hand), and determine whether contact has ceased (for example, detect a finger lift event, or detect a lift event from the side of the hand).

Wherein, the above-mentioned "movement of the contact gesture" can also be referred to as the movement of the contact area of the contact gesture, and the data related to the movement of the contact area can include, but not limited to, the rate (magnitude), speed (magnitude and direction) and/or acceleration (change in magnitude and/or direction).

Furthermore, the detection module can perform various actions related to contact gesture detection based on the contact data, so as to determine the gesture type indicated by the contact gesture, wherein the gesture type is, for example but not limited to, click, double click, long press, drag, hand Sideways stationary contact with the touchscreen, sideways hand drag, sideways hand turn, sideways double tap, and more.

In a possible implementation, the detection module may be a pre-trained neural network model, and the neural network model has the ability to recognize the gesture category indicated by the contact gesture based on the contact data of the contact gesture.

302. Based on the contact gesture indicating that the side of the user's hand is in contact with the touch screen, display a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is strip-shaped area, the virtual ruler is attached to the long side of the strip-shaped area.

In the embodiment of the present application, the detection module may determine the gesture type indicated by the contact gesture. In one implementation, the detection module may determine that the contact gesture indicates that the side of the user's hand is in contact with the touch screen.

In order to reduce the operational difficulty and cost for the user to trigger the virtual ruler display function, in the embodiment of the present application, when a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual ruler is triggered. On the one hand, only the user needs to On the other hand, when the user's hand is extended, the contact area between the side of the hand and the touch screen is a flat strip area, which is close to a straight line. Based on this operation mode Triggering the display of the virtual ruler is more in line with the user's operating habits.

Next, the contact between the side of the user's hand and the touch screen will be described with reference to the accompanying drawings.

1. The side of the hand

The side of the hand in the embodiment of the present application can be understood as the side of the user's hand located on the side of the little finger in the unfolded state (for example, including at least one of the surface of the hypothenar part of the user's hand or the surface of the little finger).

Wherein, the unfolded state of the hand may be a state where the user's fingers (or only the little finger) are on the same plane (or close to the same plane) as the palm.

Referring to Fig. 4, a user's hand posture is shown in Fig. 4. In the hand posture shown in Fig. 4, the lower side of the hand is the side of the user's hand, which includes the little finger and the user's hand. The surface of the hypothenar region. Referring to Fig. 5, another user's hand posture is shown in Fig. 5. In the hand posture shown in Fig. 5, the lower right side of the hand is the user's hand side, which includes the little finger and the user's hand. The surface of the hypothenar region of the hand.

In this embodiment of the present application, the contact between the side of the user's hand and the touch screen is equivalent to the contact between the hypothenar part of the side of the user's hand and the touch screen and/or the contact of the little finger part of the side of the user's hand with the touch screen.

2. Contact between the side of the hand and the touch screen

In the embodiment of the present application, when the contact intensity and contact area of the touch screen meet certain conditions, it can be considered as the contact of the side of the user's hand with the touch screen.

Referring to Fig. 6 and Fig. 7, among them, Fig. 6 is the schematic diagram when the side of the user's hand is in contact with the touch screen from the perspective of facing the touch screen, and Fig. 7 is the schematic diagram of the side of the user's hand in contact with the touch screen from the perspective of the side view of the touch screen From the point of view of the user's behavior, based on the difference in touch intensity and contact area, the side of the user's hand can be swipe the touch screen (small contact area, low contact intensity), or lightly press the touch screen (large contact area, Medium contact strength), or heavy pressure touch screen (large contact area, high contact strength). In the embodiment of the present application, when the side of the user's hand swipe the touch screen, it can be considered that there is a false touch between the user and the touch screen, and when the side of the user's hand lightly presses the touch screen or presses the touch screen heavily, it can be considered that the user needs to trigger the display of the virtual scale.

In one implementation, when the side of the user's hand lightly presses the touchscreen or presses heavily on the touchscreen for more than a certain period of time, it may be considered that the user needs to trigger the display of the virtual scale. Specifically, referring to FIG. 8 , when the side of the user's hand lightly presses the touchscreen or presses heavily on the touchscreen for more than X seconds, it can be considered that the user needs to trigger the display of the virtual scale.

From the perspective of the detection module, it needs to be processed and analyzed based on the contact intensity of the touch gesture and the contact area of the touch gesture and the touch screen. When it meets certain conditions, it can be considered as the contact between the side of the user's hand and the touch screen.

Next, the conditions (including shape conditions and area size conditions) that need to be satisfied by the contact area are introduced:

In one implementation, the detection module may acquire the contact area of the contact gesture, and based on the contact area meeting a preset condition, determine that the contact gesture indicates that the side of the user's hand is in contact with the touch screen; the preset The conditions include: the contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar position and/or little finger on the side of the user's hand is in contact with the touch screen .

The side of the hand may include the area of the hypothenar area and the area of the little finger. When the side of the user's hand is in contact with the touch screen, at least one of the following contact situations may occur due to the posture of the hand or the characteristics of the user's hand shape:

The area of the hypothenar is in contact with the touch screen;

The area of the little finger is in contact with the touch screen;

The area of the hypothenar area and the area of the little finger together make contact with the touch screen;

Referring to Figure 10, when the area of the hypothenar part is in contact with the touch screen, the contact area is a flat strip-shaped area;

When the area of the little finger is in contact with the touch screen, the contact area is a flat strip-shaped area, and the size of the area can be smaller than the size of the contact area when the area of the hypothenar is in contact with the touch screen;

Referring to Fig. 9, when the area of the hypothenar part and the area of the little finger are in contact with the touch screen together, the contact area is two strip-shaped areas arranged in the same direction, and the long sides of the two strip-shaped areas in the same direction.

The detection module can acquire the data of the contact area, and then analyze the shape and area size of the contact area, when the shape and area size of the contact area conform to the hypothenar part and/or the little finger part of the side of the user's hand in contact with the touch screen When the touch area is characterized, it may be determined that the touch gesture indicates contact of the side of the user's hand with the touch screen.

Among them, the shape can be understood as the outline feature of the boundary line of the contact area. When the outline shape of the boundary line of the contact area is flat and long, it can be considered that the shape of the contact area conforms to the hypothenar position and/or little finger on the side of the user's hand. The shape characteristics of the contact area when the part is in contact with the touch screen;

Wherein, the shape can also be understood as the distribution characteristics of the contact points included in the contact area, and whether the above shape characteristics are satisfied can be judged by detecting the distribution shape of the touch points. (For example, if a group of densely distributed elongated touch points are detected, it can be considered that the shape of the contact area conforms to the shape characteristics of the contact area when the hypothenar position on the side of the user's hand and/or the little finger position is in contact with the touch screen) ;

Wherein, when the area size of the contact area is within the preset range (the maximum value in this range cannot be too large, and the minimum value in this range cannot be too small, it can be set based on the characteristics of the side of the hand of the human hand. ), then it can be considered that the size of the contact area conforms to the characteristics of the area size of the contact area when the hypothenar part and/or little finger part of the side of the user's hand is in contact with the touch screen.

Next, the conditions that the contact strength needs to meet are introduced:

In some scenarios, the user may accidentally touch or perform a touch gesture with the touch screen, but this contact gesture is not used to trigger the virtual ruler. It can be based on the screening of the contact intensity of the touch gesture to avoid triggering the virtual ruler in the above scenarios. show.

Specifically, when the user performs a swipe touch (a touch with a small contact intensity) on the touch screen, it can generally be considered that the user has made a false touch. When it is determined that the contact intensity of the touch gesture is too small, the display of the virtual scale may not be triggered, and When the touch gesture is a press gesture, the display of the virtual ruler may be triggered (provided that the contact area of the touch gesture satisfies the above-mentioned shape and area size conditions).

Wherein, the press gesture may include a light press and a deep press, and when the contact intensity increases from an intensity lower than the light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold, it may be called a touch surface detected "light press" input on the An increase in contact intensity from an intensity below the deep press intensity threshold to an intensity above the deep press intensity threshold may be referred to as detecting a "deep press" input on the touch surface. An increase in contact intensity from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold may be referred to as detecting a "swipe touch" input on the touch surface. When the contact intensity decreases from an intensity higher than the contact detection intensity threshold to an intensity lower than the contact detection intensity threshold, it may be referred to as detecting that the contact is lifted from the touch surface (ie, a hand-raising gesture). In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold may be greater than zero.

In a possible implementation, the detection module can be a pre-trained neural network model, and the neural network model has the ability to identify the gesture category indicated by the contact gesture based on the contact data of the contact gesture. In the embodiment of the present application, the detection module Gesture data of the contact gesture may be acquired, and according to the gesture data, it is determined through a neural network that the contact gesture indicates that the side of the user's hand is in contact with the touch screen.

In a possible implementation, besides when the user swipes the touch screen, it will be considered as a false touch, and if the duration of the contact gesture is too short, it can also be considered as a false touch. In this case, the display of the virtual ruler may not be triggered.

In the embodiment of the present application, the maintenance time of the contact gesture can also be used as the condition for triggering the display of the virtual scale, that is, it needs to be detected that the maintenance time of the contact gesture is greater than the preset time, for example, the preset time can be set to 0.1S, 0.15 S, 0.2S, 0.25S, 0.3S, 0.35S, 0.4S, 0.45S, 1S, 2S.

Wherein, the maintenance time of the contact gesture can start counting when the contact gesture is detected, or after detecting that the contact intensity between the contact gesture and the touch screen is greater than a certain contact intensity threshold (threshold greater than 0), the maintenance time can be The timing ends when it is detected that the contact intensity of the contact gesture is 0, or the timing ends when it is detected that the contact intensity of the contact gesture is less than a certain contact intensity threshold (threshold greater than 0).

Wherein, the maintenance time of the contact gesture may be the maintenance time when the contact gesture remains in a static state (or the movement is less than a certain range), for example, the maintenance time of the contact gesture may be the maintenance time when the pressing gesture is at rest (or the movement is less than a certain range) . It should be understood that the above static state may be understood as a static state of the contact area of the touch gesture.

In this embodiment of the present application, if it is determined that the contact gesture indicates that the side of the user's hand is in contact with the touch screen, a virtual ruler may be displayed on the touch screen according to the contact area of the contact gesture.

The virtual ruler is described next:

The virtual ruler is an affordance displayed on the touch screen. The display shape of the virtual ruler is similar to a straightedge (for example, it can be a flat rectangle or a line segment). To measure the length of the straight line segment of the drawing displayed on the touch screen, or to measure the distance between the drawing points displayed on the touch screen, the user can also use the virtual ruler as a reference tool when drawing a straight line on the touch screen, specifically The process is similar to the process in which a user draws a line with a ruler in a physical real space.

Next, the relationship between the display position of the virtual ruler and the contact area of the touch gesture is described:

In order to enable the user to adjust the display position of the virtual ruler based on their own intentions, it is necessary to associate the user's touch gesture with the display position of the virtual ruler, that is, the display position of the virtual ruler needs to be based on the user's touch with the touch screen. region as a reference.

In one implementation, since the contact gesture is the contact between the side of the hand and the touch screen after the hand is unfolded, the shape of the contact area is a flat strip, and the direction indicated by the long side of the strip is in line with the four directions of the user's hand. The directions pointed by the fingers (or only the little finger) are parallel (or nearly parallel). Referring to FIG. 11 , the display direction of the virtual scale can be made to be parallel or nearly parallel to the direction indicated by the long side of the bar, and the display mode of this virtual scale conforms to the user's operating habits.

Wherein, the display direction of the virtual ruler (which may also be simply referred to as the direction of the virtual ruler in the embodiment of the present application) may be understood as the direction of the side on the virtual ruler used for measurement or drawing a reference.

In the embodiment of the present application, the virtual scale is attached to the long side of the strip-shaped area, that is, in the display position, the virtual scale and the strip-shaped area need to be very close, and in the direction, the virtual scale and the strip-shaped area need to be very close. The long sides of the elongated regions are parallel or nearly parallel.

Specifically, at the display position, the virtual ruler and the strip-shaped area satisfy one of the following conditions:

There is overlap between the virtual ruler and the strip-shaped region (such as shown in Figure 13); or, the virtual ruler is tangent to the strip-shaped region (such as shown in Figure 12); or, the virtual ruler and the distance between the strip-shaped area is less than a preset value (such as shown in Figure 14 and Figure 15, wherein Figure 15 shows the situation when the virtual ruler is a line segment), the preset value can be 1cm, 2cm, 3cm, 4cm, 5cm, 10cm, 15cm, etc.

Specifically, in the display direction, the acute angle between the direction of the virtual scale and the direction indicated by the long side of the strip-shaped area is smaller than a preset value, and the preset value can be 1 degree, 2 degrees, or 3 degrees. degrees, 4 degrees, 5 degrees, 10 degrees, etc.

Next, describe the display rules of the virtual ruler (including display position control, hidden trigger, fixed display, posture adjustment after fixed display, etc.), respectively:

1. Regarding the control of the display position of the virtual ruler:

In the embodiment of the present application, after triggering the display of the virtual ruler, the posture of the virtual ruler can be adjusted through a specific gesture, for example, changing the posture (position and angle) of the virtual ruler so that the virtual ruler can measure different elements (drawing line length or the distance between plotted points). In this embodiment, combined with the user's contact gesture, the movement of the virtual scale can be driven by the movement of the contact area on the side of the user's hand.

Specifically, the detection module may detect the movement of the contact area of the contact gesture, and adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

Wherein, the so-called movement can be understood as a change in position and/or a change in direction;

Among them, the so-called following can be understood as always fitting the display position of the virtual ruler with the contact area of the touch gesture. Since the contact area of the touch gesture moves, the display position and direction of the virtual ruler will also change, and in the virtual The display area of the ruler is always in close contact with the long side of the contact area.

Referring to FIG. 16 and FIG. 17 , the contact area of the touch gesture moves, and then the display position and direction of the virtual ruler will also change following the contact area.

2. About canceling the display of the virtual ruler:

In some scenarios, when the user does not need to use the virtual ruler, a certain gesture can be used to trigger the display of the hidden virtual ruler on the touch screen. The gesture can be a gesture of raising the hand. In the embodiment of the present application, referring to FIG. In response to the user's hand-raising gesture, the display of the virtual scale is hidden on the touch screen in response to the user's hand-raising gesture, wherein the hand-raising gesture can be understood as the side of the hand leaving the touch screen, or the contact strength between the side of the hand and the touch screen is less than A certain threshold, or the contact area between the side of the hand and the touch screen is smaller than a preset value.

3. Regarding the fixed display of the virtual ruler

In some scenarios, the virtual ruler still needs to be used after the user raises his hand, so it is necessary to trigger the fixed display of the virtual ruler on the touch screen based on a certain gesture.

In a possible implementation, the user can tap the touch screen for a preset number of times by the side of the hand to trigger the fixed display of the virtual scale on the touch screen. Referring to FIG. 30 , the detection module can indicate the The side of the user's hand touches the touch screen for a preset number of times, and after the user's hand-raising gesture is detected, the display of the virtual scale is maintained on the touch screen (for example, as shown in FIG. 31 ).

Optionally, the preset number of taps needs to occur within a preset time, and the preset time is a short time (for example, 0.1S, 0.2S, 0.3S, 0.4S, 0.5S, etc.);

Wherein, the tapping positions of the preset number of taps on the touch screen need to be kept consistent, or slight deviations are allowed.

Wherein, the preset number of times can be 2 times, 3 times, etc. For example, when the side of the hand is tapped twice continuously, the virtual ruler can be fixedly displayed on the touch screen.

4. Regarding the attitude adjustment after the virtual ruler is fixed and displayed

In one implementation, when the virtual ruler is fixedly displayed on the touch screen, a certain gesture operation can be used to adjust the display posture of the virtual ruler (the display posture may include a display position and/or a display direction). The detection module can detect a user's selection instruction for the virtual scale, and in response to the selection instruction, a trigger control can be displayed, and the trigger control is used to indicate at least one of the following operations on the virtual scale: delete operation , position adjustment operations, and rotation operations.

For example, referring to FIG. 32 , the user can trigger the selection of the virtual scale by clicking on the fixedly displayed virtual scale, and at the same time of selection, the touch screen will pop up a trigger control, for example, the trigger control can be a prompt for indicating a deletion operation for the virtual scale (such as the "delete" prompt shown in FIG. 33, and the trash can logo shown in FIG. 35), the user can delete the virtual ruler by clicking on the "delete" prompt or dragging the virtual ruler to the trash bin logo).

For example, the user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the virtual ruler is selected, a trigger control will pop up on the touch screen. 33), the user can click on the "posture adjustment" control to enter the posture adjustment mode for the virtual ruler. For example, in this mode, the user can touch the side of the hand and the touch screen To adjust the display posture of the virtual ruler, the adjustment method can refer to the relevant description of the display position control of the virtual ruler in the above embodiment, which will not be repeated here. For another example, referring to Figure 34, if you need to rotate the virtual ruler, you can use two Click on two different positions (on the virtual ruler or within a preset distance of the virtual ruler) respectively, and the two touch points rotate clockwise or counterclockwise at the same time, so as to rotate the virtual ruler. Optionally, when operating the virtual scale to rotate, the included angle between the virtual scale before and after the rotation can be displayed in real time.

It should be understood that after the user selects the virtual ruler, the display posture of the virtual ruler can be adjusted directly through gesture operations.

In one implementation, referring to FIG. 36 , when the virtual ruler is not selected, if a touch gesture between the side of the user's hand and the touch screen is detected (the condition for triggering the display of the virtual ruler is met), another virtual ruler can be additionally displayed , the new virtual ruler can also follow the translation of the receiving area on the side of the hand. In addition, the new virtual ruler can also be fixed and displayed with continuous tapping gestures.

In one implementation, if the touch screen exits the drawing interface and enters the display interface (such as displaying the screen content of other electronic devices), then the virtual ruler displayed on the drawing interface will also follow the exit of the drawing interface Disappearing will not affect the display of the projected content.

In the embodiment of the present application, by rotating or moving the side of the hand to give corresponding rotation or movement operations to the virtual ruler, tapping and fixed display of the virtual ruler and other gestures are in line with the user's operating habits and can greatly improve the operating efficiency.

Next, combine the interaction between the user and the virtual ruler to describe the function realization of the virtual ruler (including the length measurement function, the distance measurement function, and the line drawing function):

1. Length measurement function

In the embodiment of the present application, the virtual ruler may be used to measure the length of the drawn straight line segment displayed on the touch screen.

In a possible implementation, referring to FIG. 19, at least one drawing straight line segment can be displayed on the touch screen. When the user wants to measure the length of one of the drawing straight line segments, the user can adjust the virtual position by moving the touch gesture on the touch screen. The display position of the ruler, so that the display position of the virtual ruler is close to or accurately fits the straight segment of the drawing to be measured.

In one implementation, referring to FIG. 20 , when the display position of the virtual ruler accurately fits the drawn straight line segment to be measured, the detection module can acquire the length value of the drawn straight line segment where the virtual scale fits, and display the length value.

However, when the user may not be able to accurately fit the drawing line segment to be measured, or the operation required to fit the drawing line segment to be measured is relatively difficult, the user can draw when the virtual ruler is located near the drawing line segment to be measured Length measurement and length value display of straight line segments.

Specifically, the detection module detects that there is a drawing line segment associated with the virtual scale on the touch screen, and displays the length value of the drawing line segment. The degree of fit is very high. For example, posture association can be expressed by direction difference and distance value. When the direction difference between the drawing line segment and the virtual ruler is less than the preset value (for example, less than 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees , 45 degrees, etc.); and/or, when the distance between the drawing line segment and the virtual ruler is less than a preset value (such as 1cm, 2cm, 3cm, etc.), it can be considered that there is a distance between the virtual ruler and the virtual ruler on the touch screen. There is a straight line segment of the drawing associated with the pose.

Among them, the distance can be understood as the distance between the nearest point on the virtual ruler and the drawing straight line segment, or the average value of the distance between the point on the virtual ruler and the drawing straight line segment, or can be used to characterize the virtual ruler and drawing The method for the distance between straight line segments.

It should be understood that when the detection module detects that there are multiple drawing straight line segments associated with the virtual ruler in gestures on the touch screen, the length value of the drawing straight line segment with the highest degree of gesture association can be acquired and displayed.

For example, there are multiple drawing straight segments (drawing straight segment A and drawing straight segment B) associated with postures, the distance value between the virtual ruler and the drawing straight segment A is A1, and the distance value between the virtual ruler and the drawing straight segment B is B1, the direction difference between the virtual ruler and the drawing line segment A is A2, the direction difference between the virtual ruler and the drawing line segment B is B2, when A1 is equal to A2, and B1 is greater than B2, the drawing line segment B and the virtual ruler The degree of correlation between the attitude of the drawing line segment B and the virtual scale is the largest when A1 is greater than A2 and B1 is equal to B2. In addition, by calculating the distance value and direction difference, a feature quantity that can represent the distance value and direction difference can be obtained. The feature quantity is positively correlated with the distance value and direction difference. smaller.

In addition, when the detection module detects that there is a drawing line segment associated with the posture of the virtual ruler on the touch screen, and the display position of the virtual ruler does not accurately fit the drawing line segment, it can be adjusted based on the position of the drawing line segment. The display position of the virtual ruler, so that the display position of the virtual ruler fits the drawing line, that is, a display effect similar to magnetic attraction is realized.

Wherein, adjusting the display position of the virtual ruler may include adjusting the position and display direction of the virtual ruler.

Specifically, when measuring through the virtual ruler, the drawing straight line segment within the predefined range of the virtual ruler has a magnetic attraction effect on the virtual ruler. fit.

Exemplarily, referring to FIG. 21, the touch screen in FIG. 21 displays two straight line segments for drawing. Referring to FIG. ), the virtual ruler will move along with the user's gestures, refer to Figure 22b, when the virtual ruler moves to the left near the drawing straight line segment, the virtual ruler will automatically adjust the display position to fit the drawing straight line segment 1, refer to Figure 22c , when the contact area between the side of the user's hand and the touch screen continues to the right, the virtual ruler will cancel the magnetic effect, that is, it will not fit the straight line segment 1 of the drawing, and continue to follow the user's gesture. Referring to Figure 22d, the side of the user's hand and The contact area of the touch screen moves to the lower right, and when the virtual ruler moves to the vicinity of the lower right drawing line segment (drawing line segment 2), the virtual ruler will automatically adjust the display position to fit the drawing line segment 2.

In the embodiment of the present application, if the drawing line is near the virtual scale, the drawing line will have a magnetic attraction effect on the virtual scale, and automatically measure the length of the drawing line, avoiding the process of manually fitting the virtual scale and the drawing line, reducing It reduces the difficulty of measurement, avoids the problem of inaccurate fitting when fitting manually, and improves the accuracy of measurement.

The length measurement function of the virtual ruler is introduced above, and the distance measurement function of the virtual ruler is introduced next:

2. Distance measurement function

In the embodiment of the present application, the virtual ruler may be used to measure distances between drawing points displayed on the touch screen.

In a possible implementation, referring to FIG. 23 , at least one drawing line segment can be displayed on the touch screen. When the user wants to measure the distance between two points on the drawing line segment, the virtual point can be adjusted by moving the touch gesture on the touch screen. The display position of the ruler is such that the virtual ruler intersects with two points (or multiple points) on the measured drawing line segment, and then, referring to Figure 24, the detection module can obtain two intersection points (or at least two of the plurality of intersection points) intersection point) and display the distance value. Specifically, the detection module may detect that there is a first intersection point and a second intersection point between the drawing line segment on the touch screen and the virtual scale, and display the distance value between the first intersection point and the second intersection point.

The distance measurement function of the virtual ruler is introduced above, and the line drawing function of the virtual ruler is introduced next:

3. Line drawing function

In the embodiment of the present application, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

Similar to the process of using a ruler to draw a line in physical space, the virtual ruler can be used as a reference tool when the user draws a straight line on the touch screen (see Figure 25). The so-called reference tool can be understood as when the drawing track fits the ruler When , due to the obstruction of the ruler, the drawing track is limited to a straight line.

Different from the straightedge in the physical space, the straightedge in the physical space can be implemented based on its own structural characteristics and can be used as a reference tool when drawing a straight line (when placed on the drawing plane, when the user draws with a brush, the drawing track When there is an intersection with the ruler, the drawing trajectory will be restricted to a straight line by the ruler, and then a straight line can be drawn), and the virtual ruler is only an affordance displayed on the touch screen, so it is necessary to recognize the user's line drawing trajectory (or call it is the positional relationship between the line drawing position) and the virtual scale, and based on the positional relationship between the line drawing position and the virtual scale, limit the line drawing track.

Next, describe how to limit the line drawing trajectory based on the positional relationship between the line drawing position and the virtual ruler, and then realize the function of drawing a straight line based on the virtual ruler:

On the one hand, when writing within a certain range of the virtual ruler (without raising your hand), you can restrict the range of writing and drawing not to cross the virtual ruler, that is, you can only write on one side of the virtual ruler. When the drawing range exceeds the virtual ruler , displaying the straight line that fits with the virtual ruler (the side that can be used for writing and drawing), which is equivalent to that the virtual ruler has a function similar to that of the ruler in the physical space (refer to Figure 27, the left picture in Figure 27 is the user The actual drawing line trajectory, the right picture is the drawing straight line displayed after correction).

On the other hand, in response to the writing and drawing behavior within the predefined range of the virtual scale, the drawing trajectory of the writing and painting can be corrected as a straight line (refer to Figure 26, the left figure in Figure 26 is the user's actual line drawing trajectory, and the right figure is the drawing straight line displayed after correction), when the user draws a line along the virtual ruler through the above method, a straight line will be drawn automatically.

Specifically, the detection module may detect a line-drawing gesture on the touch screen, based on the fact that the virtual scale is displayed on the touch screen, and the distance between the line-drawing position of the line-drawing gesture and the virtual scale is within Within a preset distance (such as 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, etc.), according to the line drawing position of the line drawing gesture, a drawing line segment is displayed, wherein the drawing line segment is parallel to the virtual ruler.

And in order to get a line of expected length, when drawing along the ruler, the drawing line between the first and last two drawing points can be calculated and displayed, and the length of the drawing line can be displayed. In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position (refer to FIG. 28 , The left picture in Fig. 28 is the user's actual drawing line trajectory, and the right picture is the drawing straight line displayed after correction), and the detection module can display the drawing line length based on the real-time drawing line position, and the drawing line length is the described The distance value between the starting point position and the real-time drawing line position.

That is to say, if the length of the line segment drawn exceeds the expected value, you can draw it back, which will have a deletion effect. As long as the distance between the end point and the start point is the expected value, the generated line segment is the expected value, and the drawn Curves (the distance between the curve and the virtual ruler is within the preset range) will be automatically corrected to a straight line.

As shown in Figure 29, suppose the user needs to draw a line segment with a length of 15cm (that is, the expected length of the line segment is 15cm) when drawing a picture, starting from the starting point. However, when the user draws a line segment on the screen, he cannot draw a 15cm line segment as accurately as when drawing a line along a ruler on a white paper. Realize that what the user sees is a straight line) The length of the obtained line segment A1 is 25 (as shown in the left figure in Figure 29).

At this time, the general solution is to erase a part or redraw, but it is also difficult to obtain a line segment of an accurate length by erasing or redrawing. In this example, as shown in the right figure in Figure 29, if you move the stylus in the opposite direction, the position of the end point will move back (the position of the start point is the starting point of drawing with the stylus, and the position of the end point is the position of the stylus when drawing a picture) The last position, during the drawing process, the position of the end point can be regarded as a dynamic movement, until the drawing is completed, the position where the stylus is detached is the final end point position).

In the right picture in Figure 29, when the stylus moves in the opposite direction, since the virtual ruler will correct the straight line/curve drawn by the stylus to a straight line, the line segment A2 is equivalent to drawing twice, the first time ( Forward drawing) to draw the trajectory line, when the second (reverse drawing) is repeated with the first drawing, the repeated line segment A2 is deleted, that is, part of the first straight line is erased, and finally only the starting point position is retained and the line segment A3 between the position of the end point. When marking the length within the line segment, only mark the length of the line segment A3 between the start point position and the end point position. After the line segment is deleted, the stylus can be drawn again when it is redrawn. Therefore, a line segment with an expected length of 15 cm can be obtained by (repeatedly) adjusting the position of the end point.

It should be understood that in order to avoid drawing geometric figures, the intersection of two line segments is deleted (for example, the intersection of the diagonals of a square, when drawing two different diagonals, the intersection will also be drawn twice). The deletion effect of repeated line drawing can increase the time judgment mechanism. Only when the same line segment is drawn twice within the predetermined time can the deletion effect be achieved. Or it can be realized through other judgment mechanisms. For example, only when the direction of movement of the stylus is opposite twice before and after is met, the direction mechanism such as deletion judgment is satisfied.

In the embodiment of this application, the two functions of the ruler: measuring and drawing a straight line are converted into intelligent operations suitable for the virtual interface. The measurement is to automatically display the measured value, and the length value is displayed when drawing a straight line, which removes the limitations of the physical world and saves It takes less time for users to take readings, greatly improving user efficiency.

An embodiment of the present application provides a method for displaying a virtual scale, including: detecting a contact gesture with the touch screen; indicating the contact of the user's hand side with the touch screen based on the contact gesture, and according to the contact area of the contact gesture, A virtual scale is displayed on the touch screen, wherein the contact area of the contact gesture is a strip-shaped area, and the virtual scale is attached to a long side of the strip-shaped area. When a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual scale is triggered. On the one hand, only the user's single hand is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand When the upper part is unfolded, the contact area between the side of the hand and the touch screen is a flat strip area, which is close to a straight line. Based on this operation mode, triggering the display of the virtual scale is more in line with the user's operation habits.

Referring to Fig. 37a, Fig. 37a is a schematic flowchart of an object copying method provided by the embodiment of the present application. As shown in Fig. 37a, an object copying method provided by the embodiment of the present application includes:

3701. Detect a drag gesture for a target object displayed on the touch screen.

There are gestures related to the copy operation on mainstream terminal devices (such as smartphones and tablet computers). For example, the existing operation logic includes three-finger pinch to complete copy, and three-finger release to paste. At the same time, for copy and paste, the industry has a set of basic logic: select the content, right click or special button to enter the semantic menu, click the copy button, move to the target position to call up the semantic menu, and click the paste button. It can be seen that the mainstream method must select elements as a prerequisite before copying and pasting. There are many operation steps, and the operation cost of copying and pasting is very high.

In the embodiment of the present application, when performing a copy and paste operation, the copied object can be dragged to a position to be pasted, and the position where the hand is raised is the pasted position.

The following describes how to select the copied object:

1. By long press gesture:

In this embodiment of the present application, when a long press gesture for the target object is detected, the copy function for the target object may be enabled. Wherein, the long press gesture may be a two-finger long press gesture, for example, refer to FIG. 37b.

Wherein, the target object is a drawing object displayed on the touch screen, for example, it may be a character, pattern, stroke or the like.

Wherein, the long press gesture may include a light press and a deep press, and when the contact intensity increases from an intensity lower than the light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold, it may be called a touch detection "Light press" input on the surface. An increase in contact intensity from an intensity below the deep press intensity threshold to an intensity above the deep press intensity threshold may be referred to as detecting a "deep press" input on the touch surface. An increase in contact intensity from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold may be referred to as detecting a "swipe touch" input on the touch surface. When the contact intensity decreases from an intensity higher than the contact detection intensity threshold to an intensity lower than the contact detection intensity threshold, it may be referred to as detecting that the contact is lifted from the touch surface (ie, a hand-raising gesture). In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold may be greater than zero.

In a possible implementation, besides when the user swipes the touch screen, it will be considered as a false touch, and if the duration of the long press gesture is too short, it can also be considered as a false touch. In this case, the display of the virtual ruler may not be triggered.

In this embodiment of the present application, the duration of the long-press gesture can also be used as a trigger to enable the copy function for the target object, that is, it needs to be detected that the duration of the long-press gesture is greater than a preset time, such as a preset The time can be set as 0.1S, 0.15S, 0.2S, 0.25S, 0.3S, 0.35S, 0.4S, 0.45S, 1S, 2S.

Wherein, the maintenance time of the long press gesture can start counting when the long press gesture is detected, or after detecting that the contact intensity between the long press gesture and the touch screen is greater than a certain contact intensity threshold (threshold greater than 0), the maintenance time The timing can be ended when the contact intensity of the long press gesture is detected to be 0, or when the contact intensity of the long press gesture is detected to be less than a certain contact intensity threshold (threshold greater than 0).

Wherein, the maintenance time of the long press gesture can be the maintenance time when the long press gesture remains in a static state (or the movement is less than a certain range), for example, the maintenance time of the long press gesture can be when the press gesture is at rest (or the movement is less than a certain range) the maintenance time. It should be understood that the above static can be interpreted as that the contact area of the long press gesture does not move or moves in a small range.

Among them, the so-called enabling the copy function for the target object can be understood as activating the copy state. When the copy state is not activated, the display elements on the drawing and writing interface cannot be copied. The operation of the display element itself; only after the copy function for the display element is enabled, the copy status of the display element is activated, and it can respond to the next operations related to copy and paste.

Wherein, the so-called long press gesture for the target object can be understood as the target object is covered by the contact area of the long press gesture.

In a possible implementation, when the detection module detects the long press gesture, if there is no display element in the contact area of the long press gesture, the copy command will not be triggered (that is, the copy function for the display element will not be enabled. ). If there is a display element (for example, a target object), the display element can be selected, and the display elements within this range are placed in a copy activation state (that is, the copy function for the target object is enabled).

Taking the long-press gesture as an example of a two-finger long-press gesture, if the touch gesture is a two-finger long-press gesture, and there is a display element in the contact area, then select the display element and put the element in a copy-activated state (that is, enable replication for the target object).

In one implementation, the so-called long press gesture for the target object may also be understood as the contact area of the long press gesture is within a preset distance around the target object.

In the embodiment of the present application, the contact area of the long press gesture can be associated with a fixed predefined range, or can be extended on the basis of the predefined range, for example, within the preset distance of the boundary of the predefined range, it can be found pattern, the range can be extended to the newly discovered pattern, and the newly discovered pattern can also be applied to this extension. For example, referring to Figure 43, when two fingers long press the word "cow", a circle is determined according to the predefined range area, and select the pattern in the circular area, that is, select the word "cow". For example, referring to Figure 44, you can also perform a judgment near the boundary to find whether there is a pattern within the preset distance (such as 5mm) of the circular boundary , then select at least one stroke of the word "Nian", then select the entire "Nian" character, and then select the characters "Da" and "Ji" in turn. In this way, the four words "good luck in the year of the ox" can be selected at one time. When performing range extension, the extended shape can be adapted to a predefined shape, for example, a circular translation is performed to finally obtain an ellipse. It can also be adapted to the preset distance of the boundary judgment, such as an irregular pattern that is adapted to the actual shape and the boundary of the word.

It should be understood that this extension can be interrupted when a pattern that does not need to be copied is selected. For example, referring to FIG. 45 , an additional finger slide can be used to divide the selected area into two parts, and then the area corresponding to the two-finger part is deselected. As shown in the figure below, after splitting, if the word "Ji" is not selected, when you move two fingers, only "Nian Da of the Ox" will be copied.

2. By single-click gesture:

In the embodiment of the present application, the copy function for the target object may be enabled after detecting the click gesture for the target object.

In a possible implementation, referring to FIG. 40 , the detection of the click gesture for the target object may be understood as the detection of a click gesture in which the contact area with the touch screen covers the target object.

The difference between single-click to select and long-press to select and copy is that single-click to select an object, and two-finger selection selects the display elements within a predefined range, which is crucial for copying on the drawing board. When writing on the drawing board, because the text is composed of strokes, when writing, the strokes are independent. Therefore, one stroke is a pattern, and a word is composed of several patterns (strokes). Therefore, when you click to select, only If one stroke can be selected (regardless of continuous strokes), the copy can only copy the selected stroke (such as the horizontal stroke in the picture above), and cannot completely copy the entire character. When selecting with two fingers, the pattern in the area of the two fingers is selected, and the entire character can be selected, so the entire character can be completely copied.

In addition, with reference to Fig. 46 and Fig. 47, the copy selection of the target object can also be performed based on the frame selection control (which can be physical or virtual) displayed on the touch screen. After the button is selected, the frame selection of the icon can be carried out , after selecting the icon, select the target object, and then enable the copy function for the target object.

The above describes how to enable the copy function for the target object, and then describes how to copy the target object:

In the embodiment of the present application, the user performs a drag gesture on the touch screen for the target object, and the end point of the drag gesture (that is, the position of the hand-raising gesture) may be the position where the paste operation for the target object needs to be performed. Wherein, the drag gesture may be a two-finger drag gesture.

In a possible implementation, the user triggers the copy function for the target object through a long press gesture, then the drag gesture may be to keep the long press gesture in contact with the touch screen and move on the touch screen gesture.

In a possible implementation, referring to Figure 41, the user triggers the copy function for the target object through the click gesture, and then uses the drag gesture to perform the copy operation, that is, the selection operation and the copy operation are separated. At this time, The starting point of the touch area of the drag gesture may not be (or be) near the display position of the target object, but can be dragged at any display position on the touch screen, and the copied target object can be copied, and the target object can be dragged The object slides in the direction of the two-finger swipe.

3702. In response to the drag gesture, display the mirror image of the target object on the touch screen, and update the display position of the mirror image in real time according to the drag gesture, so that the mirror image follows the drag gesture move;

Referring to FIG. 38 , in the embodiment of the present application, after the detection module detects a drag gesture for the target object, the target object has a tendency to be dragged away, and after detecting the instruction of the drag gesture, the target object is copied Operation, copy the target object, generate a copy body (or called a mirror image), and make the mirror image move with the movement of the drag gesture, and the body of the target object is still in the original position without sliding.

In a possible implementation, in order to indicate the replica, the mirror image of the target object may appear in a semi-blurred state.

3703. A hand raising gesture is detected.

For a specific description of the hand-raising gesture, reference may be made to relevant descriptions in the foregoing embodiments, and details are not repeated here.

3704. In response to the hand-raising gesture, fix and display the mirror image at the display position where the mirror image is located.

Referring to Fig. 39 and Fig. 42, in the embodiment of the present application, when the drag gesture slides to the expected position, the hand gesture is performed, that is, when the hand is out of contact with the display screen or the contact intensity is less than the threshold, the paste operation is triggered. It is to paste the target object at the position of the hand gesture, and then complete the copy and paste process.

In addition, the above-mentioned gesture rules for the copy operation are not only applicable to the drawing elements on the drawing interface, but also applicable to the copying of text on the display interface. Referring to Figure 48, after selecting the text, press and hold the selected text (for example, in Figure 48 Press and hold with two fingers as shown), drag to copy a copy, after release, the copy will be displayed at the position where the finger leaves.

An embodiment of the present application provides an object copying method, the method comprising: detecting a drag gesture for a target object displayed on a touch screen; in response to the drag gesture, displaying the target object on the touch screen and according to the drag gesture, update the display position of the mirror image in real time, so that the mirror image moves with the drag gesture; detect a hand-raising gesture; respond to the hand-raising gesture, move the mirror image It is fixedly displayed at the display position where the mirror image is located. The method of copying and pasting by dragging gestures is simple and intuitive, which reduces the number of steps required for existing copy and paste, and shortens the process of copying and pasting.

In addition, based on the long-press gesture to activate the copy activation state, the way of copying can be performed. The gesture is simple and commonly used, and the operation is simple and conforms to the user's cognition.

Referring to Fig. 49, Fig. 49 is a schematic structural diagram of a virtual ruler display device provided by an embodiment of the present application, which is applied to electronic equipment, the electronic equipment includes a touch screen, and the device 4900 includes:

A detection module 4901, configured to detect a contact gesture with the touch screen;

For a specific description of the detection module 4901, reference may be made to the description of step 301, which will not be repeated here.

The display module 4902 is configured to indicate the contact of the side of the user's hand with the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is a strip-shaped area, and the virtual scale is attached to the long side of the strip-shaped area.

For a specific description of the display module 4902, reference may be made to the description of step 302, which will not be repeated here.

In a possible implementation, the side of the hand is the side of the user's hand that is located on the side of the little finger in an unfolded state.

In a possible implementation, the contact area includes:

the contact area of the hypothenar portion of the side of the user's hand with the touch screen; or,

The contact area between the little finger on the side of the user's hand and the touch screen.

In a possible implementation, the device also includes:

A determining module 4904, configured to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on that the contact area of the contact gesture satisfies a preset condition; the preset condition includes:

The contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar part and/or the little finger part of the side of the user's hand is in contact with the touch screen.

In a possible implementation, the device also includes:

An acquisition module 4903, configured to acquire gesture data of the contact gesture;

The determination module 4904 is further configured to determine, according to the gesture data, through a neural network, that the contact gesture indicates that the side of the user's hand is in contact with the touch screen.

In a possible implementation, the virtual ruler is attached to the long side of the strip-shaped region, including:

The acute angle between the direction of the virtual scale and the direction indicated by the long side of the strip-shaped area is smaller than a preset value; and the virtual scale and the strip-shaped area meet one of the following conditions:

There is overlap between the virtual scale and the strip-shaped area; or, the virtual scale is tangent to the strip-shaped area; or, the distance between the virtual scale and the strip-shaped area is less than a preset value.

In a possible implementation, the detection module 4901 is configured to detect that the duration of the contact gesture is longer than a preset before displaying a virtual ruler on the touch screen in the contact area according to the contact gesture time.

In a possible implementation, the detection module 4901 is configured to detect the movement of the contact area of the contact gesture;

The display module 4902 is configured to adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

In a possible implementation, the detection module 4901 is configured to detect the user's hand-raising gesture;

The display module is configured to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

In a possible implementation, the instructing the contact of the side of the user's hand with the touch screen based on the contact gesture includes:

Instructing the side of the user's hand to tap the touch screen for a preset number of times based on the contact gesture;

After the virtual ruler is displayed on the touch screen, the method further includes:

The user's hand gesture is detected;

In response to the raising hand gesture, maintaining display of the virtual ruler on the touch screen.

In a possible implementation, the detection module 4901 is configured to detect a user's selection instruction for the virtual scale;

The display module is configured to display a trigger control in response to the selection instruction, and the trigger control is used to indicate at least one of the following operations on the virtual scale:

Delete operations, reposition operations, and rotation operations.

In a possible implementation, the virtual ruler is used to measure the length of a drawn line segment displayed on the touch screen; or,

The virtual ruler is used for distance measurement between drawing points displayed on the touch screen.

In a possible implementation, the detection module 4901 is configured to detect that there is a drawing line segment associated with the gesture of the virtual ruler on the touch screen;

The display module is used to display the length value of the straight line segment of the drawing; the existence gesture association includes:

The direction difference from the virtual scale is smaller than a preset value; and/or,

The distance from the virtual ruler is smaller than a preset value.

In a possible implementation, the display module 4902 is configured to adjust the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler fits with the drawn straight line.

In a possible implementation, the detection module 4901 is configured to detect that there is a first intersection point and a second intersection point between the drawn line segment on the touch screen and the virtual scale after the virtual scale is displayed on the touch screen;

The display module 4902 is configured to display the distance value between the first intersection point and the second intersection point.

In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

In a possible implementation, the detection module 4901 is configured to detect a gesture of drawing a line on the touch screen;

The display module 4902 is configured to display the virtual ruler on the touch screen, and the distance between the line drawing position of the line drawing gesture and the virtual ruler is within a preset distance, according to the line drawing The line drawing position of the gesture displays a drawing line segment, wherein the drawing line segment is parallel to the virtual ruler.

In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes:

Based on the real-time line-drawing position, display the line-drawing length, where the line-drawing length is a distance value between the starting point position and the real-time line-drawing position.

An embodiment of the present application provides a virtual ruler display device, including: a detection module 4901, configured to detect a contact gesture with the touch screen; a display module 4902, configured to indicate that the side of the user's hand is in contact with the touch screen based on the contact gesture According to the contact area of the contact gesture, a virtual ruler is displayed on the touch screen, wherein the contact area of the contact gesture is a strip-shaped area, and the virtual scale and the long side of the strip-shaped area fit. When a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual scale is triggered. On the one hand, only the user's single hand is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand When the upper part is unfolded, the contact area between the side of the hand and the touch screen is a flat strip area, which is close to a straight line. Based on this operation mode, triggering the display of the virtual scale is more in line with the user's operation habits.

Referring to FIG. 50, FIG. 50 is a schematic structural diagram of an object replication device provided in an embodiment of the present application. As shown in FIG. 50, the device 5000 includes:

The detection module 5001 is configured to detect a drag gesture for a target object displayed on the touch screen; the detection module is also configured to detect a hand-raising gesture after the display module displays a mirror image of the target object;

For a specific description of the detection module 5001, reference may be made to the descriptions of step 3701 and step 3703, which will not be repeated here.

The display module 5002 is configured to display a mirror image of the target object on the touch screen in response to the drag gesture, and update the display position of the mirror image in real time according to the drag gesture, so that the mirror image follows the The drag gesture moves; the display module is further configured to, in response to the hand-raising gesture, fix and display the mirror image at the display position where the mirror image is located.

For a specific description of the display module 5002, reference may be made to the descriptions of step 3702 and step 3704, which will not be repeated here.

In a possible implementation, the device also includes:

The enabling module 5003 is configured to enable the copy function for the target object when at least one of the following gestures is detected before the detection of the drag gesture for the target object displayed on the touch screen:

detecting a long press gesture directed at the target object; or,

A click gesture is detected for the target object.

In a possible implementation, the detection module 5001 is configured to detect a long press gesture in which the contact area with the touch screen covers the target object or is within a preset distance around the target object.

In a possible implementation, the dragging gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

In a possible implementation, the long press gesture is a two-finger long press gesture.

In a possible implementation, the detection module 5001 is configured to detect a click gesture that a contact area with the touch screen covers the target object.

An embodiment of the present application provides an object copying device, including: a detection module 5001, configured to detect a drag gesture for a target object displayed on a touch screen; the detection module is also configured to display the target on a display module A hand gesture is detected after the mirror image of the object; the display module 5002 is configured to display the mirror image of the target object on the touch screen in response to the drag gesture, and update the mirror image in real time according to the drag gesture The display position of the mirror image so that the mirror image moves along with the drag gesture; the display module is further configured to display the mirror image at the display position where the mirror image is located in response to the hand-raising gesture. The method of copying and pasting by dragging gestures is simple and intuitive, which reduces the number of steps required for existing copy and paste, and shortens the process of copying and pasting.

Next, a terminal device provided by the embodiment of the present application is introduced. The terminal device may be the device described in FIG. 49 and FIG. 50. Please refer to FIG. 51. FIG. 51 is a schematic structural diagram of the terminal device provided by the embodiment of the present application. The terminal device 5100 may specifically be an electronic whiteboard, a virtual reality VR device, a mobile phone, a tablet, a notebook computer, a smart wearable device, etc., which are not limited here. Specifically, the terminal device 5100 includes: a receiver 5101, a transmitter 5102, a processor 5103, and a memory 5104 (the number of processors 5103 in the terminal device 5100 can be one or more, and one processor is taken as an example in FIG. 51 ) , where the processor 5103 may include an application processor 51031 and a communication processor 51032 . In some embodiments of the present application, the receiver 5101, the transmitter 5102, the processor 5103, and the memory 5104 may be connected through a bus or in other ways.

The memory 5104 may include read-only memory and random-access memory, and provides instructions and data to the processor 5103 . A part of the memory 5104 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM). The memory 5104 stores processors and operating instructions, executable modules or data structures, or their subsets, or their extended sets, wherein the operating instructions may include various operating instructions for implementing various operations.

The processor 5103 controls the operation of the terminal device. In a specific application, various components of the terminal device are coupled together through a bus system, where the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, the various buses are referred to as bus systems in the figures.

The methods disclosed in the foregoing embodiments of the present application may be applied to the processor 5103 or implemented by the processor 5103 . The processor 5103 may be an integrated circuit chip and has a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 5103 or instructions in the form of software. The above-mentioned processor 5103 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application-specific integrated circuit (application specific integrated circuit, ASIC), field programmable Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 5103 may implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 5104, and the processor 5103 reads the information in the memory 5104, and completes the steps of the above method in combination with its hardware. Specifically, the processor 5103 can read the information in the memory 5104, and combine its hardware to complete the steps related to the display of the virtual scale in the steps 301 to 302 in the above embodiment, and the steps related to the display of the virtual scale in the steps 3701 to 3704 in the above embodiment. Copy the steps associated with the method.

The receiver 5101 can be used to receive input digital or character information, and generate signal input related to related settings and function control of the terminal device. The transmitter 5102 can be used to output digital or character information through the first interface; the transmitter 5102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 5102 can also include a display device such as a touch screen.

An embodiment of the present application further provides a computer program product, which, when running on a computer, causes the computer to execute the steps of the method described in the embodiment corresponding to FIG. 3 and FIG. 37a in the above embodiments.

An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for signal processing, and when it is run on a computer, the computer executes the method described in the foregoing embodiments The steps of the image processing method.

The image display device provided by the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or a circuit, etc. . The processing unit can execute the computer-executed instructions stored in the storage unit, so that the chips in the execution device execute the data processing methods described in the above embodiments, or make the chips in the training device execute the data processing methods described in the above embodiments. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit located outside the chip in the wireless access device, such as only Read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.

In addition, it should be noted that the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be A physical unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course it can also be realized by special hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, Special components, etc. to achieve. In general, all functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure used to realize the same function can also be varied, such as analog circuits, digital circuits or special-purpose circuit etc. However, for this application, software program implementation is a better implementation mode in most cases. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment of the present application .

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc.

Claims (53)

A method for displaying a virtual scale, characterized in that it is applied to an electronic device, the electronic device includes a touch screen, and the method includes: detecting a contact gesture with the touch screen; Based on the contact gesture indicating that the side of the user's hand is in contact with the touch screen, displaying a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is a strip-shaped area, The virtual ruler is attached to the long side of the strip-shaped area. The method according to claim 1, characterized in that, the side of the hand is the side of the user's hand that is located on the side of the little finger in an unfolded state. The method according to claim 1 or 2, wherein the contact area comprises: the contact area of the hypothenar portion of the side of the user's hand with the touch screen; or, The contact area between the little finger on the side of the user's hand and the touch screen. The method according to any one of claims 1 to 3, wherein the method further comprises: Based on the contact area of the contact gesture meeting a preset condition, it is determined that the contact gesture indicates that the user's hand side is in contact with the touch screen; the preset condition includes: The contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar part and/or the little finger part of the side of the user's hand is in contact with the touch screen. The method according to any one of claims 1 to 3, wherein the method further comprises: acquiring gesture data of the contact gesture; According to the gesture data, through a neural network, it is determined that the contact gesture indicates that the side of the user's hand is in contact with the touch screen. The method according to any one of claims 1 to 5, wherein the virtual ruler is attached to the long side of the elongated region, comprising: The acute angle between the direction of the virtual scale and the direction indicated by the long side of the strip-shaped area is smaller than a preset value; and the virtual scale and the strip-shaped area meet one of the following conditions: There is overlap between the virtual scale and the strip-shaped area; or, the virtual scale is tangent to the strip-shaped area; or, the distance between the virtual scale and the strip-shaped area is less than a preset value. The method according to any one of claims 1 to 6, wherein, before displaying a virtual ruler on the touch screen according to the contact area of the contact gesture, the method further comprises: The maintenance time for detecting the contact gesture is longer than a preset time. The method according to any one of claims 1 to 7, wherein the method further comprises: detecting movement of the contact area of the contact gesture; The display position of the virtual scale is adjusted so that the display position of the virtual scale follows the contact area. The method according to any one of claims 1 to 8, wherein the method further comprises: The user's hand gesture is detected; In response to the hand-raising gesture, the display of the virtual ruler is hidden on the touch screen. The method according to any one of claims 1 to 9, wherein the instructing the contact of the side of the user's hand with the touch screen based on the contact gesture comprises: Instructing the side of the user's hand to tap the touch screen for a preset number of times based on the contact gesture; After the virtual ruler is displayed on the touch screen, the method further includes: The user's hand gesture is detected; In response to the raising hand gesture, maintaining display of the virtual ruler on the touch screen. The method according to claim 10, characterized in that the method further comprises: detecting a user's selection instruction for the virtual scale; In response to the selection instruction, a trigger control is displayed, and the trigger control is used to instruct to perform at least one of the following operations on the virtual scale: Delete operations, reposition operations, and rotation operations. The method according to any one of claims 1 to 11, wherein the virtual ruler is used to measure the length of the drawn straight line segment displayed on the touch screen; or, The virtual ruler is used for distance measurement between drawing points displayed on the touch screen. The method according to any one of claims 1 to 12, further comprising: Detecting that there is a drawing straight line segment associated with the virtual scale on the touch screen, and displaying the length value of the drawing straight line segment; the existing gesture association includes: The direction difference from the virtual scale is smaller than a preset value; and/or, The distance from the virtual ruler is smaller than a preset value. The method according to claim 13, further comprising: According to the position of the line segment to be measured, the display position of the virtual ruler is adjusted so that the display position of the virtual ruler fits with the drawn straight line. The method according to any one of claims 1 to 14, wherein after displaying the virtual ruler on the touch screen, the method further comprises: It is detected that there is a first intersection point and a second intersection point between the drawn line segment on the touch screen and the virtual ruler, and a distance value between the first intersection point and the second intersection point is displayed. The method according to any one of claims 1 to 15, wherein the virtual ruler is used as a reference tool when drawing a straight line on the touch screen. The method according to any one of claims 1 to 16, further comprising: detecting a line-drawing gesture on the touch screen; Based on the virtual ruler being displayed on the touch screen, and the distance between the line-drawing position of the line-drawing gesture and the virtual ruler is within a preset distance, displaying a drawing according to the line-drawing position of the line-drawing gesture A straight line segment, wherein the drawn straight line segment is parallel to the virtual ruler. The method according to claim 17, wherein the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position. The method according to claim 17 or 18, wherein the line drawing position comprises a starting point position and a real-time line drawing position, and the method further comprises: Based on the real-time line-drawing position, display the line-drawing length, where the line-drawing length is a distance value between the starting point position and the real-time line-drawing position. A method for copying an object, characterized in that the method comprises: Detecting a drag gesture directed at the target object displayed on the touch screen; In response to the drag gesture, displaying a mirror image of the target object on the touch screen, and updating a display position of the mirror image in real time according to the drag gesture, so that the mirror image moves along with the drag gesture; A hand gesture is detected; In response to the hand-raising gesture, the mirror image is fixedly displayed at the display position where the mirror image is located. The method according to claim 20, wherein before the detection of the drag gesture for the target object displayed on the touch screen, the method further comprises: When at least one of the following gestures is detected, the copy function for the target object is enabled: detecting a long press gesture directed at the target object; or, A click gesture is detected for the target object. The method according to claim 21, wherein the detecting the long press gesture for the target object comprises: A long press gesture is detected in which the contact area with the touch screen covers the target object or is within a preset distance around the target object. The method according to claim 21 or 22, wherein the dragging gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen. The method according to any one of claims 21 to 23, wherein the long press gesture is a two-finger long press gesture. The method according to any one of claims 21 to 24, wherein the detecting the click gesture for the target object comprises: A click gesture in which a contact area with the touch screen covers the target object is detected. A virtual scale display device, characterized in that it is applied to electronic equipment, the electronic equipment includes a touch screen, and the device includes: A detection module, configured to detect a contact gesture with the touch screen; A display module, configured to indicate the contact of the side of the user's hand with the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of the contact gesture, wherein the contact area of the contact gesture is In the strip-shaped area, the virtual ruler is attached to the long side of the strip-shaped area. The device according to claim 26, wherein the side of the hand is the side of the user's hand located on the side of the little finger when the hand is unfolded. Apparatus according to claim 26 or 27, wherein said contact area comprises: the contact area of the hypothenar portion of the side of the user's hand with the touch screen; or, The contact area between the little finger on the side of the user's hand and the touch screen. The device according to any one of claims 26 to 28, wherein the device further comprises: A determining module, configured to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on that the contact area of the contact gesture satisfies a preset condition; the preset condition includes: The contact area is a strip-shaped area, and the shape and size of the strip-shaped area conform to the characteristics of the contact area when the hypothenar part and/or the little finger part of the side of the user's hand is in contact with the touch screen. The device according to any one of claims 26 to 28, wherein the device further comprises: An acquisition module, configured to acquire gesture data of the contact gesture; The determining module is further configured to determine, according to the gesture data, through a neural network, that the contact gesture indicates that the side of the user's hand is in contact with the touch screen. The device according to any one of claims 26 to 30, wherein the virtual ruler is attached to the long side of the elongated region, including: The acute angle between the direction of the virtual scale and the direction indicated by the long side of the strip-shaped area is smaller than a preset value; and the virtual scale and the strip-shaped area meet one of the following conditions: There is overlap between the virtual scale and the strip-shaped area; or, the virtual scale is tangent to the strip-shaped area; or, the distance between the virtual scale and the strip-shaped area is less than a preset value. The device according to any one of claims 26 to 31, wherein the detection module is configured to detect the The maintenance time of the contact gesture is longer than the preset time. The device according to any one of claims 26 to 32, wherein the detection module is configured to detect movement of the contact area of the contact gesture; The display module is configured to adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area. The device according to any one of claims 26 to 33, wherein the detection module is configured to detect a user's hand-raising gesture; The display module is configured to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture. The device according to any one of claims 26 to 34, wherein the instructing the contact of the side of the user's hand with the touch screen based on the contact gesture comprises: Instructing the side of the user's hand to tap the touch screen for a preset number of times based on the contact gesture; After the virtual ruler is displayed on the touch screen, the method further includes: The user's hand gesture is detected; In response to the raising hand gesture, maintaining display of the virtual ruler on the touch screen. The device according to claim 35, wherein the detection module is configured to detect a user's selection instruction for the virtual scale; The display module is configured to display a trigger control in response to the selection instruction, and the trigger control is used to indicate at least one of the following operations on the virtual scale: Delete operations, reposition operations, and rotation operations. The device according to any one of claims 26 to 36, wherein the virtual ruler is used to measure the length of a drawn straight line segment displayed on the touch screen; or, The virtual ruler is used for distance measurement between drawing points displayed on the touch screen. The device according to any one of claims 26 to 37, wherein the detection module is configured to detect that there is a drawing line segment associated with the posture of the virtual ruler on the touch screen; The display module is used to display the length value of the straight line segment of the drawing; the existence gesture association includes: The direction difference from the virtual scale is smaller than a preset value; and/or, The distance from the virtual ruler is smaller than a preset value. The device according to claim 38, wherein the display module is configured to adjust the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler is consistent with the drawing Straight fit. The device according to any one of claims 26 to 39, wherein the detection module is configured to detect that the drawing line segment on the touch screen and the virtual scale exist for the first time after the virtual scale is displayed on the touch screen. a first intersection point and a second intersection point; The display module is configured to display a distance value between the first intersection point and the second intersection point. The device according to any one of claims 26 to 40, wherein the virtual ruler is used as a reference tool when drawing a straight line on the touch screen. The device according to any one of claims 26 to 41, wherein the detection module is configured to detect a line-drawing gesture on the touch screen; The display module is configured to display the virtual ruler on the touch screen, and the distance between the line-drawing position of the line-drawing gesture and the virtual ruler is within a preset distance, according to the line-drawing gesture The position of the drawn line, displaying the drawn straight line segment, wherein the drawn straight line segment is parallel to the virtual ruler. The device according to claim 42, wherein the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position. The device according to claim 41 or 42, wherein the line drawing position includes a starting point position and a real-time line drawing position, and the method further comprises: Based on the real-time line-drawing position, display the line-drawing length, where the line-drawing length is a distance value between the starting point position and the real-time line-drawing position. An object replication device, characterized in that the device comprises: A detection module, configured to detect a drag gesture for a target object displayed on the touch screen; the detection module is also configured to detect a hand-raising gesture after the display module displays a mirror image of the target object; A display module, configured to display a mirror image of the target object on the touch screen in response to the drag gesture, and update the display position of the mirror image in real time according to the drag gesture, so that the mirror image follows the The drag gesture moves; the display module is further configured to, in response to the hand-raising gesture, fix and display the mirror image at the display position where the mirror image is located. The device according to claim 45, further comprising: An enabling module, configured to enable the copy function for the target object when at least one of the following gestures is detected before the detection of the drag gesture for the target object displayed on the touch screen: detecting a long press gesture directed at the target object; or, A click gesture is detected for the target object. The device according to claim 46, the detecting the long press gesture for the target object comprises: A long press gesture is detected in which the contact area with the touch screen covers the target object or is within a preset distance around the target object. The device according to claim 46 or 47, wherein the dragging gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen. According to the device according to any one of claims 46 to 48, the long press gesture is a two-finger long press gesture. According to the device according to any one of claims 46 to 49, the detection of the single-click gesture for the target object comprises: A click gesture in which a contact area with the touch screen covers the target object is detected. An electronic device, characterized in that the device includes a processor, a memory, a touch screen and a bus, wherein: The processor, the memory and the touch screen are connected through the bus; The memory is used to store computer programs or instructions; The processor is used to call or execute the programs or instructions stored in the memory, and is also used to control the touch screen, so as to realize the method steps described in any one of claims 1-25. A computer-readable storage medium, including a program, which, when run on a computer, causes the computer to execute the method according to any one of claims 1 to 25. A computer program product containing instructions, characterized in that, when the computer program product is run on a terminal, the terminal is made to execute the method of any one of claims 1-25.

Images