CN120612735B - Data generation methods, apparatus, electronic devices and computer-readable storage media - Google Patents
Data generation methods, apparatus, electronic devices and computer-readable storage mediaInfo
- Publication number
- CN120612735B CN120612735B CN202511101418.6A CN202511101418A CN120612735B CN 120612735 B CN120612735 B CN 120612735B CN 202511101418 A CN202511101418 A CN 202511101418A CN 120612735 B CN120612735 B CN 120612735B
- Authority
- CN
- China
- Prior art keywords
- data
- gesture
- peak value
- peak
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Analysis (AREA)
Abstract
本申请实施例公开了一种数据生成方法、装置、电子设备及计算机可读存储介质，涉及数据处理技术领域，该方法包括：获取手势运动数据，从该手势运动数据中识别出至少一个目标峰值，根据各目标峰值从手势运动数据中提取出至少一个手势片段数据，对各手势片段数据进行标注，得到目标手势数据集。通过从手势运动数据中提取目标峰值，并基于该目标峰值从手势运动数据中提取手势片段数据，提升手势片段数据的有效性和准确性，进而在基于该手势片段数据进行标注得到目标手势数据集后，也提升了目标手势数据集的精度。
This application discloses a data generation method, apparatus, electronic device, and computer-readable storage medium, relating to the field of data processing technology. The method includes: acquiring gesture motion data; identifying at least one target peak from the gesture motion data; extracting at least one gesture segment data from the gesture motion data based on each target peak; and labeling each gesture segment data to obtain a target gesture dataset. By extracting target peaks from the gesture motion data and extracting gesture segment data based on these target peaks, the effectiveness and accuracy of the gesture segment data are improved. Consequently, after labeling the gesture segment data to obtain the target gesture dataset, the accuracy of the target gesture dataset is also improved.
Description
Technical Field
The embodiment of the application relates to the technical field of data processing, in particular to a data generation method, a data generation device, electronic equipment and a computer readable storage medium.
Background
The gesture recognition technology is used as a natural and visual man-machine interaction mode, and has wide application prospects in the fields of consumer electronics, intelligent home, medical rehabilitation, virtual reality/augmented reality (VR/AR), vehicle-mounted control and the like. Especially on wearable devices (e.g. smart watches, smart bracelets), gesture recognition provides a convenient and efficient interaction complementary solution due to limited screen size, e.g. for fast starting applications, controlling music playback, answering/hanging up, browsing notifications, and device manipulation in special scenarios (e.g. sports, wet hands). And the interactive experience of the user is improved.
Gesture recognition based on Inertial Measurement Units (IMUs) is of particular interest in wearable devices because of their low power consumption, small volume, insensitivity to ambient light, and the like. The existing gesture recognition technology mainly relies on a gesture recognition model, and a high-quality and high-precision gesture recognition model is a basis for realizing high-quality interaction of the application scene, however, the training of the gesture recognition model is not separated from the accurate gesture data set labeling.
At present, a scheme for acquiring data of specific gestures through a motion capture system to obtain a gesture data set is designed, but the efficiency of continuously executing each specific gesture and acquiring is low, and a scheme for intercepting a plurality of gesture fragment data from natural and coherent gesture motion data and improving the acquisition efficiency is also designed. However, the current gesture fragment data is rough to extract, and it is difficult to extract complete and effective gesture fragment data, so that the accuracy of the collected gesture data set is limited.
Disclosure of Invention
The embodiment of the application provides a data generation method, a device, electronic equipment and a computer readable storage medium, which can improve the effectiveness and the integrity of extracted gesture fragment data and further improve the precision of an acquired gesture data set.
In a first aspect, an embodiment of the present application provides a data generating method, including:
acquiring gesture motion data;
identifying at least one target peak from the gesture motion data;
Extracting at least one gesture fragment data from the gesture motion data according to each target peak value;
labeling the gesture fragment data to obtain a target gesture data set.
In a second aspect, an embodiment of the present application further provides a data generating apparatus, where the apparatus includes:
The acquisition module is used for acquiring gesture motion data;
The identification module is used for identifying at least one target peak value from the gesture motion data;
The extraction module is used for extracting at least one gesture fragment data from the gesture motion data according to each target peak value;
And the labeling module is used for labeling each gesture fragment data to obtain a target gesture data set.
Optionally, in some embodiments of the present application, the identifying at least one target peak from the gesture motion data includes:
Identifying a local peak of the gesture motion data over at least one time window;
and screening at least one target peak value from the local peaks according to a peak value threshold.
Optionally, in some embodiments of the application, the gesture motion data includes acceleration data;
The identifying the local peak of the gesture motion data over at least one time window comprises:
For each time window, deriving the analog length data corresponding to the acceleration data in the time window to obtain a derivative data set, wherein the derivative data set comprises derivative data corresponding to the analog length data of at least one time stamp in the time window;
taking the derivative data which is larger than zero in the derivative data set as candidate peak values;
and screening the maximum value from the candidate peaks to obtain the local peak value.
Optionally, in some embodiments of the present application, the selecting a maximum value from the candidate peaks to obtain the local peak includes:
comparing adjacent candidate peaks according to the sequence of the time stamps, and comparing the larger candidate peak with the next candidate peak until the comparison of the last candidate peak in the time window is finished, so as to obtain the largest candidate peak;
the largest candidate peak is set as the local peak.
Optionally, in some embodiments of the present application, the selecting a maximum value from the candidate peaks to obtain the local peak includes:
sequentially adding each candidate peak to a peak stack in the order of time stamps, and
If the current candidate peak value is larger than the previous candidate peak value, removing the previous candidate peak value from the peak value stack, and reserving the current candidate peak value;
If the current candidate peak is smaller than the previous candidate peak, retaining the current candidate peak and the previous candidate peak, and if the next candidate peak is larger than the previous candidate peak and the next candidate peak is larger than the current candidate peak, removing the current candidate peak and the previous candidate peak, and retaining the next candidate peak;
and after the last candidate peak value is added to the peak value stack in the time window and the sizes are compared, setting the candidate peak value reserved in the peak value stack or the candidate peak value at the stack top of the peak value stack as the local peak value, wherein the sizes of the candidate peak values of the peak value stack from the stack top to the stack tail are sequentially decreased.
Optionally, in some embodiments of the present application, the extracting at least one gesture segment data from the gesture motion data according to each target peak value includes:
And extracting gesture fragment data which takes the target peak value as a center and has a preset duration from the gesture motion data aiming at each target peak value.
Optionally, in some embodiments of the application, the method is applied to an augmented reality device;
labeling each gesture fragment data to obtain a target gesture data set, including:
Labeling each gesture fragment data through a pre-deployment model to obtain an original labeling result of each gesture fragment data;
Displaying the original labeling result through a virtual display screen;
obtaining a preliminary verification result in response to receiving first modification information input for the original labeling result;
the gesture fragment data, the gesture video data and the preliminary verification result are displayed in an aligned mode, and the gesture video data are synchronously shot aiming at the gesture motion data;
responding to the received second modification information input for the preliminary verification result, and obtaining a target verification result;
and generating the target gesture data set according to the target verification result.
In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program when executed by the processor implements the steps in the data generating method described above.
In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps in the data generating method described above.
In a fifth aspect, embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the methods provided in the various alternative implementations described in the embodiments of the present application.
In summary, the embodiment of the application acquires gesture motion data, identifies at least one target peak value from the gesture motion data, extracts at least one gesture fragment data from the gesture motion data according to each target peak value, and marks each gesture fragment data to obtain a target gesture data set.
The method comprises the steps of extracting a target peak value from gesture motion data, extracting gesture fragment data from the gesture motion data based on the target peak value, and improving the effectiveness and accuracy of the gesture fragment data, so that after a target gesture data set is obtained by labeling based on the gesture fragment data, the accuracy of the target gesture data set is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of a scene in which an augmented reality device according to an embodiment of the present application executes the data generation method;
FIG. 2 is a schematic flow chart of a data generating method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a data generating device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Reference numerals illustrate:
101-augmented reality device, 102-smart watch, 301-acquisition module, 302-recognition module, 303-extraction module, 304-labeling module, 401-processor, 402-memory, 403-power supply, 404-input unit.
Detailed Description
The following description of the embodiments of the present application will be made more apparent and fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the application are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
The embodiment of the application provides a data generation method, a data generation device, electronic equipment and a computer readable storage medium. Specifically, the embodiment of the application provides a data generating device suitable for an electronic device, wherein the electronic device comprises a terminal device or a server, the terminal device comprises a mobile phone, a tablet computer, a notebook computer, a smart watch, a smart bracelet or an extended reality device, the extended reality device comprises a Virtual Reality (VR) device, an Augmented Reality (AR) device, a Mixed Reality (MR) device and the like, and the extended reality device at least comprises a product in the form of glasses or a wearable electronic device in the form of a head-mounted display and the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server or the like for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs, content Delivery Network), basic cloud computing services such as big data and artificial intelligent platforms, and the servers may be directly or indirectly connected through wired or wireless communication modes.
For example, referring to fig. 1, taking an example that an augmented reality device executes the data generating method as an example, fig. 1 is a schematic view of a scene in which the augmented reality device executes the data generating method, and specifically, a specific execution procedure of the augmented reality device executing the data generating method is as follows:
The method comprises the steps that the augmented reality device 101 obtains gesture motion data, wherein the gesture motion data are acquired by the intelligent watch 102 through a built-in IMU sensor, the intelligent watch 102 sends the acquired gesture motion data to the augmented reality device 101, then, after the gesture motion data are received by the augmented reality device 101, at least one target peak value is identified from the gesture motion data, at least one gesture fragment data are extracted from the gesture motion data according to the target peak value, and the gesture fragment data are marked, so that a target gesture data set is obtained.
For example, the gesture segment data is intercepted from the gesture motion data by taking acceleration peak value, speed peak value, angular speed peak value and the like of the gesture motion as centers, so that complete and effective gesture segment data is obtained.
In summary, the embodiment of the application extracts the target peak value from the gesture motion data, and extracts the gesture fragment data from the gesture motion data based on the target peak value, so that the validity and the accuracy of the gesture fragment data are improved, and further, after the target gesture data set is obtained by labeling based on the gesture fragment data, the accuracy of the target gesture data set is also improved.
The following will describe in detail. It should be noted that the following description order of embodiments is not a limitation of the priority order of embodiments.
Referring to fig. 2, fig. 2 is a flowchart of a data generating method according to an embodiment of the present application, in which although a logic sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than shown in the flowchart. Specifically, the specific flow of the data generation method is as follows:
201. gesture motion data is acquired.
The gesture motion data are acquired through an IMU sensor, for example, the gesture motion data are acquired through a smart watch and a smart bracelet which are internally provided with the IMU sensor.
Accordingly, in the embodiment of the application, the gesture motion data can be obtained by receiving the smart watch or the smart bracelet.
202. At least one target peak is identified from the gesture motion data.
The peak value refers to a local maximum value (also referred to as a maximum point) of a certain physical quantity in time or space, and for example, for gesture motion data, the peak value includes a peak value based on acceleration, a peak value of motion speed, a peak value of angular speed, and the like. The target peak is a specific number of peaks selected from the plurality of peaks. It will be appreciated that the target peaks are the result of screening, filtering, or both, of the peaks. For example, filtering out invalid peaks based on a peak threshold value results in a number of target peaks that can characterize the gesture motion, e.g., target peaks include peaks of acceleration, velocity, or magnitude of angular velocity at the vertex at the time of gesture motion, etc. It will be appreciated that each target peak corresponds to an effective gesture motion.
203. And extracting at least one gesture fragment data from the gesture motion data according to each target peak value.
It can be appreciated that, since the target peak is a valid peak after filtering, the validity and accuracy of the gesture fragment data extracted based on the target peak are ensured. For example, gesture segment data including peak values of each target are extracted from gesture motion data, so that each gesture segment data corresponds to a valid gesture motion.
It can be understood that the extraction of the gesture fragment data is performed by taking the target peak value as a reference, so that the accuracy of gesture fragment data extraction is improved, and the extraction of the gesture fragment data is facilitated to be improved to complete and effective gesture fragment data.
204. Labeling the gesture fragment data to obtain a target gesture data set.
The labeling refers to labeling of gesture types corresponding to each gesture segment data, for example, labeling gesture types such as finger click, finger movement, palm detection or palm stretching to which each gesture segment data belongs.
It can be understood that after labeling each piece of gesture segment data, gesture segment data labeled with a gesture type is obtained, and the data can be used as a target gesture data set for training a gesture recognition model, so that the trained gesture recognition model can learn the relationship between the gesture segment data and the gesture type, and further can recognize and classify various pieces of gesture segment data, analyze the gesture type corresponding to the gesture segment data, and realize the function of gesture recognition.
In summary, the embodiment of the application extracts the target peak value from the gesture motion data, and extracts the gesture fragment data from the gesture motion data based on the target peak value, so that the validity and the accuracy of the gesture fragment data are improved, and further, after the target gesture data set is obtained by labeling based on the gesture fragment data, the accuracy of the target gesture data set is also improved.
Optionally, in an embodiment of the present application, in order to further improve the validity and accuracy of the collected gesture fragment data, in an embodiment of the present application, the extracting of the gesture fragment data is controlled by taking the target peak value as a center according to a preset duration, that is, optionally, in some embodiments of the present application, the step of "extracting at least one gesture fragment data from the gesture motion data according to each target peak value" includes:
And extracting gesture fragment data which takes the target peak value as a center and has a preset duration from the gesture motion data aiming at each target peak value.
For example, for each target peak, data segments with time lengths of 500ms before and 500ms after the target peak are extracted from gesture motion data by taking the target peak as a center, and gesture segment data with time length of about 1s are obtained by combining.
It can be understood that the gesture fragment data is obtained by combining the front and rear intercepting part data fragments by taking the target peak value as the center, so that the gesture fragment data taking the gesture action as the center can be obtained, and the effectiveness and the accuracy of the gesture fragment data can be improved.
Optionally, in embodiments of the present application, local peaks of each time window may be extracted according to the time window, and an effective target peak may be obtained by means of peak threshold filtering, that is, optionally, in some embodiments of the present application, the step of "identifying at least one target peak from the gesture motion data" includes:
Identifying a local peak of the gesture motion data over at least one time window;
and screening at least one target peak value from the local peaks according to a peak value threshold.
The local peak value is the largest peak value in a plurality of time stamps corresponding to each time window, and belongs to the local maximum value. Wherein the length of the time window may be set based on the general duration of the gesture, e.g., the size of the time window is set according to the general duration of the gesture.
The peak threshold is set based on the peak values of the gestures, for example, the peak values of a plurality of gestures are collected, and the minimum peak value corresponding to each gesture is set as the peak threshold. It will be appreciated that the peak threshold is set to filter out invalid local peaks to obtain valid target peaks corresponding to a certain gesture. For example, filtering local peaks corresponding to common general gestures and general gesture actions which do not contain specific instructions can avoid obtaining invalid gesture fragment data based on invalid local peak interception, and the validity and accuracy of the gesture fragment data acquired by the scheme are improved.
For example, taking the peak for acceleration as an example, i.e. the gesture motion data comprises acceleration data, the step of "identifying a local peak of the gesture motion data over at least one time window" comprises:
For each time window, deriving the analog length data corresponding to the acceleration data in the time window to obtain a derivative data set, wherein the derivative data set comprises derivative data corresponding to the analog length data of at least one time stamp in the time window;
taking the derivative data which is larger than zero in the derivative data set as candidate peak values;
and screening the maximum value from the candidate peaks to obtain the local peak value.
For example, the derivative of the analog length data or the analog length signal corresponding to the acceleration data is performed according to the time window, and the derivative data of the zero crossing point is set as the candidate peak value. And screening the maximum value from the plurality of candidate peaks to be used as the maximum peak value corresponding to the time window, namely the local peak value.
Optionally, in the embodiment of the present application, a local peak value with a largest value may be screened out by comparing candidate peak values, that is, step "screening out a largest value from the candidate peak values to obtain the local peak value" includes:
comparing adjacent candidate peaks according to the sequence of the time stamps, and comparing the larger candidate peak with the next candidate peak until the comparison of the last candidate peak in the time window is finished, so as to obtain the largest candidate peak;
the largest candidate peak is set as the local peak.
For example, each candidate peak is compared in the order of the time stamps, and the larger candidate peak in the comparison result is compared with the next candidate peak in the order, so as to obtain the maximum value of the plurality of candidate peaks.
Optionally, in the embodiment of the present application, a local peak value with the largest value may also be selected from a plurality of candidate peak values by using a principle of a monotonic stack, that is, optionally, in some embodiments of the present application, the step of "selecting a maximum value from the candidate peak values to obtain the local peak value" includes:
sequentially adding each candidate peak to a peak stack in the order of time stamps, and
If the current candidate peak value is larger than the previous candidate peak value, removing the previous candidate peak value from the peak value stack, and reserving the current candidate peak value;
If the current candidate peak is smaller than the previous candidate peak, retaining the current candidate peak and the previous candidate peak, and if the next candidate peak is larger than the previous candidate peak and the next candidate peak is larger than the current candidate peak, removing the current candidate peak and the previous candidate peak, and retaining the next candidate peak;
and after the last candidate peak value is added to the peak value stack in the time window and the sizes are compared, setting the candidate peak value reserved in the peak value stack or the candidate peak value at the stack top of the peak value stack as the local peak value, wherein the sizes of the candidate peak values of the peak value stack from the stack top to the stack tail are sequentially decreased.
The peak stack has a monotonic attribute, and the element in the stack is always kept to be monotonically increased or monotonically decreased.
For example, in the embodiment of the present application, each candidate peak is sequentially added to a peak stack according to the order of the time stamps, and the added candidate peak and the candidate peak which is previously added and reserved are compared in size by using the peak stack, so that a larger candidate peak is always reserved in the stack by removing the smaller candidate peak to the maximum, and further, after each candidate peak is added to the peak stack and compared, the local peak with the maximum value is obtained.
For example, taking local peaks of 0.10, 0.15, 0.20, 0.18 and 0.22 respectively and corresponding time stamps of 1.0s, 1.2s, 1.3s, 1.5s and 1.7s respectively as examples, the processing steps of the peak stack specifically include:
The first step, 0.10 candidate peak value is received, so that 0.10 candidate peak value is pushed into the stack;
Step two, receiving candidate peak value 0.15, comparing candidate peak value 0.15 with candidate peak value 0.10, removing candidate peak value 0.10 and retaining candidate peak value 0.15 or pushing candidate peak value 0.15 into stack because of 0.15> 0.10;
Step three, receiving candidate peak value 0.20, comparing candidate peak value 0.20 with candidate peak value 0.15, removing candidate peak value 0.15 and retaining candidate peak value 0.20 or pushing candidate peak value 0.20 into stack because of 0.20> 0.15;
fourthly, receiving a candidate peak value 0.18, and comparing the candidate peak value 0.20 with the candidate peak value 0.18, wherein the candidate peak value 0.18 is 0.18<0.20, and the candidate peak value 0.18 is the newly added candidate peak value, and then the candidate peak value 0.18 is put into a stack, and the candidate peak value 0.20 and the candidate peak value 0.18 are reserved in the stack;
Fifth, candidate peak 0.22 is received, candidate peak 0.22 and candidate peaks 0.20 and 0.18 are compared, and due to 0.22>0.20, and 0.22>0.18, candidate peak 0.20 is removed and candidate peak 0.18 is removed, and candidate peak 0.22 is reserved, or candidate peak 0.22 is pushed onto the stack.
After the processing, only the candidate peak value 0.22 with the largest finger is reserved in the peak stack, and the candidate peak value 0.22 is taken as the current local peak value. Correspondingly, the corresponding time stamp is 1.7s.
For another example, if the target peak values are respectively 0.22 (corresponding time stamp is 1.7 s), 0.25 (corresponding time stamp is 3.2 s), 0.26 (corresponding time stamp is 5.4 s) and 0.27 (corresponding time stamp is 6.9 s) after filtering according to the peak value threshold value, four gesture fragment data with the duration of about 1s are obtained by intercepting from gesture motion data with the time stamp of 1.7s, 3.2s, 5.4s and 6.9s as the center respectively, and correspondingly, after marking each gesture fragment data, a gesture data set containing four marked gesture data sets is obtained. And further can be used for training a gesture recognition model.
Optionally, in the embodiment of the present application, after the gesture fragment data is obtained, gesture prediction reasoning may be performed by using a local lightweight model preferentially to obtain an original labeling result of the gesture fragment data, and then manually verified labeling data is obtained by combining with a manual confirmation mode, that is, optionally, in some embodiments of the present application, the step of labeling each gesture fragment data to obtain a target gesture data set includes:
Labeling each gesture fragment data through a pre-deployment model to obtain an original labeling result of each gesture fragment data;
Displaying the original labeling result through a virtual display screen;
obtaining a preliminary verification result in response to receiving first modification information input for the original labeling result;
the gesture fragment data, the gesture video data and the preliminary verification result are displayed in an aligned mode, and the gesture video data are synchronously shot aiming at the gesture motion data;
responding to the received second modification information input for the preliminary verification result, and obtaining a target verification result;
and generating the target gesture data set according to the target verification result.
It should be noted that the pre-deployment model includes a local lightweight model, including a pre-deployed gesture recognition model, where the pre-deployment model may be the same or different from a gesture recognition model to be trained by the target gesture data set. For example, when the pre-deployment model is different from the gesture recognition model to be trained, reasoning is achieved by using other models, the marking of gesture data is quickened by combining a manual confirmation mode, and further after the marked gesture data is obtained through manual calibration, the gesture recognition model to be trained can be trained. For another example, the pre-deployment model, that is, the gesture recognition model to be trained, performs preliminary reasoning based on the general reasoning capability of the pre-deployment model, and performs labeling in combination with manual, so that the workload of manual labeling can be reduced as well, and further, after labeled gesture data is obtained after labeling, the pre-deployment model can be trained so as to improve the reasoning capability of the pre-deployment model.
It may be understood that in the embodiment of the present application, the labeling result may be calibrated by combining the video data of the gesture, for example, when the hand performs the gesture to generate the gesture motion data, the gesture video data is synchronously captured by using the camera, and accordingly, the gesture video data and the gesture fragment data may be displayed in alignment, so that the user calibrates the gesture fragment data and the labeling result corresponding to the gesture fragment data through the displayed gesture video data.
For example, the original labeling result labeled by the pre-deployment model is displayed through a virtual display screen of the augmented reality device, and modification confirmation of the original labeling result can be manually realized through gesture operation, a matched ring or other tools or based on eye movement data, so that a preliminary verification result is obtained. Furthermore, the gesture video data and the gesture fragment data can be displayed through the virtual display screen of the augmented reality device in an aligned mode, so that a user browses the gesture video data to recalibrate the gesture fragment data and the preliminary verification result corresponding to the gesture fragment data, and a more accurate target verification result is obtained.
It can be understood that the data generation method of the embodiment of the application is suitable for watches, mobile phones, computers, and augmented reality devices or data generation systems composed of watches, mobile phones, computers, or watches, augmented reality devices, and computers.
For example, taking a data generation system consisting of a watch, a mobile phone and a computer as an example, the mobile phone is connected with the watch through the BLE, a user issues an instruction for starting to collect data through the mobile phone, the mobile phone controls the watch through the BLE to collect gesture motion data of the hand of the user based on an IMU sensor, the mobile phone also shoots simultaneously to obtain gesture video data of the hand motion, the watch can directly obtain local peaks of all time windows through peak detection, further obtain target peaks through peak stacks, obtain a plurality of gesture fragment data from the gesture motion data according to all the target peaks, then the watch invokes a pre-deployment model to identify original labeling results corresponding to all the gesture fragment data, and the watch sends the gesture fragment data, the gesture motion data and the original labeling results to the mobile phone.
After the mobile phone receives the gesture fragment data, the gesture motion data and the original labeling result sent by the watch, the original labeling result corresponding to the gesture fragment data is displayed in the mobile phone interface, and a user can confirm or modify the original labeling result through the mobile phone interface to obtain a primary verification result confirmed by a person for the first time.
Furthermore, the mobile phone can send the gesture fragment data, the gesture motion data, the original labeling result and the gesture video data to a computer (wherein the computer can also upload the cloud end first and download the gesture video data from the cloud end), the computer can display the gesture video data and the gesture fragment data in an aligned manner through an ELAN tool, and a user can check the initial checking result of the gesture fragment data again through the ELAN tool and by utilizing the gesture video data to obtain a target checking result. And then, obtaining the marked target gesture data set based on the target verification result.
In summary, the embodiment of the application extracts the target peak value from the gesture motion data, and extracts the gesture fragment data from the gesture motion data based on the target peak value, so that the validity and the accuracy of the gesture fragment data are improved, and further, after the target gesture data set is obtained by labeling based on the gesture fragment data, the accuracy of the target gesture data set is also improved.
And, by screening the target peak value based on the monotonic peak value stack, the screening efficiency of the target peak value is improved.
And extracting gesture fragment data by taking the target peak value as the center according to the preset time length, so that the extracted gesture fragment data are more effective and complete.
In order to facilitate better implementation of the data generation method, the application also provides a data generation device based on the data generation method. Wherein the meaning of nouns is the same as in the data generation method described above, specific implementation details may be referred to the description in the method embodiment.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a data generating device according to an embodiment of the present application, where the data generating device may specifically be as follows:
an acquiring module 301, configured to acquire gesture motion data;
An identification module 302, configured to identify at least one target peak from the gesture motion data;
an extracting module 303, configured to extract at least one gesture segment data from the gesture motion data according to each target peak value;
and the labeling module 304 is configured to label each gesture segment data to obtain a target gesture data set.
Optionally, in some embodiments of the present application, the identifying at least one target peak from the gesture motion data includes:
Identifying a local peak of the gesture motion data over at least one time window;
and screening at least one target peak value from the local peaks according to a peak value threshold.
Optionally, in some embodiments of the application, the gesture motion data includes acceleration data;
The identifying the local peak of the gesture motion data over at least one time window comprises:
For each time window, deriving the analog length data corresponding to the acceleration data in the time window to obtain a derivative data set, wherein the derivative data set comprises derivative data corresponding to the analog length data of at least one time stamp in the time window;
taking the derivative data which is larger than zero in the derivative data set as candidate peak values;
and screening the maximum value from the candidate peaks to obtain the local peak value.
Optionally, in some embodiments of the present application, the selecting a maximum value from the candidate peaks to obtain the local peak includes:
comparing adjacent candidate peaks according to the sequence of the time stamps, and comparing the larger candidate peak with the next candidate peak until the comparison of the last candidate peak in the time window is finished, so as to obtain the largest candidate peak;
the largest candidate peak is set as the local peak.
Optionally, in some embodiments of the present application, the selecting a maximum value from the candidate peaks to obtain the local peak includes:
sequentially adding each candidate peak to a peak stack in the order of time stamps, and
If the current candidate peak value is larger than the previous candidate peak value, removing the previous candidate peak value from the peak value stack, and reserving the current candidate peak value;
If the current candidate peak is smaller than the previous candidate peak, retaining the current candidate peak and the previous candidate peak, and if the next candidate peak is larger than the previous candidate peak and the next candidate peak is larger than the current candidate peak, removing the current candidate peak and the previous candidate peak, and retaining the next candidate peak;
and after the last candidate peak value is added to the peak value stack in the time window and the sizes are compared, setting the candidate peak value reserved in the peak value stack or the candidate peak value at the stack top of the peak value stack as the local peak value, wherein the sizes of the candidate peak values of the peak value stack from the stack top to the stack tail are sequentially decreased.
Optionally, in some embodiments of the present application, the extracting at least one gesture segment data from the gesture motion data according to each target peak value includes:
And extracting gesture fragment data which takes the target peak value as a center and has a preset duration from the gesture motion data aiming at each target peak value.
Optionally, in some embodiments of the application, the method is applied to an augmented reality device;
labeling each gesture fragment data to obtain a target gesture data set, including:
Labeling each gesture fragment data through a pre-deployment model to obtain an original labeling result of each gesture fragment data;
Displaying the original labeling result through a virtual display screen;
obtaining a preliminary verification result in response to receiving first modification information input for the original labeling result;
the gesture fragment data, the gesture video data and the preliminary verification result are displayed in an aligned mode, and the gesture video data are synchronously shot aiming at the gesture motion data;
responding to the received second modification information input for the preliminary verification result, and obtaining a target verification result;
and generating the target gesture data set according to the target verification result.
In the embodiment of the application, gesture motion data is firstly acquired by an acquisition module 301, at least one target peak value is identified from the gesture motion data by an identification module 302, at least one gesture fragment data is extracted from the gesture motion data by an extraction module 303 according to each target peak value, and each gesture fragment data is marked by a marking module 304, so that a target gesture data set is obtained.
According to the embodiment of the application, the target peak value is extracted from the gesture motion data, the gesture fragment data is extracted from the gesture motion data based on the target peak value, the effectiveness and the accuracy of the gesture fragment data are improved, and the accuracy of the target gesture data set is improved after the target gesture data set is obtained by labeling based on the gesture fragment data.
In addition, the application further provides an electronic device, as shown in fig. 4, which shows a schematic structural diagram of the electronic device provided by the embodiment of the application, specifically:
The electronic device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. Those skilled in the art will appreciate that the electronic device structure shown in fig. 4 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components. Wherein:
The processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device. Optionally, the processor 401 may include one or more processing cores, and preferably the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), etc., and a storage data area that may store data created according to the use of the electronic device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
The electronic device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power device commissioning circuit, a power converter or inverter, a power status indicator, etc.
The electronic device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.
Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In this embodiment, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement the steps in any data generating method provided in the embodiment of the present application.
According to the embodiment of the application, gesture motion data are acquired, at least one target peak value is identified from the gesture motion data, at least one gesture fragment data is extracted from the gesture motion data according to each target peak value, each gesture fragment data is marked, and a target gesture data set is obtained.
The method comprises the steps of extracting a target peak value from gesture motion data, extracting gesture fragment data from the gesture motion data based on the target peak value, and improving the effectiveness and accuracy of the gesture fragment data, so that after a target gesture data set is obtained by labeling based on the gesture fragment data, the accuracy of the target gesture data set is improved.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, the present application provides a computer readable storage medium having stored thereon a computer program that can be loaded by a processor to perform the steps of any of the data generation methods provided by the present application.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
The computer readable storage medium may include, among others, read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disks, and the like.
Because the instructions stored in the computer readable storage medium can execute the steps in any data generating method provided by the present application, the beneficial effects that any data generating method provided by the present application can achieve can be achieved, and detailed descriptions are omitted herein.
The foregoing has outlined rather broadly the principles and embodiments of the present application in order that the detailed description of the application may be better understood, and in order that the present application may be better understood, the present application should not be construed as being limited to the details of the embodiments and applications of the present application.
Claims (7)
1. A method of data generation, the method comprising:
acquiring gesture motion data;
identifying at least one target peak from the gesture motion data;
Extracting at least one gesture fragment data from the gesture motion data according to each target peak value;
labeling each gesture fragment data to obtain a target gesture data set;
The gesture motion data includes acceleration data, and the identifying at least one target peak from the gesture motion data includes:
For each time window, deriving the analog length data corresponding to the acceleration data in the time window to obtain a derivative data set, wherein the derivative data set comprises derivative data corresponding to the analog length data of at least one time stamp in the time window;
taking the derivative data which is larger than zero in the derivative data set as candidate peak values;
Screening the maximum value from the candidate peak values to obtain a local peak value;
Screening at least one target peak value from the local peak values according to a peak value threshold value;
The step of screening the maximum value from the candidate peak values to obtain a local peak value comprises the following steps:
and screening the local peak value with the largest value from the candidate peak values by utilizing the principle of a monotonic stack.
2. The method of claim 1, wherein said selecting a maximum value from each of said candidate peaks to obtain said local peak comprises:
sequentially adding each candidate peak to a peak stack in the order of time stamps, and
If the current candidate peak value is larger than the previous candidate peak value, removing the previous candidate peak value from the peak value stack, and reserving the current candidate peak value;
If the current candidate peak is smaller than the previous candidate peak, retaining the current candidate peak and the previous candidate peak, and if the next candidate peak is larger than the previous candidate peak and the next candidate peak is larger than the current candidate peak, removing the current candidate peak and the previous candidate peak, and retaining the next candidate peak;
and after the last candidate peak value is added to the peak value stack in the time window and the sizes are compared, setting the candidate peak value reserved in the peak value stack or the candidate peak value at the stack top of the peak value stack as the local peak value, wherein the sizes of the candidate peak values of the peak value stack from the stack top to the stack tail are sequentially decreased.
3. The method according to claim 1, wherein the extracting at least one gesture segment data from the gesture motion data according to each of the target peaks comprises:
And extracting gesture fragment data which takes the target peak value as a center and has a preset duration from the gesture motion data aiming at each target peak value.
4. The data generation method according to claim 1, wherein the method is applied to an augmented reality device;
labeling each gesture fragment data to obtain a target gesture data set, including:
Labeling each gesture fragment data through a pre-deployment model to obtain an original labeling result of each gesture fragment data;
Displaying the original labeling result through a virtual display screen;
obtaining a preliminary verification result in response to receiving first modification information input for the original labeling result;
the gesture fragment data, the gesture video data and the preliminary verification result are displayed in an aligned mode, and the gesture video data are synchronously shot aiming at the gesture motion data;
responding to the received second modification information input for the preliminary verification result, and obtaining a target verification result;
and generating the target gesture data set according to the target verification result.
5. A data generation apparatus, the apparatus comprising:
The acquisition module is used for acquiring gesture motion data;
The identification module is used for identifying at least one target peak value from the gesture motion data;
The extraction module is used for extracting at least one gesture fragment data from the gesture motion data according to each target peak value;
the labeling module is used for labeling each gesture fragment data to obtain a target gesture data set;
The gesture motion data includes acceleration data, and the identifying at least one target peak from the gesture motion data includes:
For each time window, deriving the analog length data corresponding to the acceleration data in the time window to obtain a derivative data set, wherein the derivative data set comprises derivative data corresponding to the analog length data of at least one time stamp in the time window;
taking the derivative data which is larger than zero in the derivative data set as candidate peak values;
Screening the maximum value from the candidate peak values to obtain a local peak value;
Screening at least one target peak value from the local peak values according to a peak value threshold value;
The step of screening the maximum value from the candidate peak values to obtain a local peak value comprises the following steps:
and screening the local peak value with the largest value from the candidate peak values by utilizing the principle of a monotonic stack.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the data generation method of any of claims 1-4 when the computer program is executed.
7. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps in the data generation method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511101418.6A CN120612735B (en) | 2025-08-07 | 2025-08-07 | Data generation methods, apparatus, electronic devices and computer-readable storage media |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511101418.6A CN120612735B (en) | 2025-08-07 | 2025-08-07 | Data generation methods, apparatus, electronic devices and computer-readable storage media |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN120612735A CN120612735A (en) | 2025-09-09 |
| CN120612735B true CN120612735B (en) | 2025-11-21 |
Family
ID=96930935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202511101418.6A Active CN120612735B (en) | 2025-08-07 | 2025-08-07 | Data generation methods, apparatus, electronic devices and computer-readable storage media |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120612735B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035785A (en) * | 2024-02-05 | 2024-05-14 | 歌尔科技有限公司 | Data generation method, device, equipment and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140168057A1 (en) * | 2012-12-13 | 2014-06-19 | Qualcomm Incorporated | Gyro aided tap gesture detection |
| US20160078287A1 (en) * | 2014-08-29 | 2016-03-17 | Konica Minola Laboratory U.S.A., Inc. | Method and system of temporal segmentation for gesture analysis |
| CN117908663A (en) * | 2023-12-14 | 2024-04-19 | 湖北星纪魅族科技有限公司 | Gesture recognition method and electronic device |
-
2025
- 2025-08-07 CN CN202511101418.6A patent/CN120612735B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118035785A (en) * | 2024-02-05 | 2024-05-14 | 歌尔科技有限公司 | Data generation method, device, equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN120612735A (en) | 2025-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111311756B (en) | Augmented reality AR display method and related device | |
| JP6986187B2 (en) | Person identification methods, devices, electronic devices, storage media, and programs | |
| CN109189879B (en) | Electronic book display method and device | |
| WO2016209517A1 (en) | Techniques to save or delete a video clip | |
| CN104239416A (en) | User identification method and system | |
| CN107368820B (en) | Refined gesture recognition method, device and equipment | |
| CN111027419B (en) | Method, device, equipment and medium for detecting video irrelevant content | |
| CN107526769A (en) | Photo processing method and device, equipment and computer-readable recording medium based on artificial intelligence | |
| CN110555102A (en) | media title recognition method, device and storage medium | |
| CN107193571A (en) | Interface push method, mobile terminal and storage medium | |
| US20250039537A1 (en) | Screenshot processing method, electronic device, and computer readable medium | |
| CN112215243B (en) | Image feature extraction method, device, equipment and storage medium | |
| WO2022111458A1 (en) | Image capture method and apparatus, electronic device, and storage medium | |
| CN112199534A (en) | Sticker recommendation method and device, electronic equipment and storage medium | |
| CN108052506B (en) | Natural language processing method, device, storage medium and electronic device | |
| CN108985215B (en) | A picture processing method, picture processing device and terminal equipment | |
| KR102040392B1 (en) | Method for providing augmented reality contents service based on cloud | |
| CN120612735B (en) | Data generation methods, apparatus, electronic devices and computer-readable storage media | |
| CN110929159A (en) | Resource delivery method, device, equipment and medium | |
| CN112150486B (en) | Image processing method and device | |
| CN103984415A (en) | Information processing method and electronic equipment | |
| CN112449249A (en) | Video stream processing method and device, electronic equipment and storage medium | |
| CN111797656A (en) | Face key point detection method and device, storage medium and electronic equipment | |
| CN114943872A (en) | Training method and device of target detection model, target detection method and device, medium and equipment | |
| CN114115530A (en) | Virtual object control method, device, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |