WO2009090584A2 - Method and system for activity recognition and its application in fall detection - Google Patents
Method and system for activity recognition and its application in fall detection Download PDFInfo
- Publication number
- WO2009090584A2 WO2009090584A2 PCT/IB2009/050093 IB2009050093W WO2009090584A2 WO 2009090584 A2 WO2009090584 A2 WO 2009090584A2 IB 2009050093 W IB2009050093 W IB 2009050093W WO 2009090584 A2 WO2009090584 A2 WO 2009090584A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- activity
- class classifier
- unknown
- feature vector
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
Definitions
- the invention relates to activity recognition, particularly to a method and apparatus for activity recognition and its application in fall detection.
- Non-wearable activity recognition may be generally subdivided into two classes: non-wearable and wearable activity recognition.
- non-wearable activity recognition user activities are primarily recognized by the movement mode of the objects related closely to the user's activities or by human-to-object interactions.
- wearable activity recognition the specific user activities are primarily recognized from signals collected by sensors worn on the user's body.
- the non-wearable mode primarily uses an indirect mode to recognize the user activities; and the wearable mode primarily uses a direct mode to recognize the activities of the user's body parts.
- the former is primarily used in analyzing the user's long-term activity characteristics and modes, such as living habits, and the latter is primarily used in monitoring the user's body activities, for example in monitoring the activities of the aged.
- the wearable sensor for wearable activity recognition may include a sound sensor, acceleration sensor, temperature sensor, and the like.
- the acceleration sensor is most widely used in wearable activity recognition because of the characteristics of the activity and the properties of the acceleration sensor.
- the validity of recognizing user activities by using the acceleration sensor has also been proved by a number of scientific experiments. For example, the researcher recognizes daily activities such as standing, walking, going up and down stairs, teeth brushing, etc., by using a plurality of sensors worn on the user's body, or monitors the processing flow of the workers in a wood-processing factory by using the wearable sound and acceleration sensors, or analyzes the characteristics of the actions of a freestyle swimmer by using the acceleration sensors worn on the wrist.
- the process may be simply divided into three steps: the first step is to collect training data for respective activities to be recognized and extract training samples for the respective activities from the training data; the second step is to train a multi-class classifier and establish a recognition model according to the above-mentioned training samples; and the third step is to extract test samples from the test data and input them into the multi-class classifier for classifying, whereupon the resulting output from the classifier represents the recognized activity class.
- the above activity recognition method has its inherent limitations in that such a classifying, algorithm-based recognition system is not scalable and may recognize only individual activities that have already been trained. It is possible that the system classifies newly occurring, i.e. unknown activities, as a certain known activity by mistake, so that the practical application of this activity recognition method and system is limited.
- the invention provides an activity recognition method comprising the steps of:
- the method By identifying an activity as a known or unknown activity and further training an identified unknown activity, the method reduces the probability of classifying a new activity as a known activity by mistake.
- the method further comprises a step of classifying the activity on the basis of the feature vector and a multi-class classifier to identify the activity group to which the known activity belongs when the activity is determined as a known activity. In this way, it is possible to identify to which known activity group an activity belongs; even it is in the overlapping area of the characteristic space of two or more activities.
- the at least one one-class classifier model comprises a combined one- class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups. This provides flexibility to classifying known or unknown activities.
- the step of training comprises the sub-steps of:
- the method By classifying and training unknown activities, the method provides scalability to conventional activity recognition methods and systems.
- the training step further comprises a sub-step of labeling the specific unknown activity group asknown activity group when the number of samples in the specific unknown activity group reaches a predetermined threshold. In this way, more new activities can be recognized.
- the training step further comprises a sub-step of setting up a new unknown activity group including the data segments and/or feature vector associated with the activity of the group as samples. In this way, a new activity can be recorded for training.
- the deriving step further comprises the sub-steps of:
- the method when the activity belongs to a known activity group, the method further comprises the steps of:
- the method improves and extends its capability of activity recognition.
- the method further comprises a step of performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity.
- the method further comprises steps of:
- an activity recognition system which comprises:
- a deriving unit for deriving a feature vector characterizing an activity from sensing data associated with the activity
- a first classifying unit for classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity;
- a training unit for training a temporary one-class classifier model relating to an unknown activity if the activity is determined as an unknown activity.
- the system when the activity is determined as a known activity, the system further comprising a second classifying unit for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
- system further comprises:
- a fall detection unit for performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity
- a user interface arranged for informing the user when the activity is identified as a fall, receiving an input from the user that negates or confirms that the activity is a fall, and generating a fall alarm when the activity is identified and/or confirmed by the user as a fall.
- Fig.1 shows a flowchart of the method of activity recognition in accordance with the invention.
- Fig. 2 shows a flowchart of how to process sensing data in accordance with the invention.
- Fig.3 shows a flowchart of how to train a temporary one-class classifier model in accordance with the invention.
- Fig.4 shows a flowchart of the activity recognition method extended with fall detection in accordance with the invention.
- Fig. 5 is a block diagram of the activity recognition system in accordance with the invention.
- Fig.6 is a block diagram of the activity recognition system extended with a fall detection function in accordance with the invention.
- Fig.l shows a flowchart of the method of activity recognition in accordance with the invention.
- the method comprises a step 120 of deriving a feature vector characterizing an activity from sensing data associated with the activity.
- the sensing data is any measurement data associated with an activity, for example, measurement data obtained from a three-dimensional acceleration sensor.
- the method further comprises a step 130 of classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity (135).
- the method further comprises a step 140 of training a temporary one-class classifier model relating to an unknown activity when the activity is determined as an unknown activity.
- the method when the activity is determined as a known activity, the method further comprises a step 150 of classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
- the method further comprises a step 160 of re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group; and a step 170 of re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
- Fig. 2 shows a flowchart of how to process sensing data in accordance with the invention.
- sensing data obtained from a three-dimensional acceleration sensor is taken as an example to explain how to process sensing data.
- the processing of sensing data comprises a step 210 of segmenting sensing data associated with the activity into a plurality of data segments.
- Activity data can be intercepted for each dimension from the three-dimensional acceleration sensor by using windows of 64 bits in length with a 32 bits of data overlap between adjacent windows. The 50% window overlap has proved to be a successful and efficient method in the existing related research.
- the window length of 64 bits corresponds to a time period of 2 seconds, which is generally longer than the cycles of most of the activities. Meanwhile, the window length of 64 bits also facilitates the Fast Fourier Transform (FFT) required for performing the extraction of characteristics.
- FFT Fast Fourier Transform
- the processing of sensing data further comprises a step 220 of extracting features from a plurality of data segments so as to form a feature vector characterizing the activity.
- the following 11 characteristics may be extracted: the mean, variance, energy, 4 amplitude statistical characteristics, and 4 shape statistical characteristics based on power spectral density (PSD).
- PSD power spectral density
- the correlation coefficients between any two dimensions among the three-dimensional data may be extracted, with 36 characteristics in total.
- the mean and variance characteristics are the average and standard variance of the data for the interception windows, and the latter is used for characterizing signal stability.
- the value of the energy characteristic is equal to the sum of the squares of the respective components of the window signals that are subjected to the Fast Fourier Transform.
- the power spectral density which is defined as the Fourier Transformation of the autocorrelation function of the signal, is used for describing the signal energy distribution over the frequency domain.
- the amplitude statistical characteristics and the shape statistical characteristics thereof may be extracted from the power spectral density.
- the 4 extracted amplitude statistic characteristics are as follows:
- the processing of sensing data further comprises a step 230 of normalizing the feature vector.
- the characteristic for each dimension is generally normalized by a Z-core normalizing method in order to remove the impact on the subsequent analysis resulting from the difference in value among the different characteristics.
- a feature vector X for characterizing the user activity to be tested can be formed by using all or part of the above characteristics.
- the processing of sensing data further comprises a step 240 of performing a principal components analysis on the normalized feature vector to reduce the dimension of the feature vector.
- the dimensions of the characteristic space may generally be lowered by a principal component analysis method in order to optimize the feature vector.
- This principal component analysis method which is a commonly used method in the field of dimensions reduction, maintains information on the characteristics to the greatest extent while reducing the vector dimensions by means of a linear transform matrix.
- the transform matrix consists of the feature vectors of the feature vector covariance matrix.
- the feature vector of the covariance matrix is represented by A 1
- the feature vector of the covariance matrix is represented by ⁇ t
- the transform matrix is built up by selecting the feature vectors corresponding to the greatest K feature vectors.
- K is
- the activity is then classified on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity in step 140.
- n one-class classifier models have been trained and established for each known activity according to the training samples of n known activities.
- the one-class classifier model determines that a border encompassing a target sample can differentiate whether a test sample, e.g, the activity to be tested that is represented by a feature vector, is known (target class) or unknown (exceptional class).
- an exclusion threshold s is generally set such that the border encompasses the target sample of 1 ⁇ s only, and the remaining $ target sample is considered the exceptional sample, which is held excluded outside the border.
- KNN K Nearest Neighbor
- SVDD Support Vector Data Description
- Gauss Gauss one-class classifier
- the set of training samples is the set of feature vectors w
- X 1 is a feature vector of a sample obtained in step 120
- N j s the number of the sample for training
- n is the length of the vector, i.e., the number of characteristics in the vector.
- the average vector ⁇ and the variance vector ⁇ can be calculated, i.e. the Gaussian distribution ⁇ v"' ⁇ ) of the class of the samples is obtained from the following equations:
- the distance vector " of the set of samples can be obtained by calculating the distance
- the distance threshold value th is taken as the K -th distance value, ⁇ L -I , , where ⁇ is the preset exclusive threshold
- the distance threshold value can be used to decide whether the category of a test sample is the same as that of the training sample.
- the activity to be tested belongs to the category of the activity to which the Gauss classifier relates.
- a test sample belongs to a known activity by using a combination of a plurality of one-class classifier models. Based on each class of classifier models among n one-class classifier models, it can be determined whether a test sample belongs to an activity or not.
- the final determining result i.e. whether a certain test sample belongs to a known activity or an unknown activity, is obtained through the decision obtained by merging the outputs of n one-class classifier models.
- a logic OR operation may be used to perform the decision merging.
- the merging rule is: a test sample belongs to a known activity if the test sample is determined as a known sample by one or more one-class classifier models; otherwise, the test sample belongs to an unknown activity.
- a one-class classifier relating to all of the known activity groups can be obtained, e.g., a combined one-class classifier, by using the samples in all of the known activity groups for training, that is, whether a test sample belongs to at least one known activity group or should be excluded from all known activity groups can be determined on the basis of the combined one-class classifier model.
- a test sample may be contained in two or more one-class classifier models. If the sample areas of respective activities do not overlap in the characteristic space, then, e.g., a test sample is contained, at most, within the border defined by a one-class classifier model. In such a situation, the test sample can be identified to belong to the known activity corresponding to this one-class classifier and no further classifying is needed.
- an activity recognition system When a test sample is in the overlap area of the characteristic space of two or more activities, the multi-class classifier is best suited to decide which activity it belongs to.
- An advantage of the method provided by the invention is that the activity recognition is performed by the at least one one-class classifier in combination with a multi-class classifier.
- the multi-class classifier is used for the distinction between two or more classes, and it requires that the training samples of each class are available.
- the multi-class classifier establishes a multi-class classifier model by using all training samples in each class so as to recognize which class a test sample is most likely to belong to.
- the multi-class classifier may be a Decision Tree (DT) algorithm, a K Nearest Neighbor (KNN) algorithm, or a Weighted Support Vector Machine (WSVM) algorithm.
- DT Decision Tree
- KNN K Nearest Neighbor
- WSVM Weighted Support Vector Machine
- Fig.3 shows a flowchart of how to train a temporary one-class classifier model in accordance with the invention.
- the training step 140 comprises a sub-step 310 of classifying the activity on the basis of the feature vector and at least one temporary one-class classifier obtained from training samples in any one of existing unknown activity groups so as to determine whether the activity belongs to any one of existing unknown activity groups (312).
- the training step 140 further comprises a sub-step 320 of adding the data segments and/or feature vector associated with the activity to the specific existing unknown activity group as samples when the activity belongs to a specific existing unknown activity group.
- the training step 140 further comprises a sub-step 330 of training the temporary one-class classifier relating to the existing unknown activity by the data segments and/or feature vector of the activity so as to update the temporary one-class classifier model.
- the training step 140 further comprises a sub-step 340 of labeling the specific unknown activity group as known activity group when the number of samples in the specific unknown activity group reaches a predetermined threshold (335). In an embodiment, when the activity does not belong to any existing unknown activity group, the training step 140 further comprises a sub-step 350 of setting up a new unknown activity group including the data segments and/or feature vector associated with the activity of the group as samples.
- the activity recognition method provided by the invention may be applied to various activity detection systems, for example to a fall system for reducing the occurrence of a false fall alarm. It is well known that falling is one of the factors which endanger the life of the aged, and it is quite useful for an aged person to wear a fall detector in case he or she is in need of help after falling.
- ADL Activities of Daily Living
- Fig. 4 is a flowchart for the activity recognition method extended with fall detection in accordance with the invention. It depicts how to apply the activity recognition method to fall detection according to the present invention.
- steps 120, 130, 140, 150 and 160, 170 are substantially the same or similar, but the method is extended with steps 410 to 440 for fall detection. In the following description, only the new steps are explained in detail.
- the method further comprises a step 410 of detecting a fall when the activity is determined by the one-class classifier model as an unknown activity in step 130, 135.
- the method goes to step 140 for training a temporary one-class classifier model relating to an unknown activity.
- the method further comprises a step 440 to issue a fall alarm if the activity is identified as a fall.
- the method further comprises steps of communicating with users when the activity is identified as a fall and before issuing the fall alarm.
- the method comprises a step 420 of informing the user that the activity is identified as a fall and a step 430 of receiving an input from the user that negates or confirms that the activity is a fall.
- the method goes to step 140 for training a temporary one-class classifier mode relating to an unknown activity. Otherwise, a fall alarm is issued.
- the activity recognition method as described in connection with Figs. 1 to 4 and fall detection may be implemented by means of software or hardware or a combination thereof.
- Fig. 5 is a block diagram of an activity recognition system in accordance with the invention.
- the activity recognition system comprises a deriving unit (520) for deriving a feature vector characterizing an activity from sensing data associated with the activity.
- the deriving unit executes the function of step 120.
- the activity recognition system further comprises a first classifying unit 530 for classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity.
- the first classifying unit 530 executes the function of step 130.
- the activity recognition system further comprises a training unit 540 for training a temporary one-class classifier model relating to an unknown activity if the activity is determined as an unknown activity or, otherwise, for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
- the training unit 540 executes the function of step 140.
- the activity recognition system further comprises a second classifying unit 550 for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs when the activity is determined as a known activity.
- the second classifying unit 550 executes the function of step 150.
- the at least one one-class classifier model comprises a combined one- class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups.
- the activity recognition system further comprises: a first unit 560 for re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group.
- the first unit 560 executes the function of step 160; and a second unit 570 for re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
- the second unit 570 executes the function of step 170.
- the activity recognition system further comprises a sensor unit 510 for obtaining data associated with an activity; a data storage unit 503 for storing the sensor data, feature vector, and other temporary data; and an internal data line 405 for connecting respective functional units in the system.
- Fig. 6 is a block diagram of the activity recognition system extended with a fall detection function in accordance with the invention.
- the system further comprises a fall detection unit 680 for performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity.
- the fall detection unit executes the function of step 410.
- the system further comprises a user interface 690 arranged for informing the user when the activity is identified as a fall, receiving an input from the user that negates or confirms that the activity is a fall, and generating a fall alarm when the activity is identified and/or confirmed by the user as a fall.
- the user interface executes the functions of steps of 420, 430, and 440.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to an activity recognition method, which comprises the steps of deriving (120) a feature vector characterizing an activity from sensing data associated with the activity; classifying (130) the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity; and training (140) a temporary one-class classifier model relating to an unknown activity when the activity is determined as an unknown activity. In an embodiment, the method further comprises a step (150) of classifying the activity on the basis of the feature vector and a multi-class classifier to identify the group to which the known activity belongs, when the activity is determined as a known activity. By identifying an activity as a known or unknown activity, and further training an identified unknown activity, the method reduces the probability of classifying a new activity as a known activity by mistake.
Description
METHOD AND SYSTEM FOR ACTIVITY RECOGNITION AND ITS APPLICATION IN
FALL DETECTION
FIELD OF THE INVENTION
The invention relates to activity recognition, particularly to a method and apparatus for activity recognition and its application in fall detection.
BACKGROUND OF THE INVENTION
The development of modern technology has led to research on a variety of Ambient Intelligence (AΙ)-based and Context Awareness-based applications, such as the Smart House, Health Care, and the like. In those fields of application, the core targets of service are the users with their different requirements. In order to provide the users with timely, smart and customized services, accurate recognition and definition of user activities is a critical issue in the application research.
Activity Recognition (AR) may be generally subdivided into two classes: non-wearable and wearable activity recognition. In non-wearable activity recognition, user activities are primarily recognized by the movement mode of the objects related closely to the user's activities or by human-to-object interactions. In wearable activity recognition, the specific user activities are primarily recognized from signals collected by sensors worn on the user's body. Generally, the non-wearable mode primarily uses an indirect mode to recognize the user activities; and the wearable mode primarily uses a direct mode to recognize the activities of the user's body parts. The former is primarily used in analyzing the user's long-term activity characteristics and modes, such as living habits, and the latter is primarily used in monitoring the user's body activities, for example in monitoring the activities of the aged.
The wearable sensor for wearable activity recognition may include a sound sensor, acceleration sensor, temperature sensor, and the like. The acceleration sensor is most widely used in wearable activity recognition because of the characteristics of the activity and the properties of the acceleration sensor. The validity of recognizing user activities by using the acceleration
sensor has also been proved by a number of scientific experiments. For example, the researcher recognizes daily activities such as standing, walking, going up and down stairs, teeth brushing, etc., by using a plurality of sensors worn on the user's body, or monitors the processing flow of the workers in a wood-processing factory by using the wearable sound and acceleration sensors, or analyzes the characteristics of the actions of a freestyle swimmer by using the acceleration sensors worn on the wrist.
As regards prior art wearable activity recognition based on accelerometers, most of the analyzing methods use the supervision learning method, i.e., handle the activity recognition as a classifying problem. The process may be simply divided into three steps: the first step is to collect training data for respective activities to be recognized and extract training samples for the respective activities from the training data; the second step is to train a multi-class classifier and establish a recognition model according to the above-mentioned training samples; and the third step is to extract test samples from the test data and input them into the multi-class classifier for classifying, whereupon the resulting output from the classifier represents the recognized activity class. The above activity recognition method, however, has its inherent limitations in that such a classifying, algorithm-based recognition system is not scalable and may recognize only individual activities that have already been trained. It is possible that the system classifies newly occurring, i.e. unknown activities, as a certain known activity by mistake, so that the practical application of this activity recognition method and system is limited.
Accordingly, it is necessary to develop an improved activity recognition method and system.
SUMMARY OF THE INVENTION
According to the first aspect of the invention, the invention provides an activity recognition method comprising the steps of:
- deriving a feature vector characterizing an activity from sensing data associated with the activity;
- classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known
activity or an unknown activity; and
- training a temporary one-class classifier model relating to an unknown activity when the activity is determined as an unknown activity.
By identifying an activity as a known or unknown activity and further training an identified unknown activity, the method reduces the probability of classifying a new activity as a known activity by mistake.
In an embodiment, the method further comprises a step of classifying the activity on the basis of the feature vector and a multi-class classifier to identify the activity group to which the known activity belongs when the activity is determined as a known activity. In this way, it is possible to identify to which known activity group an activity belongs; even it is in the overlapping area of the characteristic space of two or more activities.
In an embodiment, the at least one one-class classifier model comprises a combined one- class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups. This provides flexibility to classifying known or unknown activities.
In another embodiment, the step of training comprises the sub-steps of:
- classifying the activity on the basis of the feature vector and at least one temporary one- class classifier obtained from training samples in any one of existing unknown activity groups so as to determine whether the activity belongs to any one of existing unknown activity groups; and
- adding the data segments and/or feature vector associated with the activity to the specific existing unknown activity group as samples when the activity belongs to a specific existing unknown activity group; and
- training the temporary one-class classifier relating to the existing unknown activity by means of the data segments and/or feature vector of the activity so as to update the temporary one-class classifier model.
By classifying and training unknown activities, the method provides scalability to conventional activity recognition methods and systems.
In another embodiment, the training step further comprises a sub-step of labeling the specific unknown activity group asknown activity group when the number of samples in the specific unknown activity group reaches a predetermined threshold. In this way, more new
activities can be recognized.
In a further embodiment, when the activity does not belong to any existing unknown activity group, the training step further comprises a sub-step of setting up a new unknown activity group including the data segments and/or feature vector associated with the activity of the group as samples. In this way, a new activity can be recorded for training.
In a further embodiment, the deriving step further comprises the sub-steps of:
- segmenting sensing data associated with the activity into a plurality of data segments;
- extracting features from a plurality of data segments so as to form a feature vector characterizing the activity;
- normalizing the feature vector; and
- performing a principal components analysis on the normalized feature vector to reduce the dimension of the feature vector.
By reducing the dimension of the feature vector, the complexity of computation can be reduced dramatically.
In another embodiment, when the activity belongs to a known activity group, the method further comprises the steps of:
- re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group;
- re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
By retraining the combined one-class classifier model and multi-class classifier model using new identified known activities, the method improves and extends its capability of activity recognition.
In an embodiment, the method further comprises a step of performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity.
In a further embodiment, the method further comprises steps of:
- informing the user when the activity is identified as a fall;
- receiving an input from the user that negates or confirms that the activity is a fall; and
- generating a fall alarm when the activity is identified and/or confirmed by the user as a
fall.
By applying an activity recognition method to fall detection and using a user's input for activity recognition and training, the accuracy of activity recognition is improved, resulting in a reduction of false alarms of a fall.
According to the second aspect of the invention, the invention provides an activity recognition system, which comprises:
- a deriving unit for deriving a feature vector characterizing an activity from sensing data associated with the activity;
- a first classifying unit for classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity; and
- a training unit for training a temporary one-class classifier model relating to an unknown activity if the activity is determined as an unknown activity.
In an embodiment, when the activity is determined as a known activity, the system further comprising a second classifying unit for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
In another embodiment, the system further comprises:
- a fall detection unit for performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity; and
- a user interface arranged for informing the user when the activity is identified as a fall, receiving an input from the user that negates or confirms that the activity is a fall, and generating a fall alarm when the activity is identified and/or confirmed by the user as a fall.
Detailed explanations and other aspects of the invention are given below.
DESCRIPTION OF THE DRAWINGS
The above and other objects and features of the present invention will become more apparent from the following detailed description considered in connection with the accompanying drawings, in which:
Fig.1 shows a flowchart of the method of activity recognition in accordance with the invention.
Fig. 2 shows a flowchart of how to process sensing data in accordance with the invention.
Fig.3 shows a flowchart of how to train a temporary one-class classifier model in accordance with the invention.
Fig.4 shows a flowchart of the activity recognition method extended with fall detection in accordance with the invention.
Fig. 5 is a block diagram of the activity recognition system in accordance with the invention.
Fig.6 is a block diagram of the activity recognition system extended with a fall detection function in accordance with the invention.
The same reference numerals are used to denote similar parts throughout the figures.
DETAILED DESCRIPTION
Fig.l shows a flowchart of the method of activity recognition in accordance with the invention.
Referring to Fig.l, the method comprises a step 120 of deriving a feature vector characterizing an activity from sensing data associated with the activity. The sensing data is any measurement data associated with an activity, for example, measurement data obtained from a three-dimensional acceleration sensor.
The method further comprises a step 130 of classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity (135).
The method further comprises a step 140 of training a temporary one-class classifier model relating to an unknown activity when the activity is determined as an unknown activity.
In an embodiment, when the activity is determined as a known activity, the method further comprises a step 150 of classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
It is advantageous that, when the activity belongs to a known activity, the method further
comprises a step 160 of re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group; and a step 170 of re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
The above-mentioned steps 120, 130, 140, 150 will be explained in a detailed description and in combination with Fig.2 and Fig. 3 in the following.
Fig. 2 shows a flowchart of how to process sensing data in accordance with the invention.
In the following description, sensing data obtained from a three-dimensional acceleration sensor is taken as an example to explain how to process sensing data.
Referring to Fig.2, the processing of sensing data comprises a step 210 of segmenting sensing data associated with the activity into a plurality of data segments. Activity data can be intercepted for each dimension from the three-dimensional acceleration sensor by using windows of 64 bits in length with a 32 bits of data overlap between adjacent windows. The 50% window overlap has proved to be a successful and efficient method in the existing related research. When the sampling frequency of the sensor is set to 32, the window length of 64 bits corresponds to a time period of 2 seconds, which is generally longer than the cycles of most of the activities. Meanwhile, the window length of 64 bits also facilitates the Fast Fourier Transform (FFT) required for performing the extraction of characteristics.
The processing of sensing data further comprises a step 220 of extracting features from a plurality of data segments so as to form a feature vector characterizing the activity. For each dimensional data, the following 11 characteristics may be extracted: the mean, variance, energy, 4 amplitude statistical characteristics, and 4 shape statistical characteristics based on power spectral density (PSD). Alternatively, the correlation coefficients between any two dimensions among the three-dimensional data may be extracted, with 36 characteristics in total.
The mean and variance characteristics are the average and standard variance of the data for the interception windows, and the latter is used for characterizing signal stability. The value of the energy characteristic is equal to the sum of the squares of the respective components of the window signals that are subjected to the Fast Fourier Transform. The power spectral density,
which is defined as the Fourier Transformation of the autocorrelation function of the signal, is used for describing the signal energy distribution over the frequency domain. The amplitude statistical characteristics and the shape statistical characteristics thereof may be extracted from the power spectral density. The 4 extracted amplitude statistic characteristics are as follows:
mean: "- ^§C(l) ( 1 )
4
Kurtosis: 1=1 v σ<mv ) (4) wherein C(i) is the size of the i-th frequency unit of the power spectral density, and N is the number of the frequency unit. Similarly, the 4 extracted shape statistic characteristics are as follows:
1 N mean: ύ ■=' ( 5 )
S = ∑C{i) wherein, i=1
The association coefficients between sensor data of any two dimensions are equal to the
/ N cov(x, y) corr[x, y) = ^ — -1- ratio of their covariance to the product of their standard variances, i.e., σxσy
The processing of sensing data further comprises a step 230 of normalizing the feature
vector. After all the characteristics have been extracted, the characteristic for each dimension is generally normalized by a Z-core normalizing method in order to remove the impact on the subsequent analysis resulting from the difference in value among the different characteristics. A feature vector X for characterizing the user activity to be tested can be formed by using all or part of the above characteristics.
The processing of sensing data further comprises a step 240 of performing a principal components analysis on the normalized feature vector to reduce the dimension of the feature vector.
In a practical application, the dimensions of the characteristic space may generally be lowered by a principal component analysis method in order to optimize the feature vector. This principal component analysis method, which is a commonly used method in the field of dimensions reduction, maintains information on the characteristics to the greatest extent while reducing the vector dimensions by means of a linear transform matrix. In such a principal component analysis, the transform matrix consists of the feature vectors of the feature vector covariance matrix. The feature vector of the covariance matrix is represented by A1 , the feature vector of the covariance matrix is represented by υt ; and the transform matrix is built up by selecting the feature vectors corresponding to the greatest K feature vectors. The value of K is
determined by ~ ' ~ ' ~ , where M represents the dimensions of the characteristics before transforming, and η is the lost energy.
After obtaining the (optimized) feature vector, the activity is then classified on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity in step 140.
In the recognition method provided by the present invention, it is assumed that n one- class classifier models have been trained and established for each known activity according to the training samples of n known activities. The one-class classifier model determines that a border encompassing a target sample can differentiate whether a test sample, e.g, the activity to be tested that is represented by a feature vector, is known (target class) or unknown (exceptional class).
In order to distinguish the target class from the exceptional class as accurately as possible, the border should encompass the target sample as closely as possible. To this end, an exclusion
threshold s is generally set such that the border encompasses the target sample of 1 ~ s only, and the remaining $ target sample is considered the exceptional sample, which is held excluded outside the border.
There are many one-class classifier models as referred to above in the prior art, such as the K Nearest Neighbor (KNN) one-class classifier, the Support Vector Data Description (SVDD) one-class classifier, and the Gauss one-class classifier.
The training process and the test process for the sample classification on the basis of the classifier model will now be described below for the example of a Gauss one-class classifier. It is assumed that the set of training samples is the set of feature vectors where X1 is a feature vector of a
sample obtained in step 120, N js the number of the sample for training, and n is the length of the vector, i.e., the number of characteristics in the vector.
The average vector ^ and the variance vector σ can be calculated, i.e. the Gaussian distribution ^v"' σ) of the class of the samples is obtained from the following equations:
The distance vector " of the set of samples can be obtained by calculating the distance
from the vector } to the Gaussian distribution ^' σ> .
d} = e 2σ' ^ d = (d, , d2, - - - , dN) ( n )
1 ' ' ' is sorted in ascending order, and the distance threshold value th is taken as the K -th distance value, ~ L -I , , where δ is the preset exclusive threshold
value as described above, and L-I represents the downward rounding. The distance threshold value can be used to decide whether the category of a test sample is the same as that of the
training sample.
For any activity to be tested, the distance value d y = e 2σ' can be calculated from its associated feature vector
^AA σ) ; and if the
distance value meets the condition of ' * , it can be concluded that the activity to be tested belongs to the category of the activity to which the Gauss classifier relates.
It is reasonable to determine whether a test sample belongs to a known activity by using a combination of a plurality of one-class classifier models. Based on each class of classifier models among n one-class classifier models, it can be determined whether a test sample belongs to an activity or not. The final determining result, i.e. whether a certain test sample belongs to a known activity or an unknown activity, is obtained through the decision obtained by merging the outputs of n one-class classifier models. Here, a logic OR operation may be used to perform the decision merging. The merging rule is: a test sample belongs to a known activity if the test sample is determined as a known sample by one or more one-class classifier models; otherwise, the test sample belongs to an unknown activity.
Furthermore, a one-class classifier relating to all of the known activity groups can be obtained, e.g., a combined one-class classifier, by using the samples in all of the known activity groups for training, that is, whether a test sample belongs to at least one known activity group or should be excluded from all known activity groups can be determined on the basis of the combined one-class classifier model.
It is easily understood that, when two activities are quite similar, their samples should be adjacent to each other in the characteristic space, and even demonstate an overlap in some parts. Therefore, a test sample may be contained in two or more one-class classifier models. If the sample areas of respective activities do not overlap in the characteristic space, then, e.g., a test sample is contained, at most, within the border defined by a one-class classifier model. In such a situation, the test sample can be identified to belong to the known activity corresponding to this one-class classifier and no further classifying is needed.
However, for many cases, there is considerable similarity between some activities of the users, and the diversity of the user activities has to be considered by an activity recognition
system. When a test sample is in the overlap area of the characteristic space of two or more activities, the multi-class classifier is best suited to decide which activity it belongs to. An advantage of the method provided by the invention is that the activity recognition is performed by the at least one one-class classifier in combination with a multi-class classifier.
The multi-class classifier is used for the distinction between two or more classes, and it requires that the training samples of each class are available. The multi-class classifier establishes a multi-class classifier model by using all training samples in each class so as to recognize which class a test sample is most likely to belong to.
The multi-class classifier may be a Decision Tree (DT) algorithm, a K Nearest Neighbor (KNN) algorithm, or a Weighted Support Vector Machine (WSVM) algorithm. The process for the training and testing of the multi-class classifier can be found in the prior art literature, such as the article "Weighted-support vector machine for predicting membrane protein types based on pseudo-amino acid composition" pp. 509-516, No. 6, Vol. 17, Protein Engineering, Design & Section Journal.
Fig.3 shows a flowchart of how to train a temporary one-class classifier model in accordance with the invention.
Referring to Fig.3, the training step 140 comprises a sub-step 310 of classifying the activity on the basis of the feature vector and at least one temporary one-class classifier obtained from training samples in any one of existing unknown activity groups so as to determine whether the activity belongs to any one of existing unknown activity groups (312).
The training step 140 further comprises a sub-step 320 of adding the data segments and/or feature vector associated with the activity to the specific existing unknown activity group as samples when the activity belongs to a specific existing unknown activity group.
The training step 140 further comprises a sub-step 330 of training the temporary one-class classifier relating to the existing unknown activity by the data segments and/or feature vector of the activity so as to update the temporary one-class classifier model.
In an embodiment, the training step 140 further comprises a sub-step 340 of labeling the specific unknown activity group as known activity group when the number of samples in the specific unknown activity group reaches a predetermined threshold (335).
In an embodiment, when the activity does not belong to any existing unknown activity group, the training step 140 further comprises a sub-step 350 of setting up a new unknown activity group including the data segments and/or feature vector associated with the activity of the group as samples.
The activity recognition method provided by the invention may be applied to various activity detection systems, for example to a fall system for reducing the occurrence of a false fall alarm. It is well known that falling is one of the factors which endanger the life of the aged, and it is quite useful for an aged person to wear a fall detector in case he or she is in need of help after falling.
Although a fall signal can be reliably detected by many fall detection algorithms, many Activities of Daily Living (ADL) may be detected as a fall since they are very similar to a fall, resulting in false alarm. Furthermore, it is difficult to summarize all of the activities of daily living within one uniform classifier model because of the diversity of individual activities, so it is necessary to perform the training with respect to individual persons.
Fig. 4 is a flowchart for the activity recognition method extended with fall detection in accordance with the invention. It depicts how to apply the activity recognition method to fall detection according to the present invention.
Compared to Fig.l showing the flowchart for the method of activity recognition, steps 120, 130, 140, 150 and 160, 170 (160, 170 are not shown in Fig.4) are substantially the same or similar, but the method is extended with steps 410 to 440 for fall detection. In the following description, only the new steps are explained in detail.
Referring to Fig.4, the method further comprises a step 410 of detecting a fall when the activity is determined by the one-class classifier model as an unknown activity in step 130, 135. There are many known fall detection algorithms in the prior art that can be used for this purpose.
When the activity is identified as not a fall, the method goes to step 140 for training a temporary one-class classifier model relating to an unknown activity. The method further comprises a step 440 to issue a fall alarm if the activity is identified as a fall.
It is advantageous that the method further comprises steps of communicating with users
when the activity is identified as a fall and before issuing the fall alarm.
In an embodiment, the method comprises a step 420 of informing the user that the activity is identified as a fall and a step 430 of receiving an input from the user that negates or confirms that the activity is a fall. When the activity is negated as being a fall, the method goes to step 140 for training a temporary one-class classifier mode relating to an unknown activity. Otherwise, a fall alarm is issued. By using user input for activity recognition and training, the accuracy of activity recognition is improved, resulting in a reduction of false fall alarms.
The activity recognition method as described in connection with Figs. 1 to 4 and fall detection may be implemented by means of software or hardware or a combination thereof.
Fig. 5 is a block diagram of an activity recognition system in accordance with the invention.
Referring to Fig.5, the activity recognition system comprises a deriving unit (520) for deriving a feature vector characterizing an activity from sensing data associated with the activity. The deriving unit executes the function of step 120.
The activity recognition system further comprises a first classifying unit 530 for classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity. The first classifying unit 530 executes the function of step 130.
The activity recognition system further comprises a training unit 540 for training a temporary one-class classifier model relating to an unknown activity if the activity is determined as an unknown activity or, otherwise, for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs. The training unit 540 executes the function of step 140.
The activity recognition system further comprises a second classifying unit 550 for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs when the activity is determined as a known activity. The second classifying unit 550 executes the function of step 150.
In an embodiment, the at least one one-class classifier model comprises a combined one- class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups.
The activity recognition system further comprises: a first unit 560 for re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group. The first unit 560 executes the function of step 160; and a second unit 570 for re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group. The second unit 570 executes the function of step 170.
In an embodiment, the activity recognition system further comprises a sensor unit 510 for obtaining data associated with an activity; a data storage unit 503 for storing the sensor data, feature vector, and other temporary data; and an internal data line 405 for connecting respective functional units in the system.
It is advantageous to extend the activity recognition system to include fall detection.
Fig. 6 is a block diagram of the activity recognition system extended with a fall detection function in accordance with the invention.
Compared to Fig.5, the units 510 to 550 are substantially the same. The system further comprises a fall detection unit 680 for performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity. The fall detection unit executes the function of step 410.
The system further comprises a user interface 690 arranged for informing the user when the activity is identified as a fall, receiving an input from the user that negates or confirms that the activity is a fall, and generating a fall alarm when the activity is identified and/or confirmed by the user as a fall. The user interface executes the functions of steps of 420, 430, and 440.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim or in the description. The word "a" or "an" preceding an element does not exclude the presence of a
plurality of such elements. In the system claims enumerating several units, several of these units can be embodied by one and the same item of software and/or hardware. The usage of the words first, second and third, et cetera, does not indicate any ordering. These words are to be interpreted as names.
Claims
1. An activity recognition method, comprising the steps of:
- deriving (120) a feature vector characterizing an activity from sensing data associated with the activity;
- classifying (130) the activity on basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity; and
- training (140) a temporary one-class classifier model relating to an unknown activity when the activity is determined as an unknown activity.
2. The method as claimed in claim 1, when the activity is determined as a known activity, the method further comprises a step (150) of classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs when the activity is determined as a known activity.
3. The method as claimed in claim 1, wherein the at least one one-class classifier model comprises a combined one-class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups.
4. The method as claimed in claim 1, wherein the step (140) of training comprises sub-steps of
- classifying (310) the activity on the basis of the feature vector and at least one temporary one-class classifier obtained from training samples in any one of existing unknown activity groups so as to determine whether the activity belongs to any one of existing unknown activity groups; and
- adding (320) the data segments and/or feature vector associated with the activity to the specific existing unknown activity group as samples when the activity belongs to a specific existing unknown activity group; and
- training (330) the temporary one-class classifier relating to the existing unknown activity by the data segments and/or feature vector of the activity so as to update the temporary one-class classifier model.
5. The method as claimed in claim 4, wherein the step of training further comprises a sub- step (340) of labeling the specific unknown activity group as a known activity group when the number of samples in the specific unknown activity group reaches a predetermined threshold.
6. The method as claimed in claim 4, when the activity does not belong to any existing unknown activity group, the training step further comprises a sub-step (350) of setting up a new unknown activity group including the data segments and/or feature vector associated with the activity of the group as samples.
7. The method as claimed in claim 1, wherein the deriving step comprises the sub-steps of:
- segmenting (210) sensing data associated with the activity of a user into a plurality of data segments;
- extracting (220) features from a plurality of data segments so as to form a feature vector characterizing the activity;
- normalizing (230) the feature vector; and
- performing (240) a principal components analysis on the normalized feature vector to reduce the dimension of the feature vector.
8. The method as claimed in any one of claims 3 to 7, when the activity belongs to a known activity group, the method further comprises the steps of:
- re-training (160) the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group;
- re- training (170) the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
9. The method as claimed in any one of claims 1 to 8, further comprising a step (410) of performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity.
10. The method as claimed in claim 9, further comprising steps of:
- informing (420) the user when the activity is identified as a fall;
- receiving (430) an input from the user that negates or confirms that the activity is a fall; and
- generating (440) a fall alarm when the activity is identified and/or confirmed by the user as a fall.
11. An activity recognition system, comprising:
- a deriving unit for deriving (520) a feature vector characterizing an activity from sensing data associated with the activity;
- a first classifying unit (530) for classifying the activity on the basis of the feature vector and at least one one-class classifier model relating to known activities so as to determine whether the activity is a known activity or an unknown activity;
- a training unit (540) for training a temporary one-class classifier model relating to an unknown activity if the activity is determined as an unknown activity, otherwise, classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs.
12. The system of claim 11, comprising a second classifying unit (550) for classifying the activity on the basis of the feature vector and a multi-class classifier to identify to which known activity group the activity belongs when the activity is determined as a known activity.
13. The system of claim 12, wherein the at least one one-class classifier model comprises a combined one-class classifier model relating to all of the known activity groups or a plurality of one-class classifier models relating to respective known activity groups.
14. The system as claimed in claim 13, further comprising: - a first unit (560) for re-training the combined one-class classifier model characterizing all of the known activity groups by using a plurality of feature vectors associated with a plurality of activities in the known activity group;
- a second unit (570) for re-training the multi-class classifier model by using the plurality of feature vectors associated with a plurality of activities in the known activity group.
15. The system of claim 14, further comprising:
- a fall detection unit (680) for performing fall detection to determine whether the activity is a fall when the activity is identified as an unknown activity; and
- a user interface (690) arranged for informing the user when the activity is identified as a fall, receiving an input from the user that negates or confirms that the activity is a fall, and generating a fall alarm when the activity is identified and/or confirmed by the user as a fall.
4
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200810003518.5 | 2008-01-18 | ||
| CN200810003518 | 2008-01-18 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2009090584A2 true WO2009090584A2 (en) | 2009-07-23 |
| WO2009090584A3 WO2009090584A3 (en) | 2009-10-29 |
Family
ID=40885718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IB2009/050093 Ceased WO2009090584A2 (en) | 2008-01-18 | 2009-01-12 | Method and system for activity recognition and its application in fall detection |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2009090584A2 (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100325487A1 (en) * | 2009-06-19 | 2010-12-23 | Xerox Corporation | Method and system for automatically diagnosing faults in rendering devices |
| CN103398843A (en) * | 2013-07-01 | 2013-11-20 | 西安交通大学 | Classification method of planetary gear box sun gear faults based on multi-class relevance vector machine |
| US20140089024A1 (en) * | 2011-05-26 | 2014-03-27 | Koninklijke Philips N.V. | Control device for resource allocation |
| WO2014130577A1 (en) * | 2013-02-22 | 2014-08-28 | InvenSense, Incorporated | Systems and methods for activity recognition training |
| WO2015165260A1 (en) * | 2014-04-29 | 2015-11-05 | 华南理工大学 | Triaxial feature fusion method for human body movement identification |
| WO2017039684A1 (en) * | 2015-09-04 | 2017-03-09 | Hewlett Packard Enterprise Development Lp | Classifier |
| CN107527016A (en) * | 2017-07-25 | 2017-12-29 | 西北工业大学 | Method for identifying ID based on action sequence detection under indoor WiFi environment |
| CN108008151A (en) * | 2017-11-09 | 2018-05-08 | 惠州市德赛工业研究院有限公司 | A kind of moving state identification method and system based on 3-axis acceleration sensor |
| JP2018528525A (en) * | 2015-08-04 | 2018-09-27 | クゥアルコム・インコーポレイテッドQualcomm Incorporated | Unknown class detection and classifier initialization for unknown classes |
| CN108960056A (en) * | 2018-05-30 | 2018-12-07 | 西南交通大学 | A kind of fall detection method based on posture analysis and Support Vector data description |
| CN109086667A (en) * | 2018-07-02 | 2018-12-25 | 南京邮电大学 | Similar activity recognition method based on intelligent terminal |
| CN110428016A (en) * | 2019-08-08 | 2019-11-08 | 中国联合网络通信集团有限公司 | Feature vector generation method and system, user's identification model generation method and system |
| CN110659624A (en) * | 2019-09-29 | 2020-01-07 | 上海依图网络科技有限公司 | Group personnel behavior identification method and device and computer storage medium |
| US10545578B2 (en) | 2017-12-22 | 2020-01-28 | International Business Machines Corporation | Recommending activity sensor usage by image processing |
| US10881327B2 (en) | 2014-02-14 | 2021-01-05 | 3M Innovative Properties Company | Activity recognition using accelerometer data |
| US11324421B2 (en) | 2014-09-15 | 2022-05-10 | 3M Innovative Properties Company | Impairment detection with environmental considerations |
| US12027035B2 (en) | 2019-11-29 | 2024-07-02 | Lifeline Systems Company | Personalized fall detector |
| US12089925B2 (en) | 2014-09-15 | 2024-09-17 | Attenti Electronic Monitoring Ltd. | Impairment detection |
| US12390126B2 (en) | 2014-09-15 | 2025-08-19 | Attenti Electronic Monitoring Ltd. | Impairment detection with biological considerations |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6614348B2 (en) * | 2001-03-23 | 2003-09-02 | International Business Machines Corporation | System and method for monitoring behavior patterns |
-
2009
- 2009-01-12 WO PCT/IB2009/050093 patent/WO2009090584A2/en not_active Ceased
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8249830B2 (en) * | 2009-06-19 | 2012-08-21 | Xerox Corporation | Method and system for automatically diagnosing faults in rendering devices |
| US20100325487A1 (en) * | 2009-06-19 | 2010-12-23 | Xerox Corporation | Method and system for automatically diagnosing faults in rendering devices |
| US20140089024A1 (en) * | 2011-05-26 | 2014-03-27 | Koninklijke Philips N.V. | Control device for resource allocation |
| WO2014130577A1 (en) * | 2013-02-22 | 2014-08-28 | InvenSense, Incorporated | Systems and methods for activity recognition training |
| CN103398843A (en) * | 2013-07-01 | 2013-11-20 | 西安交通大学 | Classification method of planetary gear box sun gear faults based on multi-class relevance vector machine |
| CN103398843B (en) * | 2013-07-01 | 2016-03-02 | 西安交通大学 | Based on the epicyclic gearbox sun gear Fault Classification of many classification Method Using Relevance Vector Machines |
| US10881327B2 (en) | 2014-02-14 | 2021-01-05 | 3M Innovative Properties Company | Activity recognition using accelerometer data |
| WO2015165260A1 (en) * | 2014-04-29 | 2015-11-05 | 华南理工大学 | Triaxial feature fusion method for human body movement identification |
| US11324421B2 (en) | 2014-09-15 | 2022-05-10 | 3M Innovative Properties Company | Impairment detection with environmental considerations |
| US12390126B2 (en) | 2014-09-15 | 2025-08-19 | Attenti Electronic Monitoring Ltd. | Impairment detection with biological considerations |
| US12089925B2 (en) | 2014-09-15 | 2024-09-17 | Attenti Electronic Monitoring Ltd. | Impairment detection |
| JP2018528525A (en) * | 2015-08-04 | 2018-09-27 | クゥアルコム・インコーポレイテッドQualcomm Incorporated | Unknown class detection and classifier initialization for unknown classes |
| US11403550B2 (en) | 2015-09-04 | 2022-08-02 | Micro Focus Llc | Classifier |
| WO2017039684A1 (en) * | 2015-09-04 | 2017-03-09 | Hewlett Packard Enterprise Development Lp | Classifier |
| CN107527016A (en) * | 2017-07-25 | 2017-12-29 | 西北工业大学 | Method for identifying ID based on action sequence detection under indoor WiFi environment |
| CN107527016B (en) * | 2017-07-25 | 2020-02-14 | 西北工业大学 | User identity identification method based on motion sequence detection in indoor WiFi environment |
| CN108008151A (en) * | 2017-11-09 | 2018-05-08 | 惠州市德赛工业研究院有限公司 | A kind of moving state identification method and system based on 3-axis acceleration sensor |
| US10545578B2 (en) | 2017-12-22 | 2020-01-28 | International Business Machines Corporation | Recommending activity sensor usage by image processing |
| CN108960056A (en) * | 2018-05-30 | 2018-12-07 | 西南交通大学 | A kind of fall detection method based on posture analysis and Support Vector data description |
| CN109086667A (en) * | 2018-07-02 | 2018-12-25 | 南京邮电大学 | Similar activity recognition method based on intelligent terminal |
| CN110428016A (en) * | 2019-08-08 | 2019-11-08 | 中国联合网络通信集团有限公司 | Feature vector generation method and system, user's identification model generation method and system |
| CN110659624A (en) * | 2019-09-29 | 2020-01-07 | 上海依图网络科技有限公司 | Group personnel behavior identification method and device and computer storage medium |
| US12027035B2 (en) | 2019-11-29 | 2024-07-02 | Lifeline Systems Company | Personalized fall detector |
| US12417694B2 (en) | 2019-11-29 | 2025-09-16 | Connect America | Personalized fall detector |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2009090584A3 (en) | 2009-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2009090584A2 (en) | Method and system for activity recognition and its application in fall detection | |
| Ige et al. | State-of-the-art in 1d convolutional neural networks: A survey | |
| Lester et al. | A hybrid discriminative/generative approach for modeling human activities | |
| Frank et al. | Activity and gait recognition with time-delay embeddings | |
| Xie et al. | Bioacoustic signal classification in continuous recordings: Syllable-segmentation vs sliding-window | |
| CN112464738A (en) | Improved naive Bayes algorithm user behavior identification method based on mobile phone sensor | |
| CN111860117A (en) | Human behavior recognition method based on deep learning | |
| AU2020102094A4 (en) | GROUP ACTIVITY RECOGNITION BY INTEGRATION AND FUSION OF INDIVIDUAL MULTISENSORY IoT DATA | |
| KR20210023170A (en) | Deep learning based emotional recognition system and methods using PPG signals | |
| CN112037929A (en) | Classification method based on multi-modal machine learning, online new coronary pneumonia early warning model training method and early warning method | |
| Singh et al. | Speaker specific feature based clustering and its applications in language independent forensic speaker recognition | |
| Uslu et al. | A segmentation scheme for knowledge discovery in human activity spotting | |
| Mustaffa et al. | Durian recognition based on multiple features and linear discriminant analysis | |
| Zhou et al. | Threshold selection and adjustment for online segmentation of one-stroke finger gestures using single tri-axial accelerometer | |
| Kalantarian et al. | Probabilistic segmentation of time-series audio signals using Support Vector Machines | |
| CN109948686B (en) | Swimming stroke identification method based on nine-axis sensing signal statistical characteristics | |
| KR101483218B1 (en) | Activity diagnosis apparatus | |
| Stager et al. | Dealing with class skew in context recognition | |
| Jarchi et al. | Transition detection and activity classification from wearable sensors using singular spectrum analysis | |
| CN102789780A (en) | Method for identifying environment sound events based on time spectrum amplitude scaling vectors | |
| CN107871113B (en) | A method and device for mixed emotion recognition and detection | |
| JP2008140093A (en) | Abnormal event extraction apparatus, abnormal event extraction method, program of this method, and recording medium recording this program | |
| KR101367418B1 (en) | Apparatus and method for detecting estrus audio of livestock using a feature vector generation algorithm | |
| CN110458118B (en) | A Simple Sign Language Recognition Method Based on Channel State Information | |
| Cenedese et al. | A parsimonious approach for activity recognition with wearable devices: An application to cross-country skiing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09701596 Country of ref document: EP Kind code of ref document: A2 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 09701596 Country of ref document: EP Kind code of ref document: A2 |