WO2010036091A2

WO2010036091A2 - A system and a method for identifying human behavioural intention based on an effective motion analysis

Info

Publication number: WO2010036091A2
Application number: PCT/MY2009/000153
Authority: WO
Inventors: Mei Kuan Lim; Kim Meng Liang; Tomas Henrique Maul; Weng Kin Lai
Original assignee: Mimos Bhd
Current assignee: Mimos Bhd
Priority date: 2008-09-24
Filing date: 2009-09-18
Publication date: 2010-04-01
Anticipated expiration: 2011-03-24
Also published as: EP2327057A4; EP2327057A2; WO2010036091A3; CN102224526A; MY159289A

Abstract

With the growing market for video surveillance in security area, there is a need for an automated system which provides a way to track and detect human intention based on a particular human motion. The present invention relates to a system and a method for identifying human behavioral intention based on effective motion analysis wherein, the system obtains a sequence of raw images taken from live scene and processes the raw images in an activity analysis component. The activity analysis component is further provided with an activity enrollment component and activity detection component.

Description

A SYSTEM AND A METHOD FOR IDENTIFYING hjUMAN BEHAVIOURAL INTENTION BASED ON AN EFFECTIVE MOTION ANALYSIS

FIELD OF INVENTION

The present invention relates to a system and a method for identifying human behavioral intention based on effective human motion analysis.

BACKGROUND OF INVENTION

There is a growing interest in activity analysis or predicting human intention based on human motion due to immense need for better automate , d I! video surveillance systems that go beyond just merely identifying simple objects. The capability to automatically monitor human motion using computers in security-sensitive areas ' s iuch as airports, borders and building lobbies is of great interest to security personnel, e.g. police and military.

A recent market study from IMS Research has found that the trend from analogue CCTV to

^■ i network video surveillance is in full swing. The world market for network video surveillance products increased by an impressive 41.9% in 2006 Jϊnd is forecasted to continue growing strongly for many years to come. By 2010, the combined market for network cameras, video

^; i ^' ■ servers and NVRs is forecasted to exceed US$2.6 billion.

In China, the market for video servers (video encoders) used in security applications increased by a massive 60% in 2007, according to a 'report titled "The Chinese Market for CCTV and Video Surveillance Equipment" by IMS Research. The market is forecasted to continue growing rapidly over the coming years to exceed US$150 million by 2011. This i j dramatic growth is primarily attributed to the strong demand for IP-based video surveillance systems in China. More and more users of security systems are choosing networked solutions based on video servers, instead of traditional analogue CCTV systems.

30.3% forecast for video camera servers. Together^ these markets will be worth some

EUR151.1 million by 2008.

These are summarized in table 1.

Traditionally, motion analysis of humans has used markers attached to appropriate parts of a human body to highlight the movement of these points and how they relate to each motion sequence. These were extensively used in sports 'to enhance performance of athletes. i I

Passive reflective markers are placed on subjects in' this case, human; at specific anatomical landmarks. As the subjects walk through a lab, a three-dimensional location of each marker is detected by multiple infrared cameras. A biomechanics model is applied to the marker series to calculate the three-dimensional motion of each body segment. The processed data generates a graphical representation of each joint in all three planes and is expressed in terms of a gait cycle.

lower limbs or better known as gait analysis.

Intille, J.W. Davis, and A.F. Bobick. "Real-time closed-world tracking", In Proc. of the IEEE

Conf. on Computer Vision and Pattern Recognition, pages 697-703, Los Alamitos, CA, June 1997. IEEE Computer Society Press.

SUWI WlARY OF INVENTION

I ! • Accordingly, the present invention provides a system for identifying a behavioral intention of a human being based on an effective motion analysis, the system includes an image acquisition component, wherein the image acquisition component acquires a plurality of j I object images in sequence, an activity enrollment component includes a background and foreground detection component, an object detection component, an object partitioning component, a key point extraction component, a gait feature extraction component, an activity registration component and an activity storage means and an activity detection component includes a background and foreground detection component, an object detection component, an object partitioning component, a key point extraction component, a gait

^• / : I feature extraction component, an activity matching ' component and an activity storage means characterized in that the gait feature extraction components compute features related to the movement of an upper limb of the human in relation to the movement of a lower limb in a gait cycle, wherein the gait cycle consists of a stance phase and a swing phase. j Furthermore, the present invention also provides a' method for identifying a behavioral

^■ I intention of a human being based on an effective motion analysis, the method includes acquiring a plurality of object images in sequence, 'enrolling data in a background and i foreground detection component, an object detection component, an object partitioning component, a key point extraction component, a gait feature extraction component, an activity registration component and an activity storage means and detecting features and matching features using a background and foreground detection component, an object detection component, an object partitioning component, a key point extraction component, a gait feature extraction component, an activity matchirig component and an activity storage means characterized in that the method further includes calculating features related to the movement of an upper limb of the human in relation to the movement of a lower limb in a gait cycle using the gait feature extraction components wherein the gait cycle consists of a stance phase and a swing phase.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be fully understood from 'the detailed description given herein below and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present invention, wherein:

Fig. 1 is a block diagram representation of a system and a method for identifying behavioral intention of a human being based on an effective motion analysis based on the preferred embodiment of the present invention;

Fig. 2 shows a representation of an upper limb motion 'in a stance phase of a gait cycle; i

Fig. 3 shows a representation of an upper limb motion 'in a swing phase of a gait cycle;

Fig. 4 is a comparison table between upper limb movement and lower limb movement in a typical gait cycle;

stance phase;

of a lower limb movement in a

stance phase;

Fig. 10 is a diagram illustrating the computation of a distance of an upper limb movement in a swing phase; Fig. 11 is a diagram illustrating the computation of a distance of a lower limb movement in a swing phase.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

video images.

the moving objects.

From the detected object of interest, an object partitioning component (52) divides the object of interest into four main parts which are head, torso, arms and legs. From the arms and legs, a key point extraction component (53) computes .important points on these two parts to

; j be used in a gait feature extraction component (54). Vhe important points may include the corner points, high curvature points and joining points that are detected from the outline of the arms and legs.

The gait feature extraction component (54) computes^' the features related to movement of the arms and legs. The computed gait features from ; th Ie sequence of images are registered to a particular activity or intention, through an activity registration component (55). The gait features and the registered activity or intention are stored in the activity database (56). Fig. 6 depicts a detailed architecture of the activity detection component with an assumption that the raw video images (68) from the image acquisition component are available in real

' i time. The video images (68) are fed into the similar components that are describes in the

! i activity enrollment component (refer to Fig. 5) except the activity registration component (55) is replaced with an activity matching component (65). In the activity matching component

(65), the computed gait features from the sequence of video images (68) are compared with the registered gait features in the activity database (67). A matching process will be

^■ conducted to match the activity or intention that has Jbeen registered and the gait features detected and vice versa.

Table 2: Six main features in the gait extraction component.

phase. For feature 5, the distance of arm movement in one gait cycle is computed by adding up the circular distances from feature 1 with feature 3. Basically,

Feature 5 = Z)""" + D^"

where,

D°™ : Distance of arm movement in a stance phase

D™' : Distance of arm movement in a swing phase

For feature 6, the distance of leg movement in one gait cycle is computed by adding the horizontal distances from feature 2 to that of feature 4. Basically,

Feature 6 = D_s ^lf + D^

where,

D£^S : Distance of leg movement in a stance phase

D^¹ : Distance of leg movement in a swing phase

Claims

1. A system for identifying a behavioral intention of a human being based on an effective motion analysis, the system includes:

a. an image acquisition component, wherein the image acquisition component

I acquires a plurality of object images in siequence;

I b. an activity enrollment component includes a background and foreground detection component. (50), an object detection component (51), an object partitioning component (52), a key point extraction component (53), a gait feature extraction component (54), an activity registration component (55) and an activity storage means (56) and

c. an activity detection component includes a background and foreground i detection component (60), an object detection component (61), an object

^■ i partitioning component (62) , a key point extraction component (63), a gait feature extraction component (64), an activity matching component (65) and an activity storage means (67)

characterized in that

the gait feature extraction components (54, 64) compute features related to the movement of an upper limb of the human in relation to the movement of a lower limb j in a gait cycle, wherein the gait cycle consists of a stance phase and a swing phase.

2. The system as claimed in claim 1, wherein the background and foreground

I components (50, 60) determine presence of moving objects.

3. The system as claimed in claim.2, wherein the object detection components (51, 61)

I highlight the moving object as an object of interest

4. The system as claimed in claim 3, wherein the object partitioning components (52,

I

62) divide the object of interest into four main parts: 1) head, 2) torso, 3) upper limbs and 4) lower limbs. . .

5. The system as claimed in claim 1, wherein the key point extraction component (53,

63) compute important points of the four main parts of the divided object from the i object partitioning component.

i

6. The system as claimed in claim 1, wherein the gait feature extraction components

(54, 64) extract one or more of the following features:

a. distance of upper limb movement in stance phase; b. distance of lower limb movement in stance phase; c. distance of upper limb movement in swing phase; d. distance of lower limb movement in swing phase; e. distance of upper limb movement in one gait cycle; or f. distance of lower limb movement in one gait cycle.

7. The system as claimed in claim 1 , wherein a plurality of object images in sequence is

. i ^■ acquired by means of a video camera or a still camera.

8. The system as claimed in claim 1, wherein the computation processes are done using a computing means such as a computer.

9. A method for identifying a behavioral intention of a human being based on an effective motion analysis, the method includes:

a. acquiring a plurality of object images in sequence,

b. enrolling data in a background and foreground detection component (50), an object detection component (51), an object partitioning component (52), a key point extraction component (53), a gait feature extraction component (54), an activity registration component (55) and an activity storage means (56) and,

c. detecting features and matching features using a background and foreground i detection component (60), an object detection component (61), an object partitioning component (62), a key point extraction component (63), a gait feature extraction component (64), an activity matching component (65) and an activity storage means (67)

characterized in that

the method further includes calculating features related to the movement of an upper

! limb of the human in relation to the movement of a lower limb in a gait cycle using the i gait feature extraction components (54, 64) wherein the gait cycle consists of a stance phase and a swing phase.

10. The method as claimed in claim 9, wherein the background and foreground components (50, 60) determine presence of moving objects.

11. The method as claimed in claim 9, wherein the object detection components (51, 61) i highlight the moving object as an object of interest.

12. The method as claimed in claim 9, wherein the object partitioning components (52, 62) divide the object of interest into four main parts: 1) head, 2) torso, 3) upper limbs and 4) lower limbs.^'

13. The method as claimed in claim 9, wherein the key point extraction component (53, 63) compute important points of the four- main parts of the divided . object from the object partitioning component.

14. The method as claimed in claim 9, wherein the activity enrollment component is performed in offline mode.

15. The method as claimed in claim 9, wherein the gait feature extraction components j (54, 64) extract one or more of the following features:

a. distance of upper limb movement in stance phase; b. distance of lower limb movement in stance phase;

J c. distance of upper limb movement in swing phase; i d. distance of lower limb movement in swing phase; j e. distance of upper limb movement in one; gait cycle; or f. distance of lower limb movement in one^! gait cycle.