JP2006279111A - Information processor, information processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processor which can support look-back, and to provide an information processing method and a program. <P>SOLUTION: A multi-modal recognition section 3 observes important parts of meeting by image processing, e. g. the operation analysis of salute, standing up or board writing, speaker shift, speaker identification, the extraction of a key person, and the like, and then calculates the weight of significance every time duration. Subsequently, the multi-modal recognition section 3 totalizes the weight of significance in units of utterance and identifies the important parts of meeting. Furthermore, the multi-modal recognition section 3 calculates the significance factor of each scene by expressing the number of words in utterance, the exclamation of hesitation in saying, a volume of a voice, a variation in the pitch of a voice, and the inclination of a face in a predetermined section in terms of predetermined weight. Thereafter, the multi-modal recognition section 3 estimates a key person of each scene based on the significance factor. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  2. Description of the Related Art Conventionally, in a video conference system, a technique has been proposed in which camera control is controlled based on participant location information acquired by wireless communication. For example, the importance level information of the conference participant is incorporated in advance in the main device side of the video conference system, and the camera is controlled based on this information. In addition, data of the local station is periodically transmitted from a transmitter having a wireless communication function carried by a participant, and the main station side of the video conference system receives the local station information from the transmitter with two antennas. Then, the location information is calculated and stored from the difference in radio field intensity, the shooting position of the camera is extracted from the transmitter with the camera control right, and the participant carrying the transmitter with the camera control right is displayed correctly on the screen. It is possible to display (Patent Document 1). A method for determining the importance of a video stream from the index and the importance associated with the index has been proposed (Patent Document 2).

JP 2000-197027 A JP 2004-80769 A

  Conventionally, in meetings, minutes have been created to reflect on the status of the meeting to review, absentees and related parties. In recent years, recording / recording can be easily performed, and images and sounds may be recorded. For example, in a meeting that is regularly held at intervals, it is known that viewing the summary of the previous meeting at the beginning of the meeting increases the efficiency of the meeting. For this purpose, it is necessary to create a recording or a summary of the recording mainly by hand, and there are serious problems in cost and labor.

  On the other hand, assuming that an unedited recording or recording is viewed without summarizing, there is a problem that the efficiency for reuse is low, for example, it takes the same time as a meeting to view. For this reason, unedited recordings and recorded videos were rarely used.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that can support reflection.

  In order to solve the above problems, the present invention uses at least one of a technique for recognizing a feature quantity from acoustic information and a technique for recognizing a feature quantity from image information based on video information photographed by a photographing means. Extracting means for extracting feature quantities, totalizing means for totaling the feature quantities extracted by the extracting means, and estimation means for estimating the state of the space photographed by the photographing means based on the totaling results by the totaling means It is an information processing apparatus characterized by comprising. Thereby, the reflection can be supported by estimating the state of the space photographed by the photographing means using the feature amount.

  The estimating means estimates a state of a space photographed by the photographing means by associating a result of the summing by the summing means with a predetermined importance.

  The estimating means estimates a key person in the video based on a counting result by the counting means. The estimation means includes a part where the key person speaks, a part where the speech speed is faster than a predetermined speed, a part where the voice is larger than a predetermined level, a part where the pitch is higher than the predetermined pitch, a part after raising a hand, After that, the part where there is a change of the speaker a predetermined number of times within the predetermined period, the part where the person is standing, the part where the person who is writing on the board is present, the part where the private language is less than the predetermined number and the utterance In this case, at least one of the portions in which there is a section that does not speak only after a predetermined section is estimated as the state of the space photographed by the photographing means.

  The information processing apparatus according to the present invention includes a recording unit that records a usage history of a playback device that plays back the video information, and a control unit that learns a function for calculating the importance based on the usage history of the playback device. Further prepare. The extraction means extracts at least one of the number of utterance words, the number of utterance phonemes, the number of appearances of impression verbs, the volume of voice, the time change of the volume of voice, the pitch change of phonemes, and the inclination of the face as the feature quantity. And The technology for recognizing the feature amount from the acoustic information is at least one of sound source estimation, utterance unit detection, utterance collision detection, speech speed detection, pitch detection, sound volume detection, silence interval detection, sound quality identification, speech detection and applause detection. It is characterized by including one.

  The technique for recognizing a feature amount from the image information includes at least one of face detection, action detection, and position detection. The aggregation means is characterized in that the feature amounts are aggregated in utterance units. The information processing apparatus according to the present invention further includes an adding unit that adds an index to the video information based on a state of a space captured by the imaging unit estimated by the estimation unit.

  The present invention provides an extraction step of extracting the feature quantity using at least one of a technique for recognizing a feature quantity from acoustic information and a technique for recognizing a feature quantity from image information based on video information photographed by a photographing means. An information processing method comprising: a totaling step for totaling the feature quantities; and an estimation step for estimating the state of the space photographed by the photographing means based on the totalization results of the feature quantities. The estimation step is characterized in that the state of the space photographed by the photographing unit is estimated by associating the total result of the feature amount with a predetermined importance. The estimating step is characterized in that a key person in the video is estimated based on the total result of the feature amount.

  The information processing method of the present invention further includes a step of recording a usage history of a playback device that plays back the video information, and a step of learning a function for calculating the importance based on the usage history of the playback device. It is characterized by that. The information processing method of the present invention further includes a step of assigning an index to the video information based on the estimated state of the space photographed by the photographing means.

  The present invention includes a step of extracting the feature quantity using at least one of a technique for recognizing a feature quantity from acoustic information and a technique for recognizing a feature quantity from image information based on video information photographed by a photographing means, A program for causing a computer to execute a step of totalizing feature amounts and a step of estimating a state of a space photographed by the photographing unit based on the result of summing the feature amounts.

  ADVANTAGE OF THE INVENTION According to this invention, the information processing apparatus, the information processing method, and program which can support reflection can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described.

  First, the first embodiment will be described. FIG. 1 is a diagram showing an overall configuration of a system according to the first embodiment. As shown in FIG. 1, the system 1 includes a sensor 2, a multimodal recognition unit 3, a sound / image recording unit 4, a database 5, a reproduction / use unit 6, and a use recording unit 7. The sensor 2 is constituted by a microphone, a digital video camera (photographing means), or the like. The microphone inputs, for example, an audio signal during a conference. In the digital camera, a landscape during a meeting inputs image information. These audio signals and image information are synchronized.

  The multimodal recognition unit 3 estimates the important part (meeting rust) of the conference through image processing such as motion analysis such as raising hand, standing up, and writing, changing the speaker, identifying the speaker, and extracting the key person. is there. An important part of this conference is the part corresponding to “rust” when compared to music. Next, the multimodal recognition unit 3 adds up the importance weights for each utterance and identifies the important part of the conference.

An important part of the conference has the following characteristics, for example.
・ Key person speaks.
・ Speech speed is fast (phoneme time density is high).
・ Voice is loud and pitch is high.
・ The part immediately after raising the hand / voting.
・ Speaker changes are relatively frequent.
・ Some people are standing and writing on the board.
・ There are few other people's private languages.
・ There is often a section where no one speaks immediately after the utterance unit.
・ Duplicate utterances occur in on-site meetings.
The information integration unit 50 estimates these as the state of the space photographed by the photographing unit.

  In addition, when there are a plurality of viewers during the conference, the multimodal recognition unit 3 adds the weight and updates the importance. The audio / image recording unit 4 records audio signals and video signals obtained during the conference. The database 5 records the importance identified by the multimodal recognition unit 3 as audio and image meta information recorded in the database 5. By using this meta information at the time of reproduction, an important part can be easily searched.

  The playback / use unit 6 is a video playback device that can easily fast forward and rewind. The usage recording unit 7 records a portion viewed by a person related to the conference without fast-forwarding or skipping as a usage record. The usage recording unit 7 provides importance level information to the multimodal recognition unit 3. Here, the importance level information is information obtained from the operation when watching the video, such as a part that seems to be important, such as a recorded video being repeatedly viewed, or a part that is often skipped by fast-forwarding. . Also, it is possible to label important parts manually and use it as importance level information. The multimodal recognition unit 3 re-learns the function for calculating the importance weight according to the importance information from the usage recording unit 7 and increases the accuracy.

  Next, the multimodal recognition unit 3 will be described. FIG. 2 is a block diagram illustrating the configuration of the multimodal recognition unit 3 according to the first embodiment. As shown in FIG. 2, the multimodal recognition unit 3 includes a voice recognition unit 10, an image recognition unit 20, an unsupervised learning control unit 30, a supervised learning control unit 40, and an information integration unit 50. In the following, recognition and detection are not particularly distinguished because only common terms are used.

  The voice recognition unit 10 extracts a feature amount using a technique for recognizing a feature amount from acoustic information based on video information photographed by a photographing unit. The voice recognition unit 10 counts information that is likely to be related to the importance level from the voice signal by using a predetermined voice recognition technique. The speech recognition unit 10 includes a sound source estimation unit 11, an utterance unit detection unit 12, an utterance collision detection unit 13, a speech speed detection unit 14, a pitch / volume detection unit 15, a silence interval detection unit 16, a speech detection unit 17, and applause detection. The unit 18 is provided. The sound source estimation unit 11 estimates a sound source position using a microphone array or the like. Note that the sound source may be estimated using the face position in the image, or the sound source may be estimated by sound and image information fusion.

  The utterance unit detection unit 12 detects an utterance based on the start and end of the utterance. The utterance collision detection unit 13 detects an utterance collision from the utterance detection of a plurality of speakers. Here, the speaker collision is an overlapping portion on the time axis. The speech speed detection unit 14 can detect from phoneme recognition and the number of phonemes per unit time. The pitch / volume detector 15 detects the pitch and volume from the audio signal during the meeting. The pitch / volume detection unit 15 detects the pitch using, for example, a basic frequency (FO) such as an accent phrase. Further, the pitch / volume detector 15 detects the volume from the energy.

  The silent section detector 16 obtains a section where no one speaks from the speech detection. The utterance detection unit 17 can recognize using word spotting such as “um” or “no”. The applause detector 18 detects applause from the sound using a voice recognition technology. Note that applause may be detected from the image. Note that the speech recognition unit 10 does not perform speech recognition that involves syntactic analysis, but only performs recognition that can be easily implemented, such as pitch and voice volume detection, time density of phonemes, and verbal excitement.

  The image recognizing unit 20 extracts a feature amount using a technique for recognizing a feature amount from image information based on video information photographed by a photographing unit. Further, the image recognition unit 20 counts information that seems to be related to the importance obtained from the image signal by using a predetermined image processing technique. The image recognition unit 20 includes a face detection unit 21, an action detection unit 22, and a position detection unit 23 that detects the position of each person. The face detection unit 21 detects the face state of each person using a conventional image processing technique. The action detection unit 22 detects actions such as standing, movement of a blackboard, gestures, and the like. The position detection unit 23 detects the position of each person.

  The voice quality / face identification unit 25 performs speaker identification using voice speaker identification or face identification. The speech recognition unit 10 and the image recognition unit 20 can recognize or detect other items as the recognition technology advances. In actual operation, newly recognized information can be added and integrated. The unsupervised learning control unit 30 controls unsupervised learning, and forms a map generated from the simultaneous occurrence probability of a plurality of inputs by SOM (self-organizing map) or the like. The unsupervised learning control unit 30 performs unsupervised learning from the co-occurrence probability based on information obtained from the speech recognition unit 10 and the image recognition unit 20.

  The supervised learning control unit 40 learns the relation based on the information integrated by the information integration unit 50 and the importance information from the usage recording unit 7. For this learning, for example, a hierarchical neural network can be used. However, it is assumed that a relation with a detection result that is likely to be related to the degree of importance is generally given as an initial value based on analogy, and a corresponding utility can be obtained even with the initial value. Based on this supervised learning, even if each recognition part is misrecognized, it is possible to obtain an association with the importance based on misrecognition. can do. That is, the importance can be recognized more accurately based on actual data. Once learning is completed, the importance can be estimated from the integrated information, and further learning can be performed using the importance as an initial value. As a result, the learning result is gradually accumulated as it is used, and the importance can be estimated.

  The information integration unit 50 totals the feature amounts extracted by the voice recognition unit 10 and the image recognition unit 20 in units of utterances, for example, and estimates the state of the space photographed by the photographing unit based on the summation result. For example, the information integration unit 50 estimates the state of the space photographed by the photographing unit by associating the aggregation result with a predetermined importance. Next, the information integration unit 50 assigns an index to the video information based on the estimated state of the space photographed by the photographing unit. This makes it possible to easily look back on important parts of the conference, for example.

  In addition, the information integration unit 50 collects and integrates information from the voice recognition unit 10, the image recognition unit 20, other sensor information 60, the unsupervised learning control unit 30, and the supervised learning control unit 40, so that time Create an importance graph and send the created time / importance graph to the database 5. In the process of collecting, for example, the sound source (mouth) position of the speaker can be estimated and used as basic information for face detection in sound source estimation, or the mouth position can be used as a sound source estimation candidate. More robust identification is possible by coordinating face identification from the detected face and voice quality identification from the voice. Thus, by mutually using the results of the respective recognition elements, it is possible to integrate and obtain a plurality of information.

  Next, the operation of the system will be described. FIG. 3 is an operation flowchart of the system according to the first embodiment. In step S <b> 11, the supervised learning control unit 40 learns the association based on the information integrated by the information integration unit 50 and the importance level information from the usage recording unit 7. In step S12, an audio signal during the conference is input from the microphone. The image of the scenery during the meeting is input from the digital camera. In step S13, the voice recognition unit 10 and the image recognition unit 20 calculate importance weights for each time.

  In step S14, the information integration unit identifies the important part of the conference by counting the importance weights for each utterance. In step S15, the importance level identified by the information integration unit 50 is recorded in the database 5 as audio and image meta information. By using this meta information at the time of reproduction, an important part of the conference can be easily searched.

  To summarize this embodiment, the feature amount of the acoustic information can be easily recognized by the current technology, such as voice volume, sound source direction, voice pitch, utterance speed, utterance collision detection, speaker change, silence, In addition, speaker identification, applause, etc., which can be recognized practically to some extent, are included. Also, the feature amount from the image information includes face, face orientation, line-of-sight direction, identification of person from face, raising hand, standing, board writing, writing, body movement, lighting brightness, projector screen image switching , Admission, leaving, blinking frequency, bottle bow and so on. These recognition results are aggregated and associated with a degree of importance determined in advance (or determined by another method) (learning). Based on this association, the importance level is estimated from the feature amount of the sound or the image.

  Although it is not impossible in the future to recognize the content of the story and the interests of the viewers and weight the importance, it is not possible with the current technology. Therefore, the present invention estimates importance by collecting and learning a large amount of information that can be acquired, in particular, human behavior having advanced recognition functions.

For example,
・ When there is "(1) interesting story", there is a habit of "(2) raising your face".
・ Do “(1) The story is boring” and “(2) Poverty slurp”.
・ "(1) Before speaking""(2) Waiting with your mouth open."
・ "(1) Discussions boil" and "(2) Speaker changes occur frequently."
・ "(1) Discussion is stagnant" and "(2) Time when no one speaks" increases.
The multimodal recognition unit 3 recognizes the above (2) and totals them, thereby estimating the above (1) and adding metadata. Individual recognition is not useful, such as talk is boring or discussions boil, but if someone who is good at assessing `` importance '' shakes meta information to the important part while watching the video A certain degree of association between the “recognizable event” and the “importance” is made possible by learning.

  In addition, many people who watch the recorded video, many people were watching "President's story", but by observing that no one saw D-san's remarks, It can be inferred that the scene in which the president speaks seems to be more important than the scene in which it is speaking. From these facts, the importance can be estimated from information that can be recognized as far as technically possible, without recognizing what is specifically spoken.

  In addition to the “hand label” that is input manually while watching the voice and image, which is being studied in speech recognition research, the importance level is not repeated or fast-forwarded when playing back recorded video. Can be automatically determined and determined by something like "viewing rate". By repeatedly associating these, the degree of importance can be obtained from the feature quantity inductively.

  Next, a second embodiment will be described. FIG. 4 is a diagram illustrating an overall configuration of a system according to the second embodiment. As shown in FIG. 4, the system 100 includes a sensor 2, a multimodal recognition unit 103, an audio / image recording unit 4, a database 5, and a reproduction / use unit 6. In addition, the same code | symbol is attached | subjected and demonstrated about the same location. FIG. 5 is a block diagram illustrating a configuration of the multimodal recognition unit 103 according to the second embodiment.

  As shown in FIG. 5, the multimodal recognition unit 103 includes a voice recognition unit 10, an image recognition unit 20, an information integration unit 50, and a position / position input unit 70. The voice recognition unit 10 extracts feature amounts using a technique for recognizing feature amounts from acoustic information based on video information photographed by a photographing unit. In addition, information that seems to be related to importance is counted from the voice signal by using a predetermined voice recognition technique. The speech recognition unit 10 includes a sound source estimation unit 11, an utterance unit detection unit 12, an utterance collision detection unit 13, a speech speed detection unit 14, a pitch / volume detection unit 15, a silence interval detection unit 16, a speech detection unit 17, and applause detection. The unit 18 is provided. The image recognizing unit 20 extracts the feature amount using a technique for recognizing the feature amount from the image information based on the video information photographed by the photographing unit. The image recognition unit 20 counts information that is likely to be related to the importance obtained from the image signal using a predetermined image processing technique. The position / position input unit 70 manually inputs information related to the position / position of the speaker. Information can be given in advance from the position / position input unit 70.

  The speech recognition unit 10 and the image recognition unit 20 extract the number of uttered words, the number of uttered phonemes, the number of appearances of impression verbs, the amount of voice, the time change of the voice amount, the change of phoneme pitch, and the face inclination as feature amounts.

Generally, a key person has the following characteristics.
・ Large utterance time.
・ There is little stagnation during speaking.
・ Sound volume is high and changes are small.
・ There is relatively little pitch change of voice.
・ Position / position is high.
・ Participants often perform writing operations during keyperson utterances.

On the other hand, if you are not a key person,
・ There are few words for speaking time.
・ There are many good stagnations, and there are many impression verbs (um…, that…).
・ Voice volume is small and change in voice volume is large.
・ Voice pitch change is large.
・ Position / position is low.

  Therefore, the information integration unit 50 aggregates the extracted feature amounts, and estimates a key person in the space photographed by the photographing unit based on the summation result. In addition, the information integration unit 50 considers that the number of utterance words (the number of phonemes is acceptable), the verbal excitement verb, the voice volume, the change rate of the voice volume, the voice pitch change, the face inclination (related to the position within a predetermined section. Is converted with a predetermined weight, and an importance coefficient for each scene is calculated. The information integration unit 50 estimates a key person for each scene based on the importance coefficient. It should be noted that the coefficient can also be derived from the exchange of e-mails and included in consideration. The weight of a specific person can be added by a manual operation or the like, or a specific person can be set as a key person.

  Next, the operation of the system according to the second embodiment will be described. FIG. 6 is an operation flowchart of the system according to the second embodiment. In step S21, an audio signal during the conference is input from the microphone. The image of the scenery during the meeting is input from the digital camera. In step S22, the information integration unit 50 converts the number of uttered words, excitement verbs, voice volume, change rate of voice volume, change in voice pitch, face inclination, and the like with predetermined weights within a predetermined section. Calculate the importance factor.

  In step S23, the information integration unit 50 estimates a key person for each scene based on the importance coefficient. In step S24, the information integration unit 50 records the importance coefficient in the database 5 as audio and image meta-information. By using this meta information at the time of reproduction, the key person can be easily searched.

  According to the second embodiment, in a brainstorming type meeting, even if the key person cannot always be identified from the prior information, the key person can be estimated later. In addition, it is possible to cope with the characteristic that the key person is switched for each scene.

  The information processing method according to the present invention is realized using, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and the program is stored in a hard disk device, a CD-ROM, or a DVD. Alternatively, each step is realized by installing from a portable storage medium such as a flexible disk or downloading from a communication circuit and the CPU executing this program. This program extracts the feature amount using at least one of a technique for recognizing a feature amount from acoustic information and a technique for recognizing a feature amount from image information based on video information photographed by a photographing means, Causing the computer to execute a step of totalizing feature amounts and a step of estimating a state of a space photographed by the photographing unit based on the result of the feature amount summation.

  Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows the whole structure of the system which concerns on 1st Example. It is a block diagram which shows the structure of the multimodal recognition part 3 which concerns on 1st Example. It is an operation | movement flowchart of the system which concerns on 1st Example. It is a figure which shows the whole structure of the system which concerns on 2nd Example. It is a block diagram which shows the structure of the multimodal recognition part 103 which concerns on 2nd Example. It is an operation | movement flowchart of the system which concerns on 2nd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,100 System 2 Sensor 3,103 Multimodal recognition part 4 Voice, image recording part 5 Database 6 Playback | regeneration / use part 7 Usage recording part 10 Voice recognition part 20 Image recognition part 30 Unsupervised learning control part 40 Supervised learning control part 50 Information Integration Department 70 Position / Position Input Department

Claims (16)

Extraction means for extracting the feature quantity using at least one of a technique for recognizing the feature quantity from the acoustic information and a technique for recognizing the feature quantity from the image information based on the video information photographed by the photographing means;
Tally means for tallying the feature quantities extracted by the extracting means;
An information processing apparatus comprising: an estimation unit configured to estimate a state of a space photographed by the photographing unit based on a counting result obtained by the counting unit. The information processing apparatus according to claim 1, wherein the estimation unit estimates a state of a space photographed by the photographing unit by associating a totaling result obtained by the totaling unit with a predetermined importance. The information processing apparatus according to claim 1, wherein the estimating unit estimates a key person in a space photographed by the photographing unit based on a counting result by the counting unit. The estimation means includes a part where the key person speaks, a part where the speech speed is faster than a predetermined speed, a part where the voice is larger than a predetermined level, a part where the pitch is higher than the predetermined pitch, a part after raising a hand, After that, the part where there is a change of the speaker a predetermined number of times within the predetermined period, the part where the person is standing, the part where the person who is writing on the board is present, the part where the private language is less than the predetermined number and the utterance 2. The information processing apparatus according to claim 1, wherein at least one of a portion in which there is a section that does not speak for a predetermined section after is estimated as a state of a space photographed by the photographing unit. Recording means for recording a usage history of a playback device for playing back the video information;
The information processing apparatus according to claim 2, further comprising: a control unit that learns a function for calculating the importance based on a usage history of the playback apparatus. The extraction means extracts at least one of the number of utterance words, the number of utterance phonemes, the number of appearances of impression verbs, the volume of voice, the time change of voice volume, the change of phoneme pitch, and the inclination of the face as the feature quantity. The information processing apparatus according to claim 1. The technology for recognizing the feature amount from the acoustic information is at least one of sound source estimation, utterance unit detection, utterance collision detection, speech speed detection, pitch detection, sound volume detection, silence interval detection, sound quality identification, speech detection and applause detection. The information processing apparatus according to claim 1, comprising one. The information processing apparatus according to claim 1, wherein the technique for recognizing a feature amount from the image information includes at least one of face detection, action detection, and position detection. The information processing apparatus according to claim 1, wherein the totaling unit totalizes the feature amount in utterance units. The information processing apparatus according to claim 1, further comprising an attaching unit that assigns an index to the video information based on a state of a space photographed by the photographing unit estimated by the estimating unit. An extraction step for extracting the feature quantity using at least one of a technique for recognizing the feature quantity from the acoustic information and a technique for recognizing the feature quantity from the image information based on the video information photographed by the photographing means;
A totalizing step for totalizing the feature quantities;
An information processing method comprising: an estimation step of estimating a state of a space photographed by the photographing means based on the total result of the feature amount. The information processing method according to claim 11, wherein the estimating step estimates a state of a space photographed by the photographing unit by associating the total result of the feature amount with a predetermined importance. The information processing method according to claim 11, wherein the estimating step estimates a key person in a space photographed by the photographing unit based on the total result of the feature amount. Recording a usage history of a playback device for playing back the video information;
The information processing method according to claim 11, further comprising a step of learning a function for calculating the importance based on a usage history of the playback device. The information processing method according to claim 11, further comprising a step of assigning an index to the video information based on a space state photographed by the estimated photographing unit. Extracting the feature amount using at least one of a technology for recognizing the feature amount from the acoustic information and a technology for recognizing the feature amount from the image information based on the video information photographed by the photographing means;
Summing up the feature quantities;
A program for causing a computer to execute a step of estimating a state of a space photographed by the photographing unit based on a result of counting the feature values.

Images