WO2006098255A1 - Image display method and device thereof - Google Patents

Abstract

There are provided an image display method and an image display device capable of effectively enhancing the attractive force of a viewer viewing the image of the display device. The image display method selects a content material corresponding to a particular image element in accordance with motion of a person captured as recognition information in a display image consisting of a plurality of image elements which can be displayed independently from one another, combines the image element formed by the selected content material with the residual image element of the initial display image, and further extracts the image of the person captured as the recognition information and combines it for display.

Description

 Specification

 Image display method and apparatus

 Technical field

 TECHNICAL FIELD [0001] The present invention relates to an image display method and apparatus, and more particularly, to a display method and apparatus for advertisement information or guidance information or video information.

 Background art

 [0002] Advertisements using small-screen displays are seen everywhere in vehicles and in stores. In recent years, advertisements using large-screen displays have been seen on streets.

 These advertisements (hereinafter sometimes referred to as electronic advertisements) are performed by sequentially displaying image group data of still images or moving images prepared in advance according to a schedule.

 As a result, the latest advertisement, which was insufficient in the case of paper advertisement, can be realized in a timely and troublesome manner.

[0003] However, since an electronic advertisement is a unilateral flow of a still image or a moving image prepared in advance as described above, a viewer of the electronic advertisement (hereinafter referred to as a viewer). Is only passively watching ads.

 For this reason, even if an advertisement with an excellent visual effect that tends to remain in the impression is circulated, there is a limit to attracting viewers' attention and obtaining a high advertising effect.

[0004] In order to improve this point, a method for attracting the attention of a viewer and obtaining a high advertising effect has been proposed.

 [0005] For example, an electronic advertisement system that re-creates an advertisement display schedule by looking at an advertisement display terminal and sending a call or e-mail including an access key when a viewer is displayed in an advertisement! Has been proposed (see Patent Document 1;). According to this method, only the viewer who actually sees the advertisement can change the advertisement display data, so that the viewer can be more interested in the advertisement.

[0006] Further, for example, an advertisement is distributed from a remote place to a display device included in a moving body such as a train. In this case, a method has been proposed in which advertisements to be displayed are changed depending on the traveling position and time of the moving body (see Patent Document 2). According to this method, it is possible to display an advertisement according to the travel position and the passenger class of the time, and the advertisement can be made effective.

 [0007] Furthermore, a method has been proposed in which an advertisement display terminal is provided with an imaging device and information obtained by imaging the viewer is reflected in the advertisement display.

For example, an advertising terminal (see Patent Document 3) that displays advertisement data corresponding to an individual behavior tendency obtained by an imaging device in an image (see Patent Document 3). An advertising effectiveness confirmation system that deploys a device such as an imaging unit that collects data, transmits and records the collected data to an analysis device via a communication line, and reproduces and outputs the collected data in a predetermined format when analyzing the advertising effectiveness (Patent Document 4) In addition, an interactive display device has been proposed in which the person attribute relating to the imaged viewer is determined, the person image is cut out, and the person attribute determination result is accumulated. (See Patent Document 5.) _{0 In} this case, the human image is cut out from a plurality of human templates prepared in advance as a human image.

 Patent Document 1: JP 2002-150130 A

 Patent Document 2: Japanese Patent Laid-Open No. 2003-157035

 Patent Document 3: Japanese Patent Laid-Open No. 2002-288527

 Patent Document 4: Japanese Patent Laid-Open No. 11-153977

 Patent Document 5: Japanese Unexamined Patent Publication No. 2000-105583

 Disclosure of the invention

 Problems to be solved by the invention

[0008] However, it is considered that the above-described conventional methods are not necessarily sufficient for V and deviation to attract viewers' attention and to obtain high advertising effects and the like. Moreover, it is necessary to create each content to display an image with time and effort, making content production difficult.

[0009] The present invention has been made in view of the above problems, and provides an image display method and apparatus capable of effectively increasing the attention level of a viewer who views an image on a display device. For the purpose.

 Means for solving the problem

 [0010] The present inventor creates and displays an image content in real time based on the trigger information using a human movement as a trigger, so to speak, the image content display is not displayed on the information transmission side but on the receiver side. The present inventors have found that it is possible to effectively increase the degree of attention of viewers who view images on a display device by performing interactive display with entrusted initiative.

 [0011] An image display method according to the present invention includes:

 Determining initial content of a display image composed of a plurality of image elements that can be displayed independently of each other, generating and displaying an initial display image; and

 A recognition information acquisition process for capturing people as recognition information;

 A motion detection process for detecting the movement of a person captured as recognition information;

 A content selection step of selecting content material corresponding to a specific image element corresponding to movement or selecting image content related to the initial display image;

 The selected image element of the content material and the remaining image element of the initial display image are combined, or the image of the image content is replaced with the initial display image, and also the human image captured as recognition information Image synthesis processing step of cutting out the image as it is or synthesizing a human substitute image as an image element,

 A combined image display step for displaying the combined image;

 Have

 The recognition information acquisition process, motion detection process, content selection process, image composition processing process, and composite image display process are repeated.

 [0012] Further, the image display method according to the present invention is characterized in that, in the motion detection step, a motion at the specific image element position is detected.

[0013] In addition, the image display method according to the present invention selects a sound content corresponding to a motion or a specific image element in the content selection step, and the composite image display step in the composite image display step. At the same time, or independently of the synthesized image, the sound having the sound content power is output. [0014] Further, the image display method according to the present invention is characterized in that the content of the display image is advertisement information, guidance information, or exhibition information provided to a person captured as recognition information.

[0015] Further, in the image display method according to the present invention, the content of the display image is video information that provides a person who views the display image with the performance of the person captured as the recognition information. .

[0016] Further, an image display device according to the present invention includes:

 An image display device that determines an initial image content of a display image having a plurality of image element forces that can be displayed independently of each other, generates an initial display image, and displays the initial display image.

 Recognition information acquisition means for capturing a person as recognition information;

 Motion detection means for detecting the movement of a person captured as recognition information;

 A content selection unit that selects content material corresponding to a specific image element in response to movement or image content related to an initial display image;

 The selected image element of the content material and the remaining image element of the initial display image are combined, or the image of the image content is replaced with the initial display image, and also the human image captured as recognition information Image synthesis processing means for cutting out and synthesizing a substitute image of a person as an image element;

 Combined image display means for displaying the combined image;

 It is characterized by having.

[0017] Further, the image display device according to the present invention is characterized in that the motion detecting means detects a motion at the specific image element position.

[0018] Further, in the image display device according to the present invention, the content selection unit selects sound content corresponding to a specific image element, and the composite image display unit includes the composite image or the composite image. Independently, the sound having the sound content power is output.

[0019] Further, the image display device according to the present invention is characterized in that the content of the display image is advertisement information, guidance information, or exhibition information provided to a person captured as recognition information.

[0020] Further, in the image display device according to the present invention, the content of the display image is video information that provides a person who views the display image with the performance of the person captured as the recognition information. . The invention's effect

 [0021] The image display method and apparatus according to the present invention are selected as a display image composed of a plurality of image elements that can be displayed independently of each other, for example, corresponding to the movement of a person captured as recognition information In addition to synthesizing the image elements that are the content material power and the remaining image elements of the initial display image, and further, for example, cutting out and synthesizing and displaying the human image captured as the recognition information, The degree of attention when the three parties look at the displayed image can be increased.

 In addition, by repeatedly switching the content according to the movement of the person, the movement of the person is reflected in the content in real time and the image keeps changing, so that the time of attention can be kept longer.

 Brief Description of Drawings

 FIG. 1 is a diagram showing a schematic configuration of an image generation apparatus of the present invention.

 FIG. 2 is a diagram for explaining an example in which a changed portion is specified by a difference between two consecutive frames.

 FIG. 3 is a diagram for explaining the “direction of motion” and the “accumulated amount of motion” obtained from the accumulation of frame differences.

 FIG. 4 is a diagram for explaining a method for recognizing the state of a subject by pattern matching with an image database.

 FIG. 5 is a diagram for explaining a method of searching for an image pattern in the image database that matches the positional relationship with the image of the subject and obtaining it as image recognition information.

 FIG. 6 is a diagram for explaining a method of searching for a sound database having a sound and a characteristic that matches the received sound and obtaining it as sound recognition information.

 [Fig.7] Method for selecting and generating image information for image B when image A is at the top, and image D when image B is at the right for image B It is a figure for demonstrating.

 FIG. 8 is a diagram for explaining a method of generating image information according to the direction of movement of a subject.

FIG. 9 is a flowchart for explaining an image display method of the present invention. FIG. 10 is a diagram for explaining a hierarchical structure of a plurality of contents stored in an image database and image contents included in each content image.

 FIG. 11A is a flowchart for explaining a procedure for generating a display image using human movement as trigger information.

 FIG. 11B is a flowchart for explaining in detail the procedure of content determination processing in FIG. 11A.

 FIG. 12 is a flowchart for explaining a procedure for outputting sound together with an image.

FIG. 13 is a diagram showing an image display example in the case of a beer advertisement.

 FIG. 14 is a diagram for explaining the relationship between the position of human movement and image switching in the case of beer advertisements.

 FIG. 15 is a diagram for explaining the relationship between the position of human movement and sound in the case of beer advertisements

[Fig. 16] This is a diagram for explaining the situation when a person is performing as a video and displaying a performance.

 FIG. 17 is a diagram for explaining the relationship between the position of human movement and screen image switching when displaying an image.

 FIG. 18 is a diagram for explaining the relationship between the position of human movement and sound when displaying video. Explanation of symbols

 10 Image display device

 12 Receptor

 12a Image sensor

 12b sound sensor

 14 Main control unit

 16 Output section

 16a display

 16b sound equipment

 18 Recognition Information Processing Department

20 Image information generator 22 Sound information generator

 24 clocks

 26 Image Information Processing Department

 28 Sound Information Processing Department

 30 image database

 32 Image data determination unit

 34 sound database

 36 Sound data decision section

 40a-40c, 48a, 48b content

 44a ~ 44c, 50a ~ 50c Image content

 44a-44c, 52a-52d Image elements

 46a-46c, 54a-54c content material

 BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of an image display method and apparatus according to the present invention will be described below with reference to the drawings.

 First, an image display apparatus according to the present invention will be described with reference to FIG.

 The image display apparatus 10 according to the present invention includes a receiving unit (recognition information acquisition means) 12 that captures external information, particularly a person as recognition information, a main control unit 14, and an output unit 16. The receiving unit 12 and the output unit 16 may be integrated with the main control unit 14.

The receiving unit 12 includes, for example, an image sensor (imaging unit) 12a and a sound sensor (sound receiving unit) 1

Either one or both of 2b.

 As the former image sensor 12a, an ordinary camera, an infrared camera, an infrared sensor, or a 3D camera such as binocular parallax can be used, thereby receiving external information such as human behavior (sensing). To do. In particular, by using a 3D camera, it is possible to obtain information on temporal changes in images with distance information.

 As the latter sound sensor 12b, a sound sensor such as a microphone can be used, thereby receiving external information such as a human voice.

The receiving unit 12 is a vibration sensor that recognizes vibrations that are struck or touched. It may be a Doppler sensor that recognizes the temperature, a sign, etc., a temperature sensor that recognizes the temperature by thermography, or a pressure sensor that recognizes the force level.

 The output unit 16 includes a display device (synthetic image display means) 16a and an acoustic device (sound output means) 16b. The display device 16a is, for example, a display, a projector, or a television, and the acoustic device 16b is, for example, a speaker or headphones.

 The main control unit 14 includes a mechanism that generates output information according to the information obtained by the receiving unit 12 and outputs the output information to the output unit 16.

 The main control unit 14 is, for example, a computer, and includes a recognition information processing unit 18, an image information generation unit 20, a sound information generation unit 22, and a clock 24.

 The recognition information processing unit 18 includes an image information processing unit (motion detection means) 26 and a sound information processing unit 28.

 The image information processing unit 26 processes the person information acquired by the image sensor 12a and generates a trigger signal used for selecting an output image. The sound information processing unit 28 processes the external information received by the sound sensor 12b, and generates a trigger signal used for selecting sound information. Sound information obtained based on this trigger signal is associated with external information. Apart from this, the ability to select sound content corresponding to a specific image element or the like will be described in detail later. Note that the sound information and the sound content have the same meaning, but the triggers for generating them are different as described above.

 [0030] The image information processing unit 26 generates a subject image (chroma single image) for synthesis in the form of a person (hereinafter, sometimes referred to as a subject! /) That is imaged by the image sensor 12a. For this purpose, it is output as it is, or it is output after image processing of outlines and shadows. Alternatively, a trigger signal for generating a substitute image of a subject such as a character image is output. Further, the image information processing unit 26 performs image processing on the subject imaged by the output image sensor 12a, detects the motion of the subject, and outputs a trigger signal for selecting image content.

[0031] The subject movement is identified by a frame difference such as a difference between two consecutive frames, a difference between three consecutive frames, or a difference from the background. For example, the difference between two consecutive frames As shown in Fig. 2, the coordinate force of the part that changed between the two frames Fla and Fib, as shown in Fig. 2, detects the "movement position" as shown in the frame Flc, or the part that changed The “amount of movement” is detected from the size of.

 The former “position of movement” has changed by, for example, comparing the frames before and after one frame is divided into a predetermined number of divided frames and capturing the change in luminance of the image of the divided frames. This is done by specifying the position of the divided frame. That is, the brightness of all the pixels (pixels) of the image of the current frame and the image of the previous frame (or background image) is compared from the input image data. It is determined that the pixel is in motion, and at this time, it is determined to which partition (divided frame) the center of gravity (X coordinate, Y coordinate) of the pixel group with motion belongs. Then, the “position of movement” is detected according to the position of the vertical and horizontal sections in the frame.

 Regarding the latter “amount of motion”, for example, in the above, from the input image data, the brightness of all the pixels (pixels) of the image of the current frame and the image of the previous frame (or the background image). If the difference is equal to or greater than a predetermined value, it is determined that the pixel is in motion, and at this time, “amount of motion” is detected according to the number of pixels in motion.

 Here, as the movement of the subject, for example, as shown in FIG. 3, the “direction of movement” shown in the frame F2d and the “cumulative amount of movement” shown in the frame F2e, which are obtained by accumulating the frame differences of the three frames F2a to F2c. "

 In addition, by adding a time element, the trigger information is based on the movement of the subject (quality of movement, or characteristics of movement) of at least one of the position of movement, direction of movement, amount of movement and speed of movement. (Trigger signal) is generated.

 In addition, the motion vector (size and direction) at points on the screen can also be obtained by calculating the optical flow using the filtering method or the gradient method.

By using a 3D camera, it is possible to recognize the movement of these subjects, for example, only the movement of a person within 2 meters in front of the camera, and it is possible to exclude distant backgrounds and crowds. Similarly, using an infrared camera, it is possible to recognize only within the reach of the infrared light or in dark places such as at night. [0033] The image information processing unit 26 may be configured to acquire, for example, the state of the subject imaged by the image sensor 12a as image recognition information.

 The state of the subject is specified by, for example, pattern matching with an image database (different from the image database 30 described later) provided in the image information processing unit 26 that does not appear in FIG.

 That is, for example, as shown in Fig. 4, if the image pattern in the image database is searched for a feature that matches the image of the subject, and the features of the suit and business bag match, it is said to be `` masculine '' The state is obtained as trigger information, or when the umbrella matches the feature, “the rain is falling” is obtained as the trigger information.

 Alternatively, for example, as shown in FIG. 5, when an image database image pattern that matches the positional relationship with the subject image is searched, and the relationship between the eyes, nose, and mouth is matched, Get a specific face!

 At this time, it is also possible to obtain the distinction between adults and children or the number of people as trigger information from the position of the face.

The sound information processing unit 28 processes, for example, external sound received by the sound sensor 12b, and generates trigger information (trigger signal).

 For example, the unit time waveform data of the received sound is analyzed by FFT (Fast Fourier Transform) to obtain the frequency distribution shape, and pattern matching with the sound data base not shown in FIG. Specified by.

 That is, for example, as shown in FIG. 6, the left side of FIG. 6 from among the sound patterns of high heels F 3a, leather shoes F3b, sneakers F3c, bag roller F3d, wheelchair F3e, rain F3f, etc. The sound that has the same characteristics as the F3g sound (sound pattern) F3g is searched, and it is determined that the component distribution shape of the input sound waveform matches the component distribution shape of the sound waveform of the high heel F3g. If it is, “feminine ヽ” t ヽ ぅ is obtained as trigger information.

Also, for example, the voice tone is analyzed by FFT, and when the low frequency component is high, the voice is a low voice, for example, a male voice, and when the high frequency component is high, the voice is a high voice, for example, a female voice. You can get it as ... The image information generation unit (content selection unit and image composition processing unit) 20 generates image information based on the trigger information generated by the image information processing unit 26.

 The image information generation unit 20 includes an image database 30 and an image data determination unit 32. The image database 30 stores a group of image information accumulated for displaying an image, for example, as an advertisement, a guide, or an exhibition!

 As guidance, for example, an image of a tourist's finger or full body image is displayed at a specific position on a map screen of tourist information installed at the airport, so that image information such as the landmarks and specialties of the land at that location can be displayed. Mention may be made of what is provided. Also, as an exhibition, for example, image content and sound content groups of various insects are embedded in an exhibition panel that explains the ecology of insects installed in museums, and the hand images of visitors are overlaid at specific positions on the screen. By doing so, you can list images that explain the ecology of the insects at that location and those that provide insect calls. A specific example of advertisement will be described later.

 That is, as shown in FIG. 10, the image database 30 includes, for example, a plurality of contents for advertising a product as the content layer PF 1 (in FIG. 10, for example, content 40a that is an advertisement for beer, cosmetics Only the content 40b, which is the advertisement of the other, and the content 40c, which is the other advertisement, are displayed) and multiple contents for providing video (only the contents 48a and 48b are displayed in Fig. 10). ing. Each content includes, as the image content layer PF2, a plurality of image contents that can be displayed independently of each other (in FIG. 10, only the image elements 42a to 42c of the image content 40a are displayed). Furthermore, in each image content, a plurality of image elements (in FIG. 10, only image elements 44a to 44c and 52a to 52c are displayed as the image element layer PF3. .) Is included. Each image element is associated with a content material (only the content materials 46a to 46c and 54a to 54c are displayed in FIG. 10) as the content material layer PF4.

 The images included in each of these layers PF1 to PF4 may be still images or moving images.

[0036] The above database structure is further provided in accordance with a specific example of an image display method to be described later. Explain physically.

 For example, when 40a, which is a beer advertisement, is selected and determined as content triggered by information such as the gender and belongings of the person who is the subject or environmental information around the subject, the initial image (initial display image) is, for example, Then, a display image (image content) 42a composed of a plurality of image elements 44a to 44c, such as a company logo and a beer can, is selected. Then, for example, a content material 46a that is a company introduction image associated with the image element 44a is selected by replacing a character image element 44a that is a company logo, for example, with the movement of a person as trigger information. On the other hand, the content material 46c is included in the sound data base 34 to be described later, and the sound content material corresponding to the image element 44c is selected. In the case of the content 48a of the video A, the image element 52d is associated with the image content 50b related to the content 48a, which is different from the image content 50a as the initial image, and the movement of the person in the image element 52d is represented. The image content 50b is selected by switching to the image content 50a as a trigger. The content material 54c is a sound content material.

The image database 30, for displaying the object in the substitute image, a plurality of keys Yarakuta first image content is stored, Ru (not shown.) ₀

 For example, when the cumulative amount of movement does not reach the threshold value, the image data determination unit 32 retains the image information (image, initial image content) of the image group as it is or switches according to the display schedule. indicate. On the other hand, when the cumulative amount of motion is equal to or greater than the threshold, for example, as shown in FIG. 7, the images F4a, F4b,... Associated with the trigger information set according to the position of the motion described above. For example, select the image information for the image F4b if the movement position is at the top for the image F4a, and the image F4d for the movement position for the right portion for the image F4b. To do.

Further, the image information group includes, for example, a plurality of pieces of image information associated with the direction of movement, and the image data determination unit 32 moves from the system installation location F 5c as shown in FIG. 8, for example. When the direction is (1), image information related to the advertisement F5a existing in the direction is generated. When the direction is (2), image information related to the advertisement F5b existing in the direction is generated. Further, when referring to the image content group in FIG. 10, as described above, the image data determination unit 32 is associated with the previous trigger information by the new trigger information set according to the position of the motion. For example, instead of the image element content 42a, the content material 44a is selected to generate image information.

 [0037] At this time, as the image information group, an image group set according to the state of the subject described above is configured, and for example, the image data determination unit 32 corresponds to the image recognition information "masculine". Select and generate advertising image information for men, or select and generate information for advertising images of that brand according to the subject's appearance as a specific brand mark.

 Further, the image information generation unit 20 may be configured to generate image information based on the trigger information obtained by the sound information processing unit 28!

 That is, the image database 30 stores image information associated with the trigger information obtained by the sound information processing unit 28 (not shown), and the image data determination unit 32 is based on the trigger information. The image information corresponding to the trigger information is selected and generated.

[0039] As described above, the image information generation unit 20 may be configured to use the image information received by the image sensor 12a as generated image information and to superimpose it on the current display image.

 For example, the image is thinly superimposed on the image, or you are entering the screen, or your shadow is reflected in the image, like the chroma effect, or your image The image information is generated so that the image is displayed only where it moves.

 Further, the image information generation unit 20 may be configured to select and generate a character image according to the trigger information of the subject.

[0040] In addition, the image information generation unit 20 uses words (characters) such as “hot” when the temperature is high, or “recommended for you who is beautiful” when recognizing that it is a woman. It may be configured to be added and displayed.

[0041] The sound information generation unit (sound content selection means) 22 generates sound information based on the trigger information obtained by the sound information processing unit 28, or responds to the movement of the subject, specific image elements, and the like. Thus, the selected sound content is generated. The sound information generation unit 22 includes a sound database 34 and a sound data determination unit 36.

 The sound database 34 stores sound information corresponding to the trigger information obtained by the sound information processing unit 28, and the sound data determination unit 36 selects sound information corresponding to the trigger information based on the trigger information. , Generate.

 For example, when the trigger information is an adult male, the image information on which beer is poured is generated, and the sound of beer pouring or foam is selected and generated as sound information, or the trigger information For example, when a person is a woman, sound information is selected and generated for a calling word such as “lady”.

 In this case, the sound picked up by the microphone may be generated as it is as sound information.

 In addition, the sound database 34 includes sound material data associated with the respective content materials 46a to 46c of the image information generation unit 20 or associated with the image elements 44a to 44c (not shown). O) Then, the sound data determination unit 36 selects and generates an audio material corresponding to the image trigger signal from the image information processing unit 26, in other words, the image element.

Next, the image display method of the present invention will be outlined with reference to FIG. 9, taking the case of image display as an example.

 At a predetermined timing (S10 in FIG. 9), external information (human information) is taken in (S12 in FIG. 9) and recognition information is acquired (S14 in FIG. 9).

 Next, based on the recognition information, image information corresponding to the recognition information is generated (S16 in FIG. 9). Then, the image display is performed according to the image information signal (S18 in FIG. 9).

The image display method of the present invention will be described in more detail with reference to FIGS. 11A and 11B, taking as an example the case where a display image is generated using human movement as trigger information.

First, as shown in FIG. 11A, content is determined (S20 in FIG. 11A). The content may be determined in advance, but is more preferably performed according to the following procedure. That is, as shown in the detailed step configuration in FIG. 11B, an image of a person from the camera is acquired (S48 in FIG. 11B), and the first image data determination unit 32 via the image information processing unit 26 It is determined whether or not the image processing is performed (S50 in FIG. 11B). In the case of the first image processing, the acquired image is held in the temporary image holding area (S52 in FIG. 11B). The image of the person from the camera is acquired again (S48 in FIG. 11B). On the other hand, if it is not the first image processing, in other words, if it is already an image processing is repeated, (in FIG. 11B, S54) extracts a human manner and wherein ₀ Then, to extract the human situation and characteristics In this case, the information is selected (determination process) in the image data determination unit 32 via the image information processing unit 26 (S56 in FIG. 11B) using the person's state and feature information as a trigger. The content is, for example, a beer advertisement (advertisement information). In this case, the sound information picked up by the microphone may be used as a trigger, and the content may be determined by the image data determination unit ₃₂ via the sound information determination unit ₂₈ (S56 in FIG. 11B). On the other hand, if a person's appearance and features cannot be extracted, an image of a person with camera power is acquired again (S48 in FIG. 11B).

 When the content is determined (S20 in FIG. 11A), an image of a person from the camera is acquired (S22 in FIG. 11A), and the image data determination unit 32 passes through the image information processing unit 26. Then, it is determined whether or not it is the first image processing (S24 in FIG. 11A).

 In the case of the first image processing, the acquired image is held in the temporary image holding area (S26 in Fig. 11A) and consists of characters and figures such as company logos and beer cans that make up the content. An initial image composed of a plurality of image elements (initial display image, initial image content) is generated and displayed (S28 in FIG. 11A), and a human image from the camera is acquired again (Fig. 11). 11A, S22). On the other hand, when the first image processing is not performed, in other words, when the image processing has already been repeated, the movement of the person is detected by comparing the temporary image holding image with the current camera image (in FIG. 11A). , S30), and extract the image of the moving part (moving person) (S32 in Fig. 11A).

 Next, it is determined whether there is a movement designated at a specific position in the current image (S34 in FIG. 11A). At this time, it may be determined whether or not there is a designated movement at an arbitrary position in the current image. Here, the designated movement is, for example, the amount of movement equal to or greater than a threshold, movement in a specific direction, repetition of movement, movement in order between specific coordinates, and the like.

If there is a specified motion, the selected content material corresponding to the specific image element at the position where the motion is selected is selected to generate an image (S36 in FIG. 11A). The image is replaced and displayed, or is replaced with an image of a specific image element and is combined with other image elements and displayed (S38 in FIG. 11A). On the other hand, if there is no specified movement, If it does not move for a fixed time, it is determined whether or not the state continues (S35 in FIG. 11A).

 Then, if the stationary state continues, the content is determined again (S20 in FIG. 11A). On the other hand, if there is movement, the current image is displayed as it is, and an image of a person with camera power is acquired again (S22 in FIG. 11A).

[0045] By repeating the above steps, the image of the display device or the content of the specific image element changes one after another according to the movement of the person captured by the camera, so that the person or display device captured by the camera is viewed. Third-party eyes are attracted to the display image of the image display device.

 [0046] In addition, in the image display method of the present invention, when sound is output together with an image or independently of the image, as shown in FIG. After extracting the image of the person (moving person) (S32 in Fig. 11A), it is determined whether there is a force specified at a specific position in the image (S40 in Fig. 12). At this time, it may be determined whether or not there is a designated movement at an arbitrary position in the image. Here, the designated movement is, for example, the amount of movement equal to or greater than a threshold value, movement in a specific direction, repetition of movement, movement in order between specific coordinates, and the like as described above.

If there is a specified movement, the specified sound material is selected from the material collection and a sound is generated (S44 in FIG. 12). For example, the generated sound is output by matching the timing of the image and output. Yes (S46 in Fig. 12) ₀ On the other hand, if there is no specified movement, the current sound state is maintained with the current sound as it is, or when there is no sound (S46 in Fig. 12). As a result, the sound changes one after another according to the movement of the person captured by the camera, and a new sound is created by overlapping the sounds. The eyes of the third party who sees are attracted to the display image of the image display device.

 [0047] Here, the present invention will be described in detail by taking beer advertisements as an example.

For example, an image as shown in FIG. 13 is displayed on the display device, and a passerby views this advertisement. In addition, a camera (indicated by arrow A in FIG. 13) is attached to the display device, and a person who passes in front of the display device is captured as an image, for example, the shadow of the person is combined with an advertisement image. To display. The image on the display device is the company logo “OX Beer” (indicated by arrow C1 in FIG. 13), product display (indicated by arrow C2 in FIG. 13), and a woman with a beer mug (FIG. 13). Middle, indicated by arrow C3. Consists of multiple image elements that can be displayed independently.

 [0048] First, the relationship between the position of human movement (shadow movement) and image switching will be described with reference to FIG.

 As shown in Figure 14, in the screen F1 displayed as the initial image, the image element of the logo `` Ox Beer '' C1 includes the company introduction image as the content material, and a predetermined part of the logo image element (Indicated in arrow S1 in Fig. 14), the movement of a person is located. In other words, when a person touches a predetermined part S1 of the logo image element, the screen changes to screen F2, and the company introduction image is displayed on the entire screen. Or at a predetermined position.

 On the screen F1, when a person's movement is located at a predetermined part of the image element of the product display C2 (indicated by the arrow S2 in FIG. 14), the screen changes to the screen F3 and an image introducing one product is displayed. It is displayed on the entire screen or at a predetermined position. At this time, when the movement of a person is located at a predetermined portion S1 of the image element on the screen F3 in a state where the screen F3 is displayed, the previous image F2 is displayed.

 On the screen F1, when a person's movement is located at a predetermined part of the woman C3 with a beer mug (indicated by the arrow S3 in Fig. 14), the screen changes to the screen F4 and the message of the producer's voice is displayed. Is displayed.

 Furthermore, in the same manner as in the above example, if a human movement is located in a predetermined part on the screen F1 or in the newly generated images F2, F3, F4, etc., the raw material is included. A screen F6 including images to be introduced, which are different from the products on the screen F5 and the screen F3, are switched and displayed one after another.

 Next, the relationship between the position of human movement (shadow movement) and sound will be described with reference to FIG. 15 showing a screen similar to FIG.

On the screen shown in Fig. 15, the image element of the logo C1 "OX Beer" is the image element of the company's sound logo and the left beer mug (indicated by arrow C4 in Fig. 15) of the two beer mugs. Is the sound of pouring beer into a beer mug and the beer mug on the right ( In FIG. 15, this is indicated by arrow C5. The image element of) is associated with the sound when a beer mug is matched during a toast, and the image element of female C3 is associated with a toasting voice.

 When a person is positioned at a predetermined part of any image element, a sound corresponding to the image element is generated.

 In FIG. 15, rhythm loop A, rhythm loop B, and melody loop are provided corresponding to the areas indicated by arrows X1 to X3, respectively. BGM (music flowing in the background of the image) is created.

 Such an image display method is effective not only for advertising information but also for displaying various types of guidance information, and can be widely used without being limited thereto.

[0050] Next, as another specific example of the present invention, an example in which a person performs a performance as an image and displays an image will be described with reference to FIGS.

 As shown in Fig. 16, for example, an actor performs on the stage where the camera (indicated by arrow B in Fig. 16) is directed. A screen (display: indicated by arrow C in Fig. 16) is provided behind the stage. The screen C displays multiple image elements such as a person who looks at the window in the room, a telephone, etc. In addition, the movement of the person captured by the camera B is displayed as a contour line or a shadow image. Are combined and displayed. Theatrical performers in front of the stage watch the performers on the stage and watch the screen.

[0051] Here, as in the case of the previous advertisement, first, the relationship between the position of the person's movement (shadow movement) and the screen image switching will be described with reference to FIG.

On screen Fl 1 where the interior space of the room is displayed as the initial image, when the actor's movement is located in the part of the person who hits the window force (shown by arrow S4 in Fig. 17), screen F11 and As shown on screen F12 where another room interior space is displayed, for example, after the person has spoken, the power on screen C also disappears. On the screen F12, when the actor's movement is positioned on the telephone portion (indicated by arrow S5 in FIG. 17), for example, the screen returns to the screen F11. On the screen F11, if the performer moves leftward in a predetermined area on the lower left (indicated by arrow S6 in FIG. 17), the screen displays the room interior space that follows the room interior space on the screen F11. Switch to F13. Furthermore, when the performer moves to the door part (indicated by arrow S7 in Fig. 17) displayed on the lower left of the screen F13, the screen image shows the outside landscape. Switch to screen F14. Similarly, when the performer moves in a predetermined area on the upper right of the screen F11 (indicated by an arrow S8 in FIG. 17), the screen switches to a screen F15 that displays an image of the store or the like that is different from the room. On the screen F15, when the performer moves to the right in a predetermined area on the lower right (indicated by an arrow S9 in FIG. 17), the screen switches to a screen F16 that displays a different store from the screen F15.

Next, the relationship between the position of the performer's movement (shadow movement) and the sound will be described with reference to FIG.

 In the image shown in FIG. 18, the image element of the person standing outside the window (indicated by arrow X4 in FIG. 18) is the voice of the person and the image element of the left curtain (indicated by arrow X5 in FIG. 18). Is associated with the sound of the phone and the image element of the phone (indicated by arrow X6 in Fig. 18).

 When a person is positioned at a predetermined part of any image element, a sound corresponding to the image element is generated.

 In addition, narration sound information is provided corresponding to a predetermined area in the upper right of the screen (indicated by an arrow X7 in FIG. 18), and narration is played when a person moves in this area. .

 [0053] Regardless of the embodiment described above, the recognition means uses temperature sensors such as a vibration sensor that recognizes vibrations that have been struck or touched, a Doppler sensor that recognizes air flow and signs, thermography, and the like. A temperature sensor to be recognized or a pressure sensor to recognize a force force condition can be appropriately selected and used.

 In addition to the above-mentioned human behavior, the external information to be recognized includes what the person has and handles, the number of people, member composition, biological information (beating, sweating, body temperature, etc.), or , Movements and appearances of non-human pets (animals) and plants, movements and appearances of cars and machines, robots, weather 'climate' temperature 'wind, landscape, time or language content, voice status (bright, dark, Fun, nervous, happy, etc.), loudness, strength, etc. can be used. In addition, artificial intelligence and conversation functions are built into the main control unit, and effective call-up words are output according to the recognition information, and responses are made to the words picked up by the microphone for dialogue. It is more preferable to be able to do so.

Claims

The scope of the claims  [1] determining an initial image content of a display image composed of a plurality of image elements that can be displayed independently of each other, generating an initial display image, and displaying the initial display image;  A recognition information acquisition process for capturing people as recognition information;  A motion detection process for detecting the movement of a person captured as recognition information;  A content selection step of selecting content material corresponding to a specific image element corresponding to movement or selecting image content related to the initial display image;  The selected image element of the content material and the remaining image element of the initial display image are combined, or the image of the image content is replaced with the initial display image, and also the human image captured as recognition information Image synthesis processing step of cutting out the image as it is or synthesizing a human substitute image as an image element,  A combined image display step for displaying the combined image;  Have  An image display method characterized by repeating a recognition information acquisition step, a motion detection step, a content selection step, an image composition processing step, and a composite image display step.  2. The image display method according to claim 1, wherein in the motion detection step, a motion at the specific image element position is detected. [3] In the content selection step, a sound content corresponding to a motion or a specific image element is selected, and in the composite image display step, the sound content power is obtained together with the composite image or independently of the composite image. The image display method according to claim 1, wherein sound is output.  [4] The image display according to any one of claims 1 to 3, wherein the content of the display image is advertisement information, in-house information, or exhibition information to be provided to a person regarded as recognition information. Method.  [5] The content of the display image is video information that provides a person who views the display image with the performance of the person captured as the recognition information. The image display method described. [6] Initial image content of a display image consisting of multiple image elements that can be displayed independently of each other An image display device for generating and displaying an initial display image,  Recognition information acquisition means for capturing a person as recognition information;  Motion detection means for detecting the movement of a person captured as recognition information;  A content selection unit that selects content material corresponding to a specific image element in response to movement or image content related to an initial display image;  The selected image element of the content material and the remaining image element of the initial display image are combined, or the image of the image content is replaced with the initial display image, and also the human image captured as recognition information Image synthesis processing means for cutting out and synthesizing a substitute image of a person as an image element;  Combined image display means for displaying the combined image;  An image display device comprising: 7. The image display device according to claim 6, wherein the motion detection means detects a motion at the specific image element position. [8] The content selection unit selects a sound content corresponding to a motion or a specific image element, and the composite image display unit generates a sound having the sound content power together with the composite image or independently of the composite image. 7. The image display device according to claim 6, wherein the image display device outputs the image. [9] The image display according to any one of [6] to [8], wherein the content of the display image is advertisement information, in-house information, or exhibition information provided to a person who is recognized as recognition information. apparatus.  [10] The content of the display image is video information that provides a person who views the display image with the performance of the person captured as recognition information, according to any one of claims 6 to 8. The image display device described.