JP5685685B2 - Video display device - Google Patents

Description

  The present invention relates to a video display device that provides information to a viewer.

  As a background art of a video display device that provides information to a viewer, there is Patent Document 1 below. In Patent Document 1, in order to send an appropriate advertisement to a person who views a display device, the user's face is detected using a camera directed to the user who views the display device, and attributes such as sex, adult / children are acquired, A technique for selecting and displaying an image such as an advertisement to be displayed on a display device in accordance with the detection frequency is disclosed.

JP 2005-267611 A

When the viewer starts viewing the advertising content from the middle, there are cases where the entire content of the advertising content cannot be viewed. In this case, the advertiser side cannot provide all information to the viewer, and the advertising effect is lacking. There is also a problem that the viewer side ends without obtaining all the information. In addition, when displaying large characters such as characters in the image so that they can be viewed from a distance, it may be difficult to see for those who are viewing from close, and when viewing in a size that is easy for those who are viewing from a distance, It becomes difficult to see.
An advertisement display that satisfies both the purpose of transmitting information according to the situation and position of an unspecified number of persons and the purpose of transmitting appropriate information to the person watching the display device is desired.

  In order to solve the above-described problem, a video display device having a video display unit that displays video information, a video analysis unit that acquires camera videos supplied from a plurality of cameras and analyzes a subject, and the video analysis A distance calculation unit that calculates a distance between the subject analyzed by the unit and the video display device, and the subject included in a camera video supplied from a predetermined one of the plurality of cameras, A camera control unit that controls an angle of view captured by another camera based on a distance from the video display device calculated by a distance calculation unit, the subject calculated by the distance calculation unit, and the video display device Two or more display areas are provided in the video display unit based on the distance between the camera and the subject of the camera image supplied from the other camera controlled by the camera control unit in the video analysis unit To provide a video display device and a control unit for displaying the image information on the image display unit based on the results.

  According to the present invention, the video display apparatus can provide information to an unspecified number of viewers while providing information to each viewer.

It is an example of the block diagram of a video display apparatus. It is an example of the figure which shows the 1st installation condition of a video display apparatus. It is an example of the top view which shows the positional relationship of a video display apparatus and a to-be-photographed object. It is an example of the figure which shows the 2nd installation condition of a video display apparatus. It is an example of a processing flow diagram of the camera B. It is an example of the processing flowchart of an object subject determination process. It is an example of the processing flowchart of a video content display on a video display apparatus. It is an example of the processing flowchart of the camera A. It is an example of the figure which shows the imaging | photography area | region of a camera.

Embodiments will be described with reference to the drawings. However, the present invention is not limited to this embodiment.
FIG. 1 is an example of a block diagram of the video display device 100. The image sensors 101 and 102 image a subject. The camera signal processing units 103 and 104 convert imaging information captured by the imaging elements 101 and 102 from an optical signal to an electrical signal, and convert an analog electrical signal to a digital signal. The camera control unit 105 controls signal processing operations of the camera signal processing units 103 and 104. The system control unit 200 has a control action by a CPU (Central Processing Unit), and includes a video analysis unit 201, a distance calculation unit 202, a subject determination unit 203, a writing unit 204, a reading unit 205, a display area determination unit 301, a display size. A control unit 302 and a display content selection unit 303 are provided to control the overall operation of the video display device 100.

  The video analysis unit 201, the distance calculation unit 202, the subject determination unit 203, the writing unit 204, the reading unit 205, the display area determination unit 301, the display size control unit 302, and the display content selection unit 303 are included in the system control unit 200. By executing the video analysis program 201, the distance calculation program 202, the subject determination program 203, the writing program 204, the reading program 205, the display area determination program 301, the display size control program 302, and the display content selection program 303 stored in 500. As software, functions of video analysis, distance calculation, subject determination, writing, reading, display area determination, display size control, and display content selection can be realized.

  Each function implemented as software includes a video analysis unit 201, a distance calculation unit 202, a subject determination unit 203, a writing unit 204, a reading unit 205, a display area determination unit 301, a display size control unit 302, and a display content selection unit 303. It can also be realized as hardware by making an integrated circuit as a processing unit for performing each processing. In the following, in order to simplify the description, the video analysis program 201, the distance calculation program 202, the subject determination program 203, the writing program 204, the reading program 205, the display area determination program 301, the display size control program 302, and the display content selection program 303 In the following description, it is assumed that each processing realized by the system control unit 200 is performed mainly by each processing unit. When each processing unit is realized by hardware, each processing unit mainly performs each process.

  The display unit 300 displays, for example, video content stored in the storage unit 500, for example, advertisement information of a product. There may be a function of displaying video information input from the input / output terminal 401. The power supply unit 400 supplies power acquired from a battery or a commercial power source to the video display device 100 in order to drive the video display device 100. The input / output terminal 401 exchanges information signals with other externally connected devices. The storage unit 500 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), a DVD (Digital Versatile Disk), a BD (Blu-ray Disk), an IC card (Integrated Circuit Card), a memory such as a semiconductor, or a memory. , Information such as video, video playback order, program, table, etc. is stored. Information such as video and video playback order is supplied from, for example, the input / output terminal 401 and written into the storage unit 500 by the writing unit 204. Note that there may be a plurality of storage units 500, or different types of storage means. Further, the storage unit 500 may be internal or external.

  The information held in the storage unit 500 is read by the reading unit 205 and supplied to each component, but may be transmitted to another externally connected device via the input / output terminal 401. The input / output terminal 401 may be connected to other devices via a network (not shown). In this case, the video display device 100 can display video content transmitted from another device via the network on the display unit 300.

In this embodiment, two cameras are mounted on the video display device 100. However, a configuration may be adopted in which videos from two externally connected cameras are input to the video display device 100.
In this embodiment, the number of cameras is two, but may be two or more.

  FIG. 2 is an example of a diagram illustrating a first installation state of the video display device 100. Two cameras are installed on the top of the video display device 100. A subject A, subject B, subject C, subject D, and subject group E exist in front of the video display device 100. It is assumed that the subject B and the subject D do not look at the video display device 100, and other subjects look at the video display device 100.

FIG. 3 is an example of a plan view showing the positional relationship between the video display device 100 and the subject, and shows a state where the video display device 100 and subjects A to E in FIG. 2 are looked down from above.
When a person stops and looks at the display screen, or when an operation such as approaching the video display device 100 is performed, it is considered that the displayed content is interesting. The video display device 100 searches the video captured by the camera B for a person who stops and looks at the display screen or who is performing an operation such as approaching the video display device 100. As a search method performed by the video display device 100, for example, a person's face and its position are detected from the shape and color of the captured subject, and the subjects A, C, and E looking at the video display device 100 from their orientation and movement are detected. There is a method of detecting. The video display device 100 uses the camera A to identify the subject A that is closest to the video display device 100 from among the detected persons, and zooms the subject A so that the subject A can be imaged. To track the subject A. For example, it is assumed that the subject A is interested in an advertisement displayed in the first display area 1, stops at a certain place, and approaches the video display device 100 because he wants more information. At this time, the camera A zooms and tracks the subject A to obtain whether or not the subject A is looking at the video display device 100 and a change in the distance between the video display device 100 and the subject A.

In addition, a second display area 2 is created in a different area from the first display area 1 according to the position of the subject A, and the state is appropriate for the subject A that is relatively close to the video display device 100. To display ads. For example, the video output device 100 changes the size, position, and the like of the second display area 2 in order to obtain an appropriate state. For example, when the distance between the subject A and the video output device 100 is farther than the first threshold, the video output device 100 displays the second display area 2 larger than the standard size so that the video can be easily viewed. To do. When the distance is shorter than the second threshold, the video output device 100 displays the second display area 2 smaller than the standard size, thereby making it easy to grasp the entire video.
The first display area 1 displays characters and images large so that the object group E can be easily recognized from a distant place, for example. The second display area 2 An image can be displayed. Here, characters and images of an appropriate size are characters and images of a size that is easy for the subject A to visually recognize. When the subject A moves close to the video display device 100, there may be a case where a fine character or a small image can be sufficiently visually recognized. For this reason, when the distance between the video display device 100 and the subject is smaller than the threshold, the amount of information can be increased instead of making the characters and images smaller than the standard. Furthermore, when the video analysis unit 201 recognizes a person by a method described later and determines that the subject A is an elderly person, the video display device 100 sets the size of characters and images to be larger than the standard size. You may control to display large.

FIG. 4 is an example of a diagram illustrating a second installation state of the video display device 100. In this installation example, a large screen is realized by combining a plurality of video display devices 100A to 100I. Here, the first display area 1 is realized using six video display devices 100A to 100F, and each or any one of the video display devices 100G to 100I at the bottom of the screen is used as the second display area 2. An example of use is shown. In the example of FIG. 4 as well, the first display area 1 displays characters and images large so that they can be easily recognized from a distant place where the subject group E is present, for example. As described above, the second display area 2 is the subject. Characters and images of an appropriate size for A can be displayed.
FIG. 5 is an example of a processing flowchart of the camera B. Hereinafter, the image sensor 101 and the camera signal processing unit 103 in FIG. 1 are components of the camera A, and the image sensor 102 and the camera signal processing unit 104 are the components of the camera B.

  The video analysis unit 201 acquires the video captured by the image sensor 102 via the camera signal processing unit 104 (step S501). The video analysis unit 201 analyzes the acquired video (step S502). The video analysis unit 201 detects, for example, the face of a person who is a subject by detecting colors, eyes, nose, and mouth, or the difference between an input image and a previously acquired image that does not show a person. Analysis such as recognizing the position of. The distance calculation unit 202 calculates distance information between the subject recognized by the video analysis unit 201 and the video display device 100 from mapping information in which pixel positions and distances in the screen are mapped in advance and stored in the storage unit 500. (Step S503).

  The subject determination unit 203 determines a target subject from the subject information such as the person analyzed by the video analysis unit 201 in step S502 and the distance information acquired by the distance calculation unit 202 in step S503 (step S600). The subject determination unit 203 determines, for example, the subject A that is looking at the video display device 100 and is closest to the video display device 100 among the subjects illustrated in FIG. Step S600 will be described in detail with reference to FIG.

  Next, the system control unit 200 acquires whether the camera A is being zoomed from the camera control unit 105 (step S504). If the camera A is not being zoomed (No in the figure), the system control unit 200 controls the camera A and zooms the camera A to the target subject determined in step S600 (step S505). If the camera A is being zoomed (Yes in the figure), the system control unit 200 holds the camera A control information in, for example, the storage unit 500 (step S506). The camera A control information is, for example, position information regarding the next person to be zoomed, and is information regarding, for example, the subject C shown in FIG. The system control unit 200 stores content information such as a content name and ID indicating the content displayed on the display unit 300 during zooming in the storage unit 500 (step S507).

FIG. 6 is an example of a process flow diagram of the target subject determination process S600.
The video analysis unit 201 determines from the image of the camera B whether there is a person watching the video display device 100 other than the subject that is the current zoom target of the camera A (step S601). The analysis by the video analysis unit 201 in step S601 can use the same method as the analysis in step S502. The video analysis unit 201 determines whether there is a person watching the video display device 100 such as the subject C or the subject group E in addition to the subject A that the camera A in FIG. judge. When there is a person watching the video display device 100 (Yes in the figure), the subject determination unit 203 selects a person who is closest to the video display device 100 from the corresponding people as the subject of the camera A ( Step S602). If there is no person watching the video display device 100 (No in the figure), the video analysis unit 201 determines whether there is a person around the video display device 100 (step S603). In step S603, the video analysis unit 201 determines whether there is a person who is not looking at the video display device 100, such as the subject B or the subject D in FIG. If there is a person watching the video display device 100 (Yes in the figure), the subject determination unit 203 selects any person as the subject of the camera A (step S604). At that time, a person in a preset area may be selected, or a person in an area having the most people may be selected. On the other hand, when there is no person watching the video output device 100 (No in the figure), the subject determination unit 203 selects a predetermined position set in advance as an imaging target of the camera A (step S605).
The subject determining unit 203 selects a target subject area to be selected next by the camera B (step S603). Here, for example, a place where the person is the most other than the subject selected for the camera A is selected. In the case of the example in FIG. 2, the area of the subject group E is selected. As a method of determining the place where the person is most, the video processing unit 201 divides the video image captured by the camera A into a plurality of areas, and the area where the most faces can be detected among the divided areas An example is a method of determining that there is a spot.

As described above, according to the flow of FIG. 5 and FIG. 6, using a subject image captured by the camera B with a wide angle of view, a specific subject is selected from among them, and the camera A is zoomed. You can get information about.
Next, the video content display processing flow will be described with reference to FIG.

FIG. 7 is an example of a processing flowchart of video content display on the video display device 100.
This process is performed after the distances from the camera B and the target subject of the camera B are calculated by the camera B in step S503 in FIG. Further, when the content information held in step S507 in FIG. 5 is used, the content information may be started after the processing flow in FIG.

  The display area determination unit 301 determines the size of the display area for the target subjects of the camera A and the camera B (step S701). As a method for determining the size of the display area, a value obtained by multiplying the distance between the target subject of the camera A or the camera B and the video display device 100 by a certain ratio may be set as the size of the display area. In addition, a threshold is set in advance for the distance between the subject and the video display device 100, the display area is reduced to a size smaller than the standard size when the distance is closer than the threshold, and the standard is set when the distance is larger than the threshold. The display area may be enlarged to a size larger than the size of. The display area determination unit 301 determines the display position on the display unit 300 according to the positions of the target subjects of the camera A and the camera B. For example, as shown in FIG. 4, the display area determination unit 301 selects the display area 100H when the subject is at the position B, and selects the display area 100G when the subject is at the position A, that is, the subject A is an image. The second display area 2 is such that when the display device 100 is on the right side of the front of the display device 100, the display area located on the right is selected, and when the subject A is on the left side, the display area located on the left is selected. Is displayed near the subject A.

  The display content selection unit 303 determines the video to be displayed in the display area determined in step S701 (step S702). For example, the display content selection unit 303 selects the content held in step S507 and the content associated in advance with the content. This is to present information once again to the subject interested in the currently displayed content searched by the camera B and display it from the beginning of the content, or to present detailed information.

  The display size control unit 302 determines the display size of the determined content (step S703). In step S703, the display size control unit 302 determines in what size the content is displayed in the display area determined in step S701, for example, according to the distance to the target subject. As an example of a method of changing the display size in accordance with the distance, when the distance between the target subject and the video display device 100 is larger than a preset threshold, the content is scrolled, for example, so that characters and images are standardized. When the image is displayed larger than the size and the distance between the target subject and the video display device 100 is closer than a preset threshold, there is a method of displaying the content without scrolling on one screen.

The display unit 300 displays the video content based on the information acquired from the display area determination unit 301, the display content selection unit 303, and the display size control unit 302 (step S704).
Next, a processing flow of the camera A controlled by the camera B will be described with reference to FIG.

FIG. 8 is an example of a processing flowchart of the camera A.
This process is started after the processing flow of FIG. 5 is completed when the content information held in step S507 is used. That is, it may be performed in parallel with the processing flow of the video content display shown in FIG. 7, or any one may be performed first.
The system control unit 200 acquires a video from the camera signal processing unit 103 of the camera A (step S801). The video analysis unit 201 analyzes the acquired video (step S802). For example, face detection, organ detection, gaze detection, facial expression detection, and the like are performed. The video analysis unit 201 determines whether there is a face in the video by detecting colors, eyes, nose, and mouth (step S803). If the face can be determined (Yes in the figure), the video analysis unit 201 acquires the detected information such as the line of sight and facial expression acquired in step S803 and stores the information in the storage unit 500. Thereby, information such as which part of the video content displayed by the user is interested or not interested is collected (step S805).

  On the other hand, when the face cannot be determined (No in the figure), no person (face) is detected in the video, so the system control unit 200 instructs the camera control unit 105 to analyze the video from the camera B to the camera B. The area information to be excluded is transmitted (step S806). For example, the position and size information of the area A shown in the example of FIG. 9 is transmitted. In camera B, since area A is small, detailed analysis cannot be performed, but by zooming camera A, a detailed video can be captured, and area A that cannot be analyzed may be analyzed. Thus, for example, by feeding back to the camera B that the target subject information determined by the camera B is incorrect, the accuracy of the target subject determination can be improved. Alternatively, by omitting from the analysis target an object that could not be analyzed in detail because the area A is small at the time of video analysis in the camera B, the video analysis time can be shortened and the real-time performance is improved.

  The system control unit 200 obtains the camera control information related to the camera A held in the storage unit 500 in step S506 and supplies it to the camera control unit 105 as to which subject should be zoomed next (step S807). The camera control unit 105 performs camera control on the camera A based on the supplied control information (step S808).

According to the present embodiment, information can be appropriately transmitted to an unspecified number of people who view the display device. In addition, for example, for those who are interested in advertising content, by viewing from the top using a specific display area or viewing more detailed advertising content, the information that the target person wants is surely Can communicate to. Further, since the information of the target person can be acquired by zooming the camera to the corresponding person, useful information can be obtained also on the advertiser side. Thus, convenience is improved on both the viewer side and the advertising company side.
In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by the system control unit 200 interpreting and executing a program that realizes each function. Information such as programs, tables, and files for realizing each function can be stored in a memory, a storage device such as a hard disk or an SSD, or a storage medium such as an IC card, an SD card, or a DVD.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

  100: Video display device, 101: Image sensor, 102: Image sensor, 103: Camera signal processing unit, 104: Camera signal processing unit, 105: Camera control unit, 200: System control unit, 201: Video analysis unit, 202: Distance calculation unit, 203: subject determination unit, 204: writing unit, 205: reading unit, 300: display unit, 301: display area determination unit, 302: display size control unit, 303: display content washing unit, 400: power supply unit 401: Other device input / output terminal, 500: Storage unit.

Claims (7)

A video display device having a video display unit for displaying video information,
A video analysis unit that acquires camera video supplied from multiple cameras and analyzes the subject;
A distance calculation unit that calculates a distance between the subject analyzed by the video analysis unit and the video display device;
Based on the distance between the subject included in the camera video supplied from a predetermined one of the plurality of cameras and the video display device calculated by the distance calculation unit, another camera captures an image. A camera control unit for controlling the angle of view;
A display area for displaying video information to be provided to the subject on the video display unit based on an analysis result in the video analysis unit of the subject of the camera image supplied from the other camera controlled by the camera control unit; A control unit for displaying video information on the video display unit based on a distance between the subject calculated by the distance calculation unit and the video display device;
When displaying video information based on the analysis result in the video analysis unit of the subject of the camera image supplied from the other camera controlled by the camera control unit, of the two or more display regions, A video display device comprising: a display area for displaying video information to be provided to a subject; and a display content selection unit for displaying video information displayed in another display area . The video display device according to claim 1,
The analysis performed by the video analysis unit is to analyze from the camera video whether or not the subject is viewing the video display device,
The control unit creates a display area for providing video information to the subject in the video display unit when the subject is viewing the video display device;
If the subject does not look at the video display device, do not create a display area for providing video information to the subject in the video display unit;
A video display device characterized by the above. The video display device according to claim 1,
When the distance calculated by the distance calculation unit is farther than the threshold, the control unit, when the distance calculated by the distance calculation unit is closer than the threshold, images and characters having a size larger than the size to be displayed, Displaying in a display area that provides video information to the subject created in the video display unit;
A video display device characterized by the above. The video display device according to claim 1 ,
The display content selection unit displays the video information displayed in the other display area when displaying the video information displayed in the other display area in the display area displaying the video information provided to the subject. A video display device characterized by displaying from the top. The video display device according to claim 1,
The video display unit, wherein the video analysis unit includes removing a specific area analyzed by a camera video supplied from the other camera from a video analysis target in the predetermined one camera video. The video display device according to claim 1,
A video display device comprising: a storage unit that stores information related to a result of analysis by the video analysis unit and a video displayed on the video display unit at the time of analysis. The video display device according to claim 1,
A storage unit for storing video information;
A writing unit for receiving video information to be displayed on the video display unit from the outside via a network and storing the information in the storage unit;
A readout unit that reads out video information stored in the storage unit and transmits the video information to the outside via the network;
A video display device comprising:

Images