JP2020188328A - Video system - Google Patents

Abstract

To display photographic information that is a still image in addition to the video image of a video.SOLUTION: A video system including a main screen and a sub screen includes means that shoots video footage of a video, means that takes photographic information of a still image, means that displays the photographic information in the ground on the main screen, and means that displays still image information taken by a photographer on the sub screen. Further, the information displayed on the main screen is a video image of a video, and photographic information which is a still image is displayed on the sub screen in synchronization with the viewing of the video image of the video.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system for displaying an image.

In recent years, stadiums have become larger so that many spectators can see them. For this reason, since the match venue is large, the appearance of the match being played on the opposite side will be smaller from the spectator's point of view, and the live screen of the match should be displayed on the main screen in the ground where the match is being played. There are many. Specifically, images from a plurality of cameras are displayed on the screen. The images from multiple cameras are switched and displayed on the screen.

The video display system of Patent Document 1 is used by a streaming server that distributes live video and a user who watches live video. A user terminal that displays a screen that displays live video and an icon that is an identification mark of a user who watches live video in a movable manner in the virtual space, and visualizes and displays the landscape of the virtual space from the position of the icon in the virtual space. And. Here, there is a screen that displays live video.

The video distribution method of Patent Document 2 is a video distribution method by a server that distributes a plurality of images taken from different viewpoints by a plurality of users to a terminal device, and is one of the plurality of images and is requested by the terminal device. A distribution step of delivering the first video to the terminal device, a selection step of selecting a second video that is likely to be requested next from the terminal device from a plurality of videos, and a distribution of the first video to the terminal device. It includes a transmission step of initiating transmission of the second video to the terminal device during the process. This is a video distribution method that allows smooth switching of the video displayed on the screen.

JP-A-2016-34087 Japanese Unexamined Patent Publication No. 2016-10145

However, in the above-mentioned Patent Documents 1 and 2, only the main screen is provided, and the image from the camera is only displayed on the screen as a relay screen.

Many professional photographers are taking pictures of many still images at the venue, so if people who come to the spectators can see the pictures, the spectators will have more fun watching the game. ..

In particular, in rugby, tri-scenes and the like are often replayed. It's a shame that the audience can't see the photos taken by professional photographers in the tri-scene replay. At this time, if a still image taken by a professional photographer is displayed on the screen, it will be powerful and will give the illusion of coming to the photo studio while on the ground.

In order to solve the above problems, the video system according to the present invention
In a video system that has a main screen and a sub screen in a stadium, a means for shooting a video image of a moving image, a means for taking a still image photographic information, a means for displaying shooting information in the ground on the main screen, and a photographer It has a means for displaying the still image information on the sub screen.
Further, the information displayed on the main screen is a video image of a moving image, and photographic information which is a still image is displayed on the sub screen in synchronization with the viewing of the video image of the moving image.
There are means to record video information of videos, means to record video signals of videos including time stamp information, means to play video information of videos, and means to take pictures of still images including time stamp information. However, when the video image of the moving image is played back, the still image information is also displayed in synchronization with the video image.

According to the video system according to the present invention, it is possible to display a still image taken by a professional photographer while displaying a relay image on the main screen on a live screen in a stadium. People will be able to see powerful photographic images, and the real thrill of sports will increase. In particular, in the replay, many goal scenes are reproduced, and even during this reproduction, a still image can be displayed and a powerful scene can be enjoyed.

A diagram schematically showing the configuration of a video recording display system Diagram showing a detailed block diagram on the server side Detailed block diagram on the device side Detailed block diagram on the device side Figure showing file format Video shooting Camera control flow (camera side) Diagram showing the photography camera control flow (camera side) Block diagram showing a functional configuration example of the tracking unit 870 of a digital camera Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Figure showing screen display control flow (server side) Diagram showing remote screen display control flow (server side) Diagram showing remote screen display control flow (server side) Diagram showing the main screen and sub screens in the stadium Diagram showing the main screen and sub screens in the stadium

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

An overall image of a diagram schematically showing the configuration of a video recording display system will be described. Specifically, a plurality of overall images of the video shooting display system FIG. 1 will be described.

Here, in the stadium, a professional photographer shoots a video with a plurality of cameras, and the video image is switched by a switcher to display a live video image on the main screen. The sub-screen displays photos, video information from different angles, match information, player information, and more. Here, the photograph and the video information from another angle are often displayed in synchronization with the information displayed on the main screen. Here, synchronization means displaying a still image having a time stamp information close to the time stamp information of the video being played. Further, since the still image is displayed for a certain period of time (for example, 1 second), the moving image and the still image are not completely synchronized, and the times are almost the same.

Also, as an aid, on the server side with multiple cameras, it is possible to grasp the situation of the players in the entire field and provide valuable information in camera shooting to those who shoot individual fields. .. As this value, it is also possible to provide the position information of the player of interest in the field on the server side to the device side of the person who shoots in the field.

On the device side, a camera alone, a smartphone alone, or a camera and a smartphone can be integrated.

<Video shooting display system diagram>
<< Video shooting display system Figure 1 >>
FIG. 1 shows a video shooting display system diagram.

In FIG. 1, 101 (K camera 1), 102 (K camera 2), 103 (K camera 3), 120 (main screen), 121 (sub screen), 110 (server), 111 (input means), 112 ( The base station) is a video acquisition, audio acquisition, etc. that the cameraman outputs to display a video image to be displayed on the main screen or a photograph to be displayed on the sub screen. In addition to video acquisition and audio acquisition, input information other than video (match information, player information, etc.) can be imported from the input means to expand services to spectators.

Reference numeral 105 is a wired connection by a wired LAN or the like, and 106 is a connection for inputting the information output from the input means 111 to the server 110.

Reference numeral 107 denotes a connection for transmitting and receiving a signal to and from the base station. Reference numeral 108 denotes an antenna unit for executing wireless communication of the base station.

That is, the above-mentioned 100-series block is a block that supports a photographer or the like to shoot and display an image.

On the other hand, in FIG. 1, 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3) are image acquisitions taken by a photographer or a general audience. Here, 404 (antenna) 405 (antenna) and 406 (antenna) are antennas for 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3) to perform wireless communication, respectively.

The device that performs the video recording is connected only wirelessly.

In the stadium, the information from the terminal 1 to the terminal 3 is mainly photographic information which is a still image, and these photographic information are often displayed on the sub screen. Video images are often taken mainly by a broadcasting camera described later and displayed on the main screen. Information is sent from the terminal to the server side, and information is sent from the server side to the terminal. Here, the information taken by the terminal needs to be displayed in synchronization with the video image taken by the broadcasting camera. Since the video is transmitted by wire in the broadcast camera, stable high-speed communication is possible, so that the terminal side also needs high-speed communication, and 5G is required for wireless communication here. Here, synchronization means displaying a still image having a time stamp information close to the time stamp information of the video being played. Further, since the still image is displayed for a certain period of time (for example, 1 second), the moving image and the still image are not completely synchronized, and the times are almost the same.

122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3) are cameras for acquiring video images.

On the other hand, the 400-series block is a block in which a photographer or a general audience actually shoots an image.

The 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3) may be a smartphone or a camera.

401 (terminal 1), 402 (terminal 2), 403 (terminal 3) may be "cameras, smartphones". An explanatory diagram is shown in the lower right of FIG. 1, which is composed of 301, 304, and 320. In FIG. 1, 301 (smartphone 1), 302 (smartphone 2) (not shown), 303 (smartphone 3) (not shown) are smartphones, and mainly perform communication control with a server. In addition, various video acquisition services can be realized by installing application software on these smartphones.

Further, 304 (camera 1), 305 (camera 2) (not shown), and 306 (camera 3) (not shown) are cameras, and are mainly image acquisitions taken by a cameraman or a general audience. Here, 304 (camera 1) is connected to 301 (smartphone 1), 305 (camera 2) (not shown) is connected to 302 (smartphone 2) (not shown), and 306 (camera 3) (not shown). Is connected to 303 (smartphone 3) (not shown). These connections are connected by USB. Here, 320 (antenna) 321 (antenna) (not shown), 322 (antenna) (not shown) are 301 (smartphone 1), 302 (smartphone 2) (not shown), 303 (smartphone 3), respectively. (Not shown) is an antenna for wireless communication.

That is, the above-mentioned 300-series block is a block in which a photographer or a general spectator actually shoots an image and executes communication control with a server.

If the terminal is a smartphone, all video and control signal exchanges with the server are performed wirelessly.

If the terminal is a camera, the principle is the same as for smartphones. The camera wirelessly exchanges video and control signals with the server. Using a camera can provide high-quality information from a smartphone.

When the terminal is a "camera, smartphone", the smartphone has an environment for exchanging images and control signals with the server. Cameras are excellent for good video acquisition.

By directly connecting the two smartphones and the camera by wire, it has become possible to easily provide products that take advantage of the performance of both devices.

In FIG. 1, the connection between the terminal and the server is a wireless only connection. The connection for executing the communication from the terminal may be a wired communication in addition to the wireless communication. This has made it possible to realize communication that appropriately uses wireless communication and wired communication. For example, if the wireless communication environment is 5G, it can be configured wirelessly, and if the wireless communication environment is LTE, information with a large amount of data can be sent by wire, and control signals with a small amount of data can be sent wirelessly. became. Furthermore, depending on how busy the wireless communication line is, it has become possible to appropriately execute wired communication.

Next, a block diagram on the server side will be described.

<Block diagram on the server side>
FIG. 2 is a detailed block diagram on the server side.

In FIG. 2, 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3) are the same as those in FIG. 1, and are cameras for acquiring video images, and 205 (wired). It is connected by wire (LAN) or the like.

Reference numeral 201 denotes an Ethernet (registered trademark) controller, and blocks 201 to 205 and 211 to 214 are servers.

202 is an input means. It shows that the server captures match information, player information, and so on. In the present invention, the role as a metadata input unit before the start of the game is important. Specifically, the information before the start of the game is input. The information before the start of the game includes, firstly, the information from two to three months ago, for example, the game card, the game venue, and the game time. The second is the member list of each team announced 48 hours ago. Thirdly, it is the information immediately before the game corresponding to the change of the member immediately before.

The 203 (base station) wirelessly communicates with the photographer who takes the picture or the device (camera, smartphone, etc.) owned by the person taking the picture, and supports taking a picture from the server. ..

The server blocks are 704 to 713 as described above. These blocks will be described in order.

Reference numeral 204 denotes a switcher, which inputs videos from 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3), and switches any of these videos to the main screen. It is a block to be displayed on. Further, it is a block in which photographs of still images from 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3) are input, and any of these photographs is switched and displayed on a sub screen.

Reference numeral 205 denotes a video synchronization circuit, which includes time stamp information included in video signals from 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3), and 401 (terminal). Based on the time stamp information included in the photographic signals from 1), 402 (terminal 2), and 403 (terminal 3), the images on the main screen and sub screen are synchronized and displayed.

Reference numeral 211 denotes a CPU (Central Processing Unit), which is a central processing unit that executes the following controls based on a control program on the program memory 712 described later. It also serves as a display control means, and controls information to be displayed on the display unit 214, which will be described later.

The program memory 212 stores a program for executing control.

The data memory 213 is a data memory for storing past game information, past player information, information about today's game, information such as weather regarding the number of spectators, information on players to watch, and the current situation of players. is there.

Next, a detailed block diagram on the device side will be described.

<Detailed block diagram on the device side>
FIG. 3 is a block diagram on the device side (digital camera side).

3a and 3b show the device side (digital camera side) on page 2. Here, in FIGS. 3a and 3b, the 318 CPU (Central Processing Unit), the program memory 319, and the data memory 320 are overlapping blocks, but in reality, they are one and the same block.

In FIGS. 3a and 3b, the digital camera can shoot moving images and still images and record shooting information.

In FIG. 3a, 301 is an Ethernet controller. Reference numeral 302 denotes a storage means, in which moving images and still images taken by a digital camera are formatted and stored. Reference numeral 303 denotes a sensor, which converts an image at the time of shooting from an optical signal to an electric signal, and further converts this information from analog information to digital information and outputs the information. Reference numeral 304 denotes a signal processing unit, which corrects the device and outputs the digital information read by the sensor. Reference numeral 305 is a sensor processing unit, which creates line management information for reading information from the sensor, timing at which the sensor outputs data, and the like, and inputs the information to the sensor 803.

Reference numeral 306 is an operation unit input means. There are operations for setting conditions for shooting with a digital camera, or operations for shooting, operations for using a flash, operations for replacing batteries, and the like, and these information are output to the signal line 370. In addition, the operation unit can be set to select whether or not to AF (autofocus) the player of interest. Information on whether to AF (autofocus) this attention player is output from 306 to 370. Further, in the operation unit, it is possible to select whether or not to automatically chase the attention player. Information on whether or not to automatically chase the attention player is output from 306 to 370. Furthermore, it serves as a voice information input unit for professional photographers. This inputs the cameraman's voice when the cameraman leaves a voice message at the time of shooting. At the time of shooting, photographers can leave the intention of shooting as a message in the photo as metadata. This information becomes metadata at the time of shooting.

Reference numeral 307 is a wireless communication unit, in which a photographer who takes a picture or a device (digital camera, smartphone, etc.) owned by a person who takes a picture wirelessly communicates with the server side. Reference numeral 308 is a block for detecting the magnification when shooting with a digital camera. Reference numeral 309 is an operation unit output means, and includes a display unit that displays information taken by a digital camera or the like. In 310, the data from the image sensor in the camera is compressed into a PEG image file or a HEIF image file after the RAW data is developed, or compressed as the RAW data into a RAW image file. The block here is a block to be compressed. Further, when the RAW image file is developed in the camera to generate a JPEG image file or a HEIF image file, the compressed information is decompressed and returned to the RAW data. The blocks here also have blocks that extend.

Reference numeral 311 is a video synchronization circuit. It is a circuit that incorporates the date and time of shooting as time stamps into the video as metadata for the information shot by the camera. In 316, the device (digital camera, smartphone, etc.) detects the state of the power supply. Moreover, when the power switch is off, only the pressing of the power button is detected. Then, when the power button is pressed, power is supplied to the entire device (digital camera, smartphone, etc.). Reference numeral 317 is an image processing unit, which is a block for inputting information read and signal-processed by a sensor and outputting the image-processed information to signal line 370.

Reference numeral 318 is a CPU (Central Processing Unit), which is a central processing unit that executes the following controls based on a control program on the program memory 319 described later. It also serves as a display control means, and controls information to be displayed on the display unit 309, which will be described later.

The program memory 319 stores a program for executing control.

The data memory 320 is a data memory for storing setting conditions of a digital camera, and for storing captured still images, moving images, and attribute information of still images and moving images.

In FIG. 3b, 850 is a photographing lens (lens unit). The photographing lens (lens unit) includes a fixed 1-group lens 851, a zoom lens 852, an aperture 855, a fixed 3-group lens 858, a focus lens 859, a zoom motor 853, an aperture motor 856, and a focus motor 860. The fixed 1-group lens 851, the zoom lens 852, the aperture 855, the fixed 3-group lens 858, and the focus lens 859 constitute a photographing optical system. For convenience, the lenses 851, 852, 858, and 859 are shown as one lens, but each may be composed of a plurality of lenses. Further, the photographing lens 850 may be configured as a detachable interchangeable lens.

The diaphragm control unit 857 controls the operation of the diaphragm motor 856 that drives the diaphragm 855, and changes the aperture diameter of the diaphragm 855.

The zoom control unit 854 controls the operation of the zoom motor 853 that drives the zoom lens 852, and changes the focal length (angle of view) of the photographing lens 101.

The focus control unit 861 calculates the defocus amount and the defocus direction of the photographing lens 850 based on the phase difference of the pair of focus detection signals (A image and B image) obtained from the sensor 803. Then, the focus control unit 861 converts the defocus amount and the defocus direction into the drive amount and the drive direction of the focus motor 860. The focus control unit 861 controls the operation of the focus motor 860 based on the drive amount and the drive direction, and controls the focus of the photographing lens 850 by driving the focus lens 859. In this way, the focus control unit 861 implements the phase difference detection type autofocus (AF). The focus control unit 861 may execute AF of the contrast detection method based on the contrast evaluation value obtained by obtaining the image signal obtained from the sensor 803.

Reference numeral 870 is a player tracking unit of the digital camera itself.

When a photograph is taken, it becomes, for example, a JPEG-compressed image file after image processing. Date and time when the photo was taken, position information (latitude, longitude, altitude, direction), information taken (aperture value, exposure time, ISO sensitivity, shutter speed, photometric method, lens focal length, presence / absence of flash, exposure compensation value , Camera shake correction, color space, etc.), camera information taken (camera model name, lens name), etc. are stored as metadata corresponding to the photograph. Exif (Exchangeable Image File Format) is a format standard for storing this metadata. The metadata described in detail below is stored in the undefined data in this Exif. The DCF (Design rule for Camera File system) defines a file standard for realizing mutual use in digital cameras, personal computers, and the like. Exif is an image file format used in digital cameras.

FIG. 4 shows a DCF thumbnail file and a DCF basic file. The DCF thumbnail file is composed of SOI, Exif information, a thumbnail file, and EOI. On the other hand, the DCF basic file is composed of SOI, Exif information, thumbnails, DCF basic main image, and EOI.

Here, SOI represents the beginning of the JPEG data in the Start of Image, and EOI represents the end of the JPEG data in the End of Image. In addition, Application 1 is Application marker segment 1 and stores Exif information.

In the present invention, as Exif information, as information newly defined, the date, the match card, the match venue, the match time, the shooting time, the time in the match, and the like. Furthermore, when the player is photographed, the player name is stored. In addition, time stamp information is also stored.

Next, the video shooting camera control flow (camera side) displayed on the main screen will be described.

FIG. 5 shows 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3), which are control flows for shooting a video image displayed on the main screen in the stadium (camera side). Is illustrated.

In FIG. 5, S101 represents initialization.

In S102, it is determined whether or not video shooting is selected, and if video shooting is selected, the process proceeds to S103, and if video shooting is not selected, the process proceeds to S101.

S103 represents obtaining the setting information of the camera.

In S104, it is determined whether or not the actual video shooting to be displayed on the main screen is selected, and if the actual video shooting to be displayed on the main screen is selected, the process proceeds to S105 and the actual video shooting to be displayed on the main screen is selected. If is not selected, the process proceeds to S110 and other processing is performed.

S105 represents adding a time stamp to the actual video image being shot.

S106 represents sending the captured video information to the server.

In S107, it is determined whether or not the video shooting is finished, and if the video shooting is finished, the process proceeds to S101, and if the video shooting is not finished, the process proceeds to S105.

FIG. 6 shows a control flow (camera side) for taking a photographic image to be displayed on a sub-screen of 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3).

In FIG. 6, S101 represents initialization.

In S102, it is determined whether or not photography is selected, and if photography is selected, the process proceeds to S103, and if photography is not selected, the process proceeds to S101.

S103 represents obtaining the setting information of the camera.

In S104, it is determined whether or not the actual photography to be displayed on the sub screen is selected, and if the actual video shooting to be displayed on the sub screen is selected, the process proceeds to S105 and the actual photography to be displayed on the sub screen is selected. If is not selected, the process proceeds to S110 and other processing is performed.

S105 represents adding a time stamp to the actual photographic information being photographed.

S106 represents sending the photographed photograph information to the server.

In S107, it is determined whether or not the photography is completed, and when the photography is completed, the process proceeds to S101, and if the photography is not completed, the process proceeds to S105.

Next, the tracking unit of the digital camera will be described.

FIG. 7 is a block of the tracking unit of the device.

<Digital camera tracking unit>
FIG. 7 is a block diagram showing a functional configuration example of the tracking unit 870 of the digital camera.

The tracking unit 870 includes a collation unit 871, a feature extraction unit 872, and a distance map generation unit 873. The tracking unit 870 specifies an image area (subject area) to be tracked from a designated position, and extracts a feature amount from the subject area. Then, in the supplied individual captured images, the extracted feature amount is used to search for a region having a high degree of similarity to the subject region of the previous frame as the subject region. Further, the tracking unit 870 acquires distance information from a pair of parallax images and uses it to identify the subject area.

The collation unit 871 searches for a subject area in the image supplied from the feature extraction unit 872. The method of searching the region based on the feature amount of the image is not particularly limited, but the collation unit 871 uses at least one of template matching and histogram matching.

Next, the screen display control flow (server side) when the live video is displayed on the main screen and the photo display is selected on the sub screen at the timing synchronized with the sub screen will be described.

<Example 1: Photograph display on the sub screen>
8a and 8b show 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3), which capture video images to be displayed on the main screen in the stadium at the same time. , 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3), the control flow (server side) of taking a photographic image to be displayed on the sub screen is illustrated.

In FIGS. 8a and 8b, S201 represents initialization.

In S202, it is determined whether or not video shooting is selected, and if video shooting is selected, the process proceeds to S203, and if video shooting is not selected, the process proceeds to S211.

S203 represents obtaining the setting information of the camera.

In S204, it is determined whether or not photography is selected, and if photography is selected, the process proceeds to S205, and if photography is not selected, the process proceeds to S205.

S205 represents displaying a video on the main screen. At the same time, the video information is compressed and recorded. The compression ratio here is encoded with higher compression than the video image displayed on the main screen in real time. This recording corresponds to replay and others.

S206 represents displaying a photograph on the sub-screen. At the same time, the still image information is compressed and recorded. The compression ratio here is encoded with higher compression than the photographic information displayed on the main screen in real time. This recording corresponds to replay and others.

S207 represents displaying a photograph on a sub-screen. Here, the pictures on the sub screen are synchronized with the video image displayed on the main screen.

In S208, it is determined whether or not the video is displayed or ended, and when the video is displayed or finished, the process proceeds to S208-2, and if the video display is not finished, the process proceeds to S206.

S208-2 indicates that the display of the video on the main screen is stopped.

S208-3 indicates that the display of the photograph on the sub screen is stopped.

S211 represents other processing.

FIG. 8b will be described.

S212 represents displaying a video on the main screen.

S213 indicates that the photograph is not displayed on the sub screen.

In S214, it is determined whether or not the video is displayed or ended, and when the video is displayed or finished, the process proceeds to S215, and if the video display is not finished, the process proceeds to S212.

S215 indicates that the display of the video on the main screen is stopped.

FIG. 11 is a diagram showing a main screen and a sub screen in the stadium. Here, S501 displays the moving image information in the main screen. On the other hand, S502 is a still image, which is a photograph taken by a professional photographer. S503 represents an audience. Audiences can see the video and the photos taken by professional photographers at that time to increase the sense of presence and re-recognize the goodness of still images.

FIG. 12 is a diagram showing a stadium including a main screen and a sub screen. Here, S504 displays the moving image information in the main screen. On the other hand, S505 is a sub-screen, the display is a still image, and is a photograph taken by a professional photographer. S506, S507, and S508 represent a stadium. The area between the elliptical line of S506 and the elliptical line of S507 is the audience seats. Many spectators come here to watch the game. Audiences can see the video and the photos taken by professional photographers at that time to increase the sense of presence and re-recognize the goodness of still images.

Further, the inside of S507 represents a field, and S508 represents a rugby ground.

The S504 main screen and S505 subscreen, like many athletics stadiums, are above the spectators' seats in the third to fourth corners of the track on land.

<Example 2: Sub-screen displays match information>
8a and 8c show the 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3), which capture the video image to be displayed on the main screen in the stadium and sub. The control flow (server side) for displaying match information is shown on the screen.

FIG. 8c will be described. In S204 of FIG. 8a, when No is determined, the process proceeds to S216.

In S216, it is determined whether or not the display of the match information is selected, and if the display of the match information is selected, the process proceeds to S217, and if the display of the match information is not selected, the process proceeds to S201 (FIG. 8a). The selection of the display of the game information is determined by inputting the information from the input means 202. In addition, as the match information to be displayed here, the history of the score information so far, the status of the current match, and the like shall be displayed.

S217 represents displaying a video on the main screen.

S218 indicates that the match information is displayed on the sub screen. Here, the main screen and the sub screen are not synchronized on the time axis.

In S219, it is determined whether or not the video is displayed or ended, and when the video is displayed or finished, the process proceeds to S220, and if the video display is not finished, the process proceeds to S217.

S220 indicates that the display of the video on the main screen is stopped.

When displaying video information on the sub screen in the stadium, by synchronizing with the video video on the main screen using the time stamp, the audience can be provided with a powerful video and photo of the game. It became possible.

On the other hand, the match information does not need to be closely synchronized with the video image on the main screen. It has become possible to achieve this.

As mentioned above, it is very effective to change the way of synchronizing with the main screen depending on the content displayed on the sub screen.

<Example 3: Player information is displayed on the sub screen>
8a and 8d show video images to be displayed on the main screen with 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3), and player information is displayed on the sub screen. The control flow (server side) that displays is illustrated.

FIG. 8d will be described. In S204 of FIG. 8a, when No is determined, the process proceeds to S221.

In S221, it is determined whether or not the display of player information is selected, and if the display of player information is selected, the process proceeds to S222, and if the display of player information is not selected, the process proceeds to S201 (FIG. 8a). The selection of the display of the player information is determined by inputting the information from the input means 202. Further, as the player information to be displayed here, information when a player is replaced, information when a player gives a try, or the like is displayed.

S222 represents displaying a video on the main screen.

S223 indicates that the match information is displayed on the sub screen. Here, the main screen and the sub screen are not synchronized on the time axis.

In S224, it is determined whether or not the video is displayed or ended, and when the video is displayed or finished, the process proceeds to S225, and if the video display is not finished, the process proceeds to S222.

S225 indicates that the display of the video on the main screen is stopped.

When displaying video information on the sub-screen, by synchronizing with the video video on the main screen using a time stamp, it has become possible to provide the audience with a powerful video and photo of the game.

On the other hand, player information does not need to be closely synchronized with the video image on the main screen. It has become possible to achieve this.

As mentioned above, it is very effective to change the way of synchronizing with the main screen depending on the content displayed on the sub screen. This is the same as displaying the match information on the sub screen described above.

<Example 4: The sub screen displays video information different from the main screen>
8a and 8e show a video image to be displayed on the main screen in the stadium with 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3). The screen shows a control flow (server side) that displays a video different from the main screen.

FIG. 8e will be described. In S204 of FIG. 8a, when No is determined, the process proceeds to S226.

In S226, it is determined whether or not a video display different from the main screen is selected for the sub screen, and if a video display different from the main screen is selected for the sub screen, the process proceeds to S222. If a video display different from that of the main screen is not selected for the sub screen, the process proceeds to S201 (FIG. 8a). The selection of the display of a video different from the main screen on the sub screen is determined by inputting the information from the input means 202. In addition, the sub-screen displayed here displays a video different from the main screen among 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3). A broadcast camera other than the video image displayed in is selected.

S227 represents displaying a video on the main screen.

S228 indicates that a video different from the main screen is displayed on the sub screen. Here, the time axes of the main screen and the sub screen are synchronized by the time stamp.

In S229, it is determined whether or not the video is displayed or finished, and when the video is displayed or finished, the process proceeds to S230, and if the video display is not finished, the process proceeds to S227.

S230 indicates that the display of the video on the main screen is stopped.

When displaying a video different from the main screen on the sub screen, by synchronizing with the video image on the main screen using the time stamp, the audience can be provided with a powerful image of the game with multiple video images. Is now possible.

Even when displaying photos on the sub-screen, by synchronizing with the video image on the main screen using the time stamp, it became possible to provide the audience with a powerful image of the game with video and photos.

On the other hand, the match information and player information on the sub screen do not need to be strictly synchronized with the video image on the main screen. It has become possible to achieve this.

As mentioned above, it is very effective to change the way of synchronizing with the main screen depending on the content displayed on the sub screen.

In the first to fourth embodiments, the video is live, so if the video is encoded at this time, the coding with as low a delay as possible is considered. For example, in the compression of moving images, it is supported by not compressing in multiple planes.

Above, I thought about the live performance in the stadium. However, there can be live performances outside the stadium. In this case, except when the distance is short, there is a limitation on the communication capacity of video information, and coding with a high compression rate is considered.

Next, the replay is selected, the video that is the replay is displayed on the main screen, and the photo display that is the replay is displayed on the sub screen at the timing synchronized with the video image displayed on the main screen by the replay. The screen display control flow (server side) when selected will be described.

<Example 5: Replay (sub-screen displays a photo)>
9a and 9b show video images taken by 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3) as replays on the main screen in the stadium. At the same time, a control flow (server side) for displaying the video images taken by the 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3) as replies on the sub screen is shown.

In FIGS. 9a and 9b, S301 represents initialization.

In S302, it is determined whether or not the video replay display is selected, and if the video replay display is selected, the process proceeds to S303, and if the video replay display is not selected, the process proceeds to S310.

S303 represents obtaining the setting information of the camera.

In S304, it is determined whether or not the display of the replay photo is selected, and if the replay photo display is selected, the process proceeds to S305, and if the replay photo display is not selected, the process proceeds to S311.

S305 represents displaying the replay video on the main screen.

S306 represents displaying a photo of the replay on the subscreen. Here, the picture of the replay displayed on the sub screen is synchronized with the video image of the replay displayed on the main screen.

In S307, it is determined whether or not the display of the replay video is finished, and when the display or the end of the replay video is completed, the process proceeds to S308, and if the display of the replay video is not finished, the process proceeds to S305.

S308 indicates that the display of the replay video is stopped on the main screen.

S309 indicates that the display of the replay photo is stopped on the sub screen.

FIG. 9b will be described.

S311 represents displaying the replay video on the main screen.

S312 indicates that the photograph is not displayed on the sub screen.

In S313, it is determined whether or not the display of the replay video is finished, and when the display or the end of the replay video is completed, the process proceeds to S314, and if the display of the replay video is not finished, the process proceeds to S311.

S314 indicates that the display of the replay video is stopped on the main screen.

<Example 6: Replay (match information is displayed on the sub screen)>
9a and 9c show video images taken by 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3) as replays on the main screen in the stadium. At the same time, the control flow (server side) for displaying match information is shown on the sub-screen.

FIG. 9c will be described. In S304 of FIG. 9a, when No is determined, the process proceeds to S315.

In S315, it is determined whether or not the display of match information is selected during replay, and if the display of match information is selected during replay, the process proceeds to S316, and the display of match information is not selected during replay. And S301 (FIG. 9a). The selection of the display of the game information is determined by inputting the information from the input means 202. In addition, as the match information displayed here, the history of the score information up to now at the timing of displaying on the main screen, the current match status, and the like are displayed.

S316 represents displaying the replay video on the main screen.

S317 indicates that the match information of the replay is displayed on the sub screen. Here, the main screen and the sub screen are not synchronized on the exact time axis. Rough timing adjustment is fine.

In S318, it is determined whether or not the display of the replay video is finished, and when the display or the end of the replay video is completed, the process proceeds to S319, and if the display of the replay video is not finished, the process proceeds to S316.

S319 indicates that the display of the replay video is stopped on the main screen.

When displaying video information during replay on the sub-screen, by synchronizing with the video video at the time of replay on the main screen using the time stamp, the spectators can see the state of the game with video and photos. It is now possible to provide.

On the other hand, at the time of replay, the match information displayed on the sub screen does not need to be strictly synchronized with the video image on the main screen. It has become possible to achieve this.

As mentioned above, it is very effective to change the synchronization with the main screen depending on the content displayed on the sub screen during replay.

<Example 7: Replay (player information is displayed on the sub screen)>
9a and 9d show video images taken by 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3) as replays on the main screen in the stadium. At the same time, the control flow (server side) for displaying player information is shown on the sub-screen.

FIG. 9d will be described.

In S304 of FIG. 9a, when No is determined, the process proceeds to S320.

In S320, it is determined whether or not the display of player information is selected during replay, and if the display of player information is selected during replay, the process proceeds to S321, and the display of player information is not selected during replay. And S301 (FIG. 9a). At the time of replay, the selection of the display of player information is determined by inputting the information from the input means 202. Further, here, as the player information to be displayed at the time of reply, the information when the player is replaced, the information when the player gives a try, or the like is displayed. Here, in the stadium, the main screen and the sub screen are not synchronized on the exact time axis. Rough timing adjustment is fine.

S321 represents displaying the replay video on the main screen.

S322 indicates that the player information at the time of replay is displayed on the sub screen. Here, the main screen and the sub screen are not synchronized on the time axis.

In S323, it is determined whether or not the display of the replay video is finished, and when the display or the end of the replay video is completed, the process proceeds to S324, and if the display of the replay video is not finished, the process proceeds to S321.

S225 indicates that the display of the replay video is stopped on the main screen.

When displaying the replay video information on the sub screen, by synchronizing with the video video on the main screen using the time stamp, it is possible to provide the audience with a powerful video and photo of the game. It was.

On the other hand, the player information of the replay does not need to be closely synchronized with the video image on the main screen. It has become possible to achieve this.

As mentioned above, it is very effective to change the way of synchronizing with the main screen depending on the content displayed on the sub screen. This is the same as displaying the match information on the sub screen described above.

<Example 8: Replay (the sub screen displays video information different from the main screen)>
9a and 9e show replays of video images taken by 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3) on the main screen in the stadium. When doing so, the sub-screen shows a control flow (server side) that displays a video of the reply different from the main screen.

FIG. 9e will be described.

In S304 of FIG. 9a, when No is determined, the process proceeds to S325.

In S325, at the time of replay, it is determined whether or not the display of the replay video different from the main screen is selected for the sub screen, and the display of the replay video different from the main screen is selected for the sub screen. If so, the process proceeds to S326, and if the display of the replay video different from the main screen is not selected for the sub screen, the process proceeds to S301 (FIG. 9a). The selection of the display of the replay video different from the main screen on the sub screen is determined by inputting the information from the input means 202. Further, on the sub screen displayed here, the display of the replay video different from the main screen is among 122 (broadcast camera 1), 123 (broadcast camera 2), and 124 (broadcast camera 3). A broadcast camera other than the replay video image displayed on the main screen is selected.

S326 represents displaying the replay video on the main screen.

S327 indicates that the sub screen displays a replay video different from the main screen. Here, the time axes of the main screen and the sub screen are synchronized by the time stamp.

In S328, it is determined whether or not the display of the replay video is finished, and when the display or the end of the replay video is completed, the process proceeds to S329, and if the display of the replay video is not finished, the process proceeds to S326.

S329 indicates that the display of the replay video is stopped on the main screen.

When displaying a replay video different from the main screen on the sub screen, by synchronizing with the video image of the main screen using the time stamp, the audience can see the state of the game with multiple video images. It is now possible to provide video.

Even when displaying the replay photo on the sub screen in the stadium, by synchronizing with the video image of the replay on the main screen using the time stamp, the audience can see the game with video and photo. Video can now be provided.

On the other hand, the match information and player information on the sub screen do not need to be strictly synchronized with the video image of the replay on the main screen in the stadium. It has become possible to achieve this.

As mentioned above, it is very effective to change the way of synchronizing with the main screen depending on the content displayed on the sub screen.

In the fifth and eighth embodiments, the video is a replay of the video once accumulated. Here, due to the limitation of the storage capacity of video information, coding with a high compression rate is considered.

<In Example 9: (Example 1: Sub-screen displays a photo), live viewing outside the stadium is supported>
Example 1 is illustrated in FIGS. 8a and 8b. Here, we are considering displaying the captured video information, photo information, etc. on the main screen and sub screen in the stadium. Consider delivering these videos and photos far away. Here, compression of the amount of data is indispensable, and it is necessary to match the timing of the delay due to compression and decompression between moving images and photographs.

On the side receiving these videos and photos, after expanding the video and photo information, the time is adjusted with a time stamp and displayed on the main screen and sub screen of a distant venue. Here, the distribution side will be specifically described.

In the ninth embodiment, FIGS. 10a and 10b are 122 (broadcasting camera 1), 123 (broadcasting camera 2), and 124 (broadcasting camera 3), which are video images to be displayed on a main screen at a distant place. At the same time, the control flow (server side) of shooting a photographic image to be displayed on a sub-screen at a distant place is illustrated on 401 (terminal 1), 402 (terminal 2), and 403 (terminal 3). There is. Here, the main screen and the sub screen are close to each other at a distance.

In FIGS. 10a and 10b, S401 represents initialization.

In S402, it is determined whether or not the video is selected to be transmitted to a remote location, and if the video is selected to be transmitted to a remote location, the process proceeds to S403 and the video is transmitted to a remote location. If it is not selected, the process proceeds to S411.

S403 represents obtaining the setting information of the camera.

In S404, it is determined whether or not the transmission of the photo to a distant place is selected, and if the transmission of the photo to a distant place is selected, the process proceeds to S405 and the photo is transmitted to the distant place. If it is not selected, the process proceeds to S405.

S405 represents compressing and transmitting the video in order to display the video on the main screen at a remote location. The compression rate here shall be higher than the compression rate in the stadium.

S406 represents compressing and transmitting photographic information in order to display the photographic on a sub-screen at a distant place. The compression rate here shall be higher than the compression rate in the stadium.

Here, since the time required for encoding and decoding the compression of the moving image of S405 is longer than the time required for encoding and decoding the still image of S406, the still image information is at the timing of delaying the time of this difference. A signal may be transmitted.

S407 represents displaying a photograph on a sub-screen at a remote location. Here, the photo on the sub-screen at a remote location is synchronized with the video image displayed on the main screen at a remote location.

In S408, it is determined whether or not the transmission for displaying the video at a distant place has ended, and when the transmission or the end for displaying the video at a distant place is completed, the process proceeds to S408-2 and the distance is far away. If the transmission for displaying the video is not completed at the location, the process proceeds to S406.

S408-2 indicates that the transmission for displaying the video on the main screen at a distant place is stopped.

S408-3 indicates that the transmission for displaying the photograph on the sub screen at a distant place is stopped.

S411 represents other processing.

FIG. 10b will be described.

S412 represents compressing and transmitting the video for display on the main screen at a remote location.

S213 indicates that the photograph is not compressed and transmitted in order to display the photograph on a sub-screen at a remote location.

In S414, it is determined whether or not the transmission for displaying the video at a distant place has ended, and when the transmission for displaying the video at a distant place or the end is completed, the process proceeds to S415 and at a distant place. If the transmission for displaying the video is not completed, the process proceeds to S412.

S415 indicates that the transmission of the compressed video for displaying the video on the main screen is stopped at a remote place.

Here, not only the live viewing but also the replay display may be displayed on the main screen or the sub screen at a distant place.

<In Example 10: (Example 1: subscreen is a photo display), switching between the display inside the stadium and the display outside the stadium>
In the first embodiment, it is considered to display the video and the photo on the full screen and the sub screen in the studio, and in the ninth embodiment, the video and the photo are displayed on the full screen and the sub screen in a place far away from the stadium. Thought. These Examples 1 and 9 may be switched or both may be carried out. Therefore, it may be possible to select the implementation of the first embodiment only, the implementation of the ninth embodiment only, or the implementation of the first embodiment and the ninth embodiment.

<Example 11: (Example 1: About main screen and sub screen>
As a video system with a main screen and a sub screen, we have explained the case where the main screen and the sub screen are in the stadium, and the case where the main screen and the sub screen are located far away. In the past, many would look at the main and sub screens. Here, as one screen, the screen of the screen may be divided into two, and an area for displaying a moving image and an area for displaying a still image may be provided.

On the other hand, an individual may consider looking at the main screen and the sub screen. Specifically, the main screen may be a tablet and the sub screen may be a smartphone.

<Example 12: (Example 1: Change video shooting and photo shooting>
In the first embodiment, it is considered to display a video and a photo on a full screen and a sub screen in the studio. Here, a video image of the video is played on the main screen and a still image is synchronized with the sub screen ( As mentioned above, it is displayed at the timing (synchronized, which is almost the same time). Here, I think that it is better that the still image photograph gives the enjoyment unique to the still image as compared with the video photograph which is a moving image. For example, a still image may be shot in which the focus is focused on the player and the background is blurred. This can be done by reducing the aperture (F value) value.

In other words, displaying images that can only be expressed in photographs on the sub-screen is important for many visitors to understand the appeal of photographs. In order to realize this, the shooting conditions when taking a picture and the video conditions when taking a video image are changed.

According to the above-described embodiment, in a video system having a main screen and a sub-screen in a stadium, a means for shooting a video image of a moving image, a means for shooting photographic information of a still image, and shooting information in the ground are mainly used. It has become possible to have a means for displaying on a screen and a means for displaying still image information taken by a photographer on a sub-screen. In addition, the information displayed on the main screen is a video image of a moving image, and it has become possible to display photographic information, which is a still image, on the sub screen in synchronization with the viewing of the video image of the moving image. In addition, a means for recording video information of a moving image, a means for recording a video signal of a moving image including time stamp information, a means for reproducing video information of a moving image, and a means for taking a picture of a still image including the time stamp information. When playing back a video image of a moving image, it is possible to display still image information in synchronization with the video image.

As a result, in the stadium, while displaying the live image on the main screen on the live screen, it is possible to display the still image taken by the professional photographer in a form synchronized with this time, and the audience. People are now able to see powerful photographic images, and the taste of sports has increased. Especially in the replay, many goal scenes are played. Even during this playback, a still image can be displayed, and you can enjoy a powerful scene.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101 K camera 1, 102 K camera 2, 103 K camera 3,
120 main screen, 121 sub screen, 110 server,
111 input means, 112 base stations

Claims (23)

In a video system having a main screen and a sub screen, a means for taking a video image of a moving image, a means for taking a still image photographic information, a means for displaying the shooting information in the ground on the main screen, and a still image information taken by a photographer. A video system characterized by having a means for displaying an image on a sub-screen. The first aspect of claim 1, wherein the information displayed on the main screen is a video image of a moving image, and photographic information which is a still image is displayed on the sub screen in synchronization with the viewing of the video image of the moving image. Video system. There are means to record video information of video, means to record video signal of video including time stamp information, means to play video information of video, and means to take pictures of still images including time stamp information. The video system according to claim 1 or 2, wherein when the video image of the moving image is reproduced, still image information is also displayed in synchronization with the video image. Synchronized with the viewing of the video image of the video, when displaying the photo information that is a still image on the sub screen, the synchronization is to display the still image with the time stamp information close to the time stamp information of the video being played. The video system according to claim 2 or 3, characterized in that it is displayed. Synchronized with the viewing of the video image of the video, when displaying the photo information that is a still image on the sub screen, the synchronization is to display the still image with the time stamp information close to the time stamp information of the video being played. Claim 2 or claim 3 is characterized in that the moving image and the still image are not completely synchronized and are at substantially the same time because they are displayed and the still image is displayed for a certain period of time. The video system described in. There are means to record video information of video, means to record video signal of video including time stamp information, means to play video information of video, and means to take pictures of still images including time stamp information. The method according to any one of claims 1 to 5, wherein when recording a video image of a moving image, the video image is encoded with high compression and the still image information is also encoded with high compression. Video system. There are means to record video information of videos, means to record video signals of videos including time stamp information, means to play video information of videos, and means to take pictures of still images including time stamp information. However, when recording a video image of a video, the video image and still image information should be encoded with high compression compared to the compression rate of the video image and still image information when displayed in real time in the stadium. The video system according to any one of claims 1 to 5, wherein the video system is characterized. Means for recording video information of videos, means for recording video signals of videos including time stamp information, means for playing video information of videos, means for taking pictures of still images including time stamp information, videos It has a means to select whether to display the still image information in synchronization when playing the video image of the video, and when it is selected to display the still image information in synchronization when playing the video image of the moving image, When playing the video image of the video, the still image information is also displayed in synchronization with the video image, while when playing the video image of the video, the still image information is not selected to be displayed in synchronization. The video system according to any one of claims 1 to 7, wherein when the video image of the moving image is reproduced, the still image information is not displayed in synchronization with the video image. The information displayed on the main screen in the stadium is the video image of the video, and when displaying the photo information that is a still image on the sub screen in the stadium in synchronization with the viewing of the video image of the video, the video The video system according to any one of claims 1 to 8, wherein both the video signal and the photograph, which is a still image, are suppressed to low-delay compression. The information displayed on the main screen outside the stadium is the video image of the video, and when displaying the photo information that is a still image on the sub screen outside the stadium in synchronization with the viewing of the video image of the video, the video The video system according to any one of claims 1 to 9, wherein both the video signal and the photograph which is a still image are compressed with high compression. The information displayed on the main screen outside the stadium is synchronized with the compression rate of the video image of the video and the viewing of the video image of the video, and the sub screen outside the stadium is still when displaying the still image photographic information. The video system according to any one of claims 1 to 9, wherein the compression ratio of the image is higher than the compression ratio of the main screen and the sub screen in the stadium. The information displayed on the main screen outside the stadium is the video image of the video, and when displaying the photo information that is a still image on the sub screen outside the stadium in synchronization with the viewing of the video image of the video, the video Both the video signal and the still image are compressed with high compression, while the main screen outside the stadium is delayed in encoding with the still image photographic information with the sub screen outside the stadium. The video system according to any one of claims 1 to 11, wherein reproduction is performed in correspondence with a delay in communication. The information displayed on the main screen outside the stadium is the video image of the video, and when displaying the photo information that is a still image on the sub screen outside the stadium in synchronization with the viewing of the video image of the video, the video Both the video signal and the still image are compressed higher than the compression ratio inside the stadium, while the main screen outside the stadium is the code with the photographic information that is the still image with the sub screen outside the stadium. The video system according to any one of claims 1 to 11, wherein communication is performed by delaying the difference from the time required for encoding and decoding the moving image. Synchronized with the viewing of the video image of the video, when displaying the photo information that is a still image on the sub screen, the synchronization is to display the still image with the time stamp information close to the time stamp information of the video being played. The video system according to any one of claims 2 to 13, wherein the video system is displayed. Synchronized with the viewing of the video image of the video, when displaying the photo information that is a still image on the sub screen, the synchronization is to display the still image with the time stamp information close to the time stamp information of the video being played. Claims 2 to 2, wherein the moving image and the still image are not completely synchronized and are at substantially the same time because they are displayed and the still image is displayed for a certain period of time. 13. The video system according to any one of 13. The video system according to any one of claims 1 to 15, wherein the video displayed on the sub screen is moving image information other than the main screen. The video system according to any one of claims 1 to 15, wherein the video displayed on the sub-screen is management information of a match and a player. The video system according to any one of claims 1 to 15, wherein the video displayed on the sub screen can be switched to video information other than the main screen or management information of a match or a player. .. Claims 1 to claim characterized in that the delay control is changed depending on which information the video displayed on the sub-screen can be switched to video information other than the main screen or management information of a match or a player. Item 2. The video system according to any one of items 15. The video system according to any one of claims 1 to 19, wherein the video displayed on the main screen switches information from a plurality of cameras. The image displayed on the main screen is any one of claims 1 to 20, wherein when the information from the plurality of cameras is switched and used, the plurality of cameras have a time stamp function. The video system described in paragraph 1. The information displayed on the main screen is the video image of the video, and in synchronization with the viewing of the video image of the video, the camera at the time of shooting the video image when displaying the photo information which is a still image on the sub screen. The video system according to any one of claims 1 to 21, wherein the setting of the shooting conditions and the setting of the shooting conditions of the camera at the time of shooting the photographic information are changed. The information displayed on the main screen is the video image of the video, and in synchronization with the viewing of the video image of the video, when displaying the photo information which is a still image on the sub screen, when taking the photo information, the video The video system according to any one of claims 1 to 22, wherein a photo that cannot be realized by shooting with a camera at the time of shooting a video is taken.