JP2006165664A - Image transmission / reception system - Google Patents

Abstract

To provide an image transmission / reception system suitable for transmitting and receiving two streams of a shooting angle-of-view image and a wide-angle image in a limited communication band.
First image transmitting means for capturing, encoding, and transmitting an image of a first range, and capturing, encoding, and transmitting an image of a wider second range including the first range. An image transmission / reception unit comprising: a camera unit having two image transmission units; a reception unit that receives an image transmitted from the camera; and a viewer unit that includes an image display unit that composites and displays the received image. In the system, the first image is transmitted in a state in which a part of the image data is deleted by a range deleting unit that deletes image data corresponding to the range of the second image. The image processing apparatus includes combining means for reducing and combining the second image in accordance with the corresponding range of the first image from which data has been deleted by the range deleting means.
[Selection] Figure 1

Description

  The present invention relates to a transmission method of image data in a network camera capable of monitoring or photographing an image of a camera connected to a network from a remote place.

  Conventionally, when shooting a camera connected to a network from a remote location, the location where the camera is installed is far away and the situation around the subject cannot be seen directly. It was difficult to find and track. In order to solve this problem, a camera that simultaneously delivers a wide-angle image for viewing the surrounding image in addition to an image angle of view for photographing has been realized (see, for example, Patent Documents 1 and 2). .

JP 2000-032319 A (page 5, FIG. 1) JP 2000-341574 (5th page, FIG. 3)

  In a system composed of a network camera that delivers two streams of a shooting angle-of-view image and a wide-angle image, if two streams are delivered at the same time, the amount of data flowing on the network increases. There is a problem that it becomes difficult to transmit.

  However, in the above-described conventional example, since a wide-angle image and a zoom image each transmit a portion corresponding to the same area, there is a problem that the redundancy is high and the amount of communication is increased.

  The present invention has been made in view of the above technical problem, and an object thereof is to transmit and receive two streams of a captured angle image and a wide angle image in a limited communication band by reducing redundancy of data to be transmitted and received. It is an object to provide an image transmission / reception system suitable for performing the above.

  In order to achieve the above object, in the image transmission / reception system according to claim 1, the camera unit has means for deleting data in an area corresponding to the angle of view of the captured image from the wide-angle image data, and a part of the image is stored. Transmission is performed in a deleted state, and the viewer side has a means for reducing and compositing and displaying the captured image data so that the received wide-angle image matches the area where the data has been deleted. .

  The image transmission / reception system according to claim 2 is configured to have means for transmitting coordinates corresponding to the angle of view of the captured image in the coordinate system of the wide-angle image.

  In the image transmission / reception system according to claim 3, the range deletion means is configured to fill the range with a numerical value that cannot be in the image, and the viewer side detects a numerical range that cannot be included in the image. The range is detected, and the captured image is reduced and combined in the detected range.

  In the image transmission / reception system of claim 4, the configuration of the range deletion means is copied for each processing unit for encoding with reference to data of surrounding processing blocks or blocks of the previous frame. In this case, the motion vector is used.

  The image transmission / reception system according to claim 5 is configured to have a frame synthesizing unit for synthesizing and displaying the frame along the boundary line to be synthesized when the synthesizing unit performs synthesis.

  7. The image transmission / reception system according to claim 6, wherein the frame composition means is configured such that the range is filled with the color of the frame when the range is deleted, and at the time of composition, the reduced data of the photographed image is excluded from the surrounding frame width. Was configured to be synthesized.

  According to the present invention, since data corresponding to the angle of view of the captured image is deleted from the wide-angle image data and then transferred, data redundancy is reduced, and two streams of the captured angle-of-view image and the wide-angle image are stored. An image transmission / reception system with high transmission / reception efficiency is realized.

  Hereinafter, an embodiment of a system including a network camera and a viewer according to the present invention will be described with reference to the drawings.

<Embodiment 1>
1 and 2 are block diagrams showing a schematic configuration of an image transmission / reception system according to Embodiment 1 of the present invention. FIG. 1 is a schematic configuration of the camera unit, and FIG. 2 is a schematic configuration of the viewer unit.

  First, the camera unit in FIG. 1 will be described.

  The lens group 100 includes a plurality of lenses for optically projecting a subject image onto a CCD (photoelectric conversion element) 101. The CCD 101 is an element for taking a photographed image projected by the lens group 102 and has a pixel number of 1920 × 1080 dots. The image converted into the analog electric signal by the CCD 101 is subjected to noise removal processing by correlated double sampling in the CDS 102 and converted into a digital signal by the A / D conversion circuit 103. The TG 105 is a timing generator that generates signals for controlling the processing timing of the CCD 101, the CDS 102, and the ADC 103, and controls data transfer and electronic shutter.

  The image processing unit 110 performs image processing such as gamma conversion, color space conversion, and white balance on the digital signal output from the A / D conversion circuit 103, and 1920 in a YUV (4: 2: 2) format. Outputs a digital image of × 1080 dots.

  The image output from the image processing unit 110 includes an enlarged image creation unit 120 composed of an image cutout unit 121 and an enlargement processing unit 122, and a wide-angle image creation unit 130 composed of a range deletion processing unit 131 and a camera reduction processing unit 132. The two are passed.

  The image cutout unit 121 cuts out a rectangular image of 640 × 480 or smaller, and the enlargement processing unit 122 enlarges the image to 640 × 480 (digital zoom processing).

  The range deletion processing unit 131 deletes an image in the same range as the rectangular image cut out by the image cutout unit 121 from among the 1920 × 1080 images sent from the image processing unit 110. The image is deleted by filling all pixels in the range with the same color. Thereafter, the camera reduction processing unit 132 reduces the size to 640 × 360, which is 1/3 each of the vertical and horizontal directions.

  Each image created by the enlarged image creation unit 120 and the wide-angle image creation unit 130 is compressed by the compression processing unit 140 by the image compression processing unit 140. JPEG encoding is used for the compression processing here. Since the range deleted by painting with the same color in the range deletion processing unit 131 has only a DC component and no AC component, the compression efficiency by the JPEG compression processing in the image compression processing unit 140 is high.

  The wide-angle image and the enlarged image compressed by the image compression processing unit 140 are transmitted to the viewer unit (FIG. 2) through the network interface 150.

  In communication through the network interface 150, commands and the like are transmitted and received between the viewer unit and the camera unit.

  In addition, the cutout position of the enlarged image and the enlargement ratio are transmitted from the camera control unit 160 to the viewer unit through the network interface 150.

  Next, the viewer unit in FIG. 2 will be described.

  The image transmitted from the camera unit through the network interface 200 is sent to the image storage unit 210 and the image expansion processing unit 220.

  The image storage unit 210 receives only an enlarged image from the sent images, and creates and saves data in the Motion JPEG format from the continuously sent JPEG images.

  The image decompression processing unit 220 decompresses the transmitted JPEG image data, and sends the enlarged image to the enlarged image display unit 230 and the viewer reduction processing unit 240, and to the image composition processing unit 250.

  In the viewer reduction processing unit 240, the enlarged image is reduced to the original size by reciprocal of the enlargement processing 122 in FIG. 1, and further at the same reduction ratio (vertical, horizontal 1/3) as the camera reduction processing unit 132 in FIG. The image is reduced and adjusted to the same size as the transmitted wide-angle image, and then sent to the image composition processing unit 250.

  The image composition processing unit 250 synthesizes the image obtained by reducing the enlarged image by the viewer reduction processing unit 240 at the position deleted by the range deletion processing unit 131 of the camera unit in the wide-angle image, and sends it to the wide-angle image display unit 260. send. In this synthesis process, a frame is synthesized along the boundary to be synthesized. This indicates the position occupied by the enlarged image in the wide-angle image and also has the meaning of hiding the unnatural boundary to be synthesized. The frame synthesizing means may be realized by setting the color to be painted by the range deletion processing unit 131 of the camera unit as the color of the frame and leaving the frame width in the synthesizing process.

  The images sent to the enlarged image display unit 230 and the wide-angle image display unit 260 are each converted into a display format, sent to the display 270, and displayed.

  The viewer control unit 280 sets the reduction ratio of the viewer reduction processing unit 240 and the composition position of the image composition processing unit 250 based on the cutout position of the enlarged image sent from the camera unit through the network interface 200 and the enlargement rate. Do.

  In addition, when an operation such as panning, tilting, or zooming is instructed from the operation unit 290, the viewer control unit 280 transmits a command to the camera unit through the network interface 200.

  3 and 4 are diagrams showing an example of processing of image data processed by the camera unit and the viewer unit configured as shown in FIGS.

  FIG. 3 shows an example of processing performed by the camera unit. The 1920 × 1080 image output from the image processing unit 110 is (a), and the output from which part of the image is deleted by the range deletion processing unit 131 is (b). An image reduced to 640 × 360 by the reduction processing unit 132 is (c), and an image in which a part of (a) is enlarged to 640 × 480 by the enlargement processing unit 122 is (d).

  FIG. 4 shows a processing example of the viewer unit. The wide-angle image and the enlarged image expanded by the image expansion processing unit 220 are (e) and (f), and the output of the image composition processing 251 is (g), (f), and ( An example of an image displayed on the display 280 by combining g) is (h).

  FIG. 5 is a flowchart showing processing when the camera unit receives pan, tilt, and zoom commands from the viewer unit.

  When the camera unit receives the command, it is first determined whether or not it is a zoom operation (step S500). If it is a zoom operation, the setting of the enlargement processing unit 122 is changed (step S510), and in any case of pan, tilt and zoom Then, the position of the frame of the enlarged image is calculated (step S520), and the settings of the range deletion processing unit 131 and the image cutout unit 121 are changed (step S530). Thereafter, the frame position and enlargement ratio are transmitted to the viewer unit (step S540).

  FIG. 6 is a flowchart showing processing when the frame position and the enlargement ratio are transmitted from the camera unit to the viewer unit.

  If the enlargement ratio of the camera unit has been changed (step S600), the setting of the viewer reduction processing unit is changed to correspond to that (step S610), and then the setting of the image composition processing 251 is performed in accordance with the frame position information. Is changed (step S620).

<Embodiment 2>
In the first embodiment, the information for notifying the composite position is explicitly transmitted from the camera control unit 160 to the viewer control unit 280 separately from the image through the network interfaces 150 and 200. However, in this embodiment, image data is transmitted. It is characterized in that it is not necessary to separately transmit information for informing the composite position.

  Since the diagram representing the present embodiment is the same as that of the first embodiment except for FIG. 2, it is not shown. FIG. 7 is a modification of FIG. 2 of the first embodiment.

  In the present embodiment, the deletion process in the range deletion process 131 of FIG. 1 is configured to fill the range with a numerical value that is impossible in the image. The numerical value that cannot be in the image is a numerical value that is out of the range of the luminance value 16 to 235 and the color difference value 16 to 240 that are specified by CCIR601, for example, the luminance value, the formula difference value, and the like. Both are numerical values such as 0.

  In the block diagram of the viewer unit in FIG. 7, a composite position detection unit 700 is added as compared to FIG. 2 in the first embodiment. The composite position detection unit 700 inspects the image received from the image expansion processing unit 220, finds a numerical value that cannot be found in the image, detects the coordinate range of the composite position, and sends it to the viewer control unit 200. The viewer control unit also calculates a reduction ratio from the coordinate range of the synthesis position, and sets the viewer reduction processing unit 250 and the image synthesis processing unit 251 by the same processing as in FIG. 6 of the first embodiment.

  With the above operation, the present embodiment is characterized in that only image data is transmitted and there is no need to separately transmit information for notifying the synthesis position.

<Embodiment 3>
In the first and second embodiments, JPEG is used for image data compression in the image compression processing unit 140 and the image expansion processing unit 220. In this embodiment, a format such as MPEG2 or MPEG4 is used for compressing the image.

  The range deletion unit 131 of the present embodiment attempts to reduce the amount of data at the time of compression by using data that can be expressed as a motion vector in a format such as MPEG2 or MPEG4 at the time of compression.

  FIG. 8 is a diagram illustrating an execution example of the range deleting unit 131.

  Arranged in a lattice form are macroblocks which are processing units such as MPEG2 or MPEG4. In the figure, the shaded part is the deletion range, the macroblock 801 is copied and copied 802, the 802 is copied, 803 and 810 are copied, and 811 is copied. ing. As a result, at the time of compression by the image compression processing unit 140, motion vectors are converted in the form of forward reference in the same screen, so that the data amount can be reduced. In this example, the motion vector is based on the intra-screen reference, but may be copied between frames to be the motion vector based on the inter-frame reference.

  Further, the compression format is not limited to JPEG, MPEG2, and MPEG4, and even in other compression formats, the data in the deletion range is discarded and the range deletion unit 131 is configured by a representation with a smaller data transfer amount. Thus, the present invention can be realized.

<Other embodiments>
The camera unit described in the above embodiment has shown an example in which an image from an input unit including a pair of lenses and a CCD is digitally cut out and enlarged to create a photographed image. It is not limited, and a wide-angle image and a photographed image are created from two pairs of lenses having different angles of view and a two-lens input unit composed of a CCD, or a lens optically inputs a light signal input in common. It is also possible to apply the present invention to a camera having a configuration in which a wide-angle image and a captured image are created using a separate CCD using a prism or a half mirror.

  The network connection means between the camera and viewer is not limited as long as it can transmit and receive images without limitation such as wireless or wired.

  Further, the display in the viewer of the above embodiment is performed by arranging the wide-angle image and the photographed image at the same time. However, the present invention is not limited to this, and the wide-angle image and the photographed image are alternately switched. Can also be applied to a so-called picture-in-picture display device in which the other screen is superimposed on a part of one image as a sub-screen.

  Further, the viewer according to the above embodiment has been described as an apparatus including dedicated image composition hardware or the like. However, in the present invention, the viewer may be executed as an application of a PC connected to a network as a viewer.

It is a block diagram which shows schematic structure of the camera part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a block diagram which shows schematic structure of the viewer part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a figure which shows the example of a process of the image data in the camera part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a figure which shows the example of a process of the image data in the viewer part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a flowchart figure which shows the PTZ process of the camera part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a flowchart figure which shows the synthetic | combination position change process of the viewer part of the image transmission / reception system which concerns on Embodiment 1 of this invention. It is a block diagram which shows schematic structure of the viewer part of the image transmission / reception system which concerns on Embodiment 2 of this invention. It is a figure which shows the mode of the range deletion process of the camera part of the image transmission / reception system which concerns on Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 130 Wide-angle image creation part 131 Range deletion process part 120 Enlarged image creation part 121 Image clipping part 140 Image compression process part 150 Network interface 200 Network interface 210 Image storage means 220 Image expansion process part 250 Image composition process part 700 Composition position detection part

Claims (6)

First image transmission means for capturing, encoding, and transmitting an image of the first range, and second image transmission for capturing, encoding, and transmitting an image of a wider second range including the first range In an image transmission / reception system comprising: a camera unit having a means; a receiving unit that receives an image transmitted from the camera; and a viewer unit having an image display unit that composites and displays the received image.
The first image is transmitted in a state where a part of the image data is deleted by a range deleting unit that deletes image data corresponding to the range of the second image, and the viewer deletes the range. An image transmission / reception system comprising combining means for reducing and combining the second image in accordance with a corresponding range of the first image from which data has been deleted by the means.   The image transmission / reception system according to claim 1, wherein the camera unit includes an information transmission unit that transmits data for specifying a deletion range and a reduction ratio by the range deletion unit.   The range deletion unit is configured to fill the range with a numerical value that cannot be in the image, and the viewer includes a range detection unit that detects a numerical range that cannot be included in the image, and the combining unit 2. The image transmission / reception system according to claim 1, wherein the second image is reduced and combined with the range detected by the range detection means.   The range deletion means is configured to copy data by referring to data of surrounding processing blocks or blocks of the previous frame for each processing unit when encoding the image of the first range. 2. The image transmission / reception system according to claim 1, wherein the image transmission / reception system is expressed by a motion vector.   2. The image transmission / reception system according to claim 1, further comprising a frame synthesizing unit for synthesizing and displaying a frame along a boundary line to be synthesized when synthesizing by the synthesizing unit.   The frame synthesizing unit fills the corresponding range with the color of the frame at the time of the range deleting unit, and the synthesizing unit synthesizes the reduced data of the second image with a portion excluding the surrounding frame width in the range. 6. The image transmission / reception system according to claim 5, wherein the image transmission / reception system is configured as described above.

Images