JP2009088666A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To record or display photographed image data of high resolution when abnormality has occurred by detecting abnormality by image analysis without using a processor structure which can bear a high load. <P>SOLUTION: The image processor is provided with: a low resolution image storage means 13a for storing a received photographed image data of the low resolution and time of day information; a high resolution image storage means 13b for storing the received photographed image data of high resolution and time of day information; an image judging means 11c for determining whether or not the photographed image data of high resolution before and after the time of day of the photographed image data of the low resolution stored last are stored; an abnormality detecting means 11d for detecting abnormality by using the photographed image data of low resolution stored last when it is determined that the photographed image data of high resolution are stored; an image selection means 11e for selecting a photographed image of high resolution at a time of day closest to the time of day of the photographed image data of low resolution stored last when abnormality is detected; and an output control means 11f for causing an optical disk drive 14 to record the selected photographed image data of the high resolution. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention is an image processing apparatus that is connected to a camera via a network, receives captured image data captured by the camera, and records or displays the captured image data based on an image analysis result of the received captured image data. About.

  In many hotels, commercial buildings, convenience stores, and the like, a surveillance camera system including a surveillance camera and an image analysis device that analyzes an image captured by the surveillance camera is often installed for the purpose of crime prevention.

  In addition, with the recent spread of networks, network cameras that can be connected to networks have become popular. Therefore, there is a surveillance camera system that includes such a network camera and an image analysis device that receives captured image data captured by the network camera and performs image analysis of the received captured image data.

  Here, the network camera transmits the captured image data, for example, a frame image of 720 × 480 pixels to the image analysis apparatus at a transmission rate of 30 frames per second. Therefore, the image analysis apparatus needs to perform image analysis processing on the photographic image data having such an enormous amount of data, and has a processing capability sufficient to withstand this load. Therefore, in order to realize this, there has been a problem that costs such as manufacturing cost increase as a whole apparatus such as mounting a high-performance CPU (Central Processing Unit).

Therefore, in Patent Document 1, a reduced image is created from an original image input from a camera, a primary determination is performed by analyzing the reduced image, and a region to be analyzed in detail is narrowed down by this primary determination, and the narrowed region An image sensor device that analyzes an original image in detail has been proposed.
JP 2006-127195 A

  However, in the image sensor device described in Patent Document 1, a reduced image is created from the original image input from the camera, and this reduction processing itself is a heavy load on the device. For this reason, in order to realize the image sensor device described in Patent Document 1, there is a problem that the cost of manufacturing the device as a whole increases, such as mounting a high-performance MPU (Micro Processing Unit).

  The present invention has been made in view of the above problems, and can detect an abnormality by image analysis and record or highlight an image at the time of an abnormality without using a device configuration that can withstand a high load. An object is to provide an image processing apparatus.

  In order to solve the above object, the first feature of the image processing apparatus according to the present invention is that a low-resolution captured image data and a high-resolution captured image data captured by the camera are connected to the camera via a network. An image processing apparatus that records or displays received high-resolution captured image data based on an image analysis result of the received low-resolution captured image data, and receives the low-resolution captured image data received from the camera And a low-resolution image storage unit that stores the time when the low-resolution captured image data is received, or time information indicating the time when the low-resolution captured image data is transmitted from the camera, The high-resolution captured image data received from the camera, the time when the high-resolution captured image data was received, or the high-resolution captured image data Before and after the time indicated by the time information of the low-resolution captured image data stored last in the high-resolution image storage means and the low-resolution image storage means stored in association with the time information indicating the time when the image is transmitted The high-resolution captured image data corresponding to the time information within a predetermined time is stored in the high-resolution image storage unit by the image determination unit that determines whether or not the high-resolution image storage unit stores the high-resolution captured image data. An abnormality detection unit that performs image analysis using the low-resolution captured image data stored last, and detects an abnormality on the image based on the image analysis result; When an abnormality on the image is detected, the time closest to the time indicated by the time information corresponding to the last-stored low-resolution captured image data from the high-resolution image storage means Image selecting means for selecting high-resolution captured image data corresponding to the time information shown, and causing the selected high-resolution captured image data to be recorded in the recording unit, or selecting the selected high-resolution captured image data on the display unit Output control means for highlighting.

  In order to solve the above-described object, the second feature of the image processing apparatus according to the present invention is that, when the abnormality detection unit determines that the image determination unit stores the high resolution image storage unit, the low resolution When the difference between the low-resolution captured image data corresponding to the oldest time information stored in the image storage means and the low-resolution captured image data stored last is equal to or greater than a predetermined threshold, The output control means determines that there is an abnormality, and the output control means stores the low resolution corresponding to the time information before the predetermined time from the time information corresponding to the low resolution photographed image data stored last. The photographic image data is deleted, and the selected high-resolution photographic image data is recorded in the recording unit, or the selected high-resolution photographic image data is highlighted on the display unit.

  In order to solve the above object, the third feature of the image processing apparatus according to the present invention is that, when the abnormality detection unit determines that the image determination unit stores the high-resolution image storage unit, In the stored low-resolution captured image data, when the number of pixels indicating a predetermined color is equal to or greater than a predetermined threshold, it is determined that there is an abnormality.

  According to the image processing apparatus of the present invention, it is possible to detect abnormality by image analysis and record or highlight high-resolution captured image data at the time of abnormality without using a device configuration that can withstand a high load. .

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  The first embodiment of the present invention is applied to a surveillance camera system using a network camera, receives low-resolution captured image data and high-resolution captured image data captured by the network camera, and captures the latest low image. An image processing apparatus for detecting an abnormality depending on whether or not a difference between image data and low-captured captured image data before a predetermined time is equal to or greater than a threshold, and storing high-resolution captured image data near the time when the abnormality is detected on an optical disc Will be described as an example.

<Configuration of surveillance camera system>
FIG. 1 is a configuration diagram illustrating a configuration of a surveillance camera system to which an image processing apparatus that is Embodiment 1 of the present invention is applied.

As shown in FIG. 1, the monitoring camera system 100 includes an image processing apparatus 1, network cameras C ₁ and C _2, and a server 3, each of which is connected via a network 2.

Although not shown, the network cameras C ₁ and C ₂ are connected to the network 2, an optical system using a lens or the like, an imaging unit having a photoelectric conversion circuit such as a CCD, an encoding unit that encodes the captured image, and the network 2. A network interface unit for communication, and the captured captured image data is encoded and transmitted to the network 2. The network cameras C ₁ and C ₂ can transmit a plurality of captured image data having different resolutions. For example, a VGA (Video Graphics Array: 640 × 480 pixels) image and a QQVGA (Quarter Quarter VGA: 160 × 120 pixels). The captured image data can be transmitted. Here, a VGA image is described as high-resolution captured image data, and a QQVGA image is described as low-resolution captured image data. The network cameras C ₁ and C ₂ include an analog camera, an A / D converter that performs analog / digital (A / D) analysis of analog captured image data captured by the analog camera, and A / D converted captured image data. It is good also as a structure provided with the encoding part to encode and the network interface part for connecting to the network 2 and communicating.

  Although not shown, the server 3 includes a storage device such as a hard disk or an optical disk drive, provides various services, and stores high-resolution captured image data received from the image processing device 1 in the storage device.

  The image processing apparatus 1 includes a CPU 11, a network interface 12, a memory 13, an optical disk drive 14, a hard disk 15, and a display unit 16, which are connected via a bus 20 as shown in FIG.

Network interface 12 is a communication device such as a network card, by connecting to the network 2 via the network interface 12 may be a network camera C ₁ and C ₂ receives the photographed image data photographed.

  The memory 13 includes a low-resolution image storage unit 13a and a high-resolution image storage unit 13b as functional blocks.

Low resolution image storing means 13a, the network camera C ₁ and the low resolution of the captured image data received from the C _2, the time receiving the captured image data of the low resolution or low-resolution captured image data is the network camera C ₁ of And time information indicating the time transmitted from C ₂ is stored in association with each other.

High resolution image storage unit 13b, the network cameras C ₁ and C ₂ and the high-resolution captured image data received from, the high-resolution time receives the captured image data, or high-resolution captured image data is the network camera C ₁ of And time information indicating the time transmitted from C ₂ is stored in association with each other.

The optical disk drive 14 records high-resolution captured image data captured by the network cameras C ₁ and C _{2 on the} optical disk in accordance with instructions from the CPU 11.

In addition to a program such as an operating system, the hard disk 15 stores an image processing program executed by the image processing apparatus 1 according to the first embodiment of the present invention. Further, the hard disk 15 records high-resolution captured image data captured by the network cameras C ₁ and C _{2 in} accordance with instructions from the CPU 11.

  The data recorded on the optical disk drive 14 may be recorded on the hard disk 15 and the optical disk drive 14 may not be provided.

The display unit 16 includes a liquid crystal display, a CRT display, and the like, and displays captured image data captured by the network cameras C ₁ and C _{2 in} accordance with instructions from the CPU 11.

  The display unit 16 may be built in the image processing apparatus 1 or may be connected to the image processing apparatus 1 as an external device.

  The CPU 11 of the image processing apparatus 1 performs central control of the apparatus itself and executes an image processing program, so that the received image determination means 11a, the storage control means 11b, the image determination means 11c, An abnormality detection unit 11d, an image selection unit 11e, and an output control unit 11f are mounted.

Receiving image determining unit 11a of the CPU11 is transmitted from the network camera C ₁ and C _2, or captured image data received by the network interface 12 via the network 2 is a photographic image data of high resolution, or low resolution shooting It is determined whether it is image data.

  When it is determined that the captured image data received by the received image determination unit 11 a of the CPU 11 is low-resolution captured image data, the storage control unit 11 b of the CPU 11 stores the received low-resolution captured image data in the memory 13. It is stored in the low resolution image storage means 13a. On the other hand, when it is determined that the captured image data received by the received image determination unit 11a of the CPU 11 is high-resolution captured image data, the received high-resolution captured image data is stored in the high-resolution image storage unit 13b of the memory 13. Remember me.

  The image determination unit 11c has a high-resolution image storage unit that stores high-resolution captured image data corresponding to time information before and after the time indicated by the time information of the low-resolution captured image data stored last in the low-resolution image storage unit 13a. It is determined whether or not it is stored in 13b.

  The abnormality detection unit 11d has a low resolution corresponding to the oldest time information stored in the low resolution image storage unit 13a when the image determination unit 11c determines that the image is stored in the high resolution image storage unit 13b. When the difference between the photographed image data and the last stored low-resolution photographed image data is equal to or greater than a predetermined threshold, it is detected as abnormal.

  When an abnormality is detected by the abnormality detection unit 11d, the image selection unit 11e indicates the time indicated by the time information corresponding to the low-resolution captured image data stored last from the high-resolution image storage unit 13b of the memory 13. A high-resolution captured image corresponding to the time information indicating the time closest to is selected.

  The output control unit 11f causes the low-resolution image storage unit 13a to delete the low-resolution photographic image data corresponding to the time information before a predetermined time from the time information corresponding to the low-resolution photographic image data stored last. The selected high-resolution captured image is recorded on the optical disk drive 14.

<Operation of Surveillance Camera System 100>
Next, the operation of the monitoring camera system 100 to which the image processing apparatus 1 that is Embodiment 1 of the present invention is applied will be described with reference to the drawings.

  FIG. 2 is a diagram for explaining transmission / reception of photographed image data in the monitoring camera system 100 to which the image processing apparatus 1 according to the first embodiment of the present invention is applied. FIG. FIG. 5B is a diagram illustrating a case where captured image data is transmitted / received in a unicast packet.

As shown in FIG. 2 (a), the network camera C ₁ and C ₂ is capable of transmitting a plurality of photographed image data. In FIG. 2 (a), the network camera C ₁ and C ₂ is transmitting a high resolution photographic image data to the network 2 in multicast m1. In general, since the multicast packet reaches all devices connected to the network 2, the high-resolution captured image data transmitted from the network cameras C ₁ and C ₂ arrives at the image processing apparatus 1 and reaches the server 3. Also reach.

Similarly, the network cameras C ₁ and C ₂ transmit low-resolution captured image data to the network 2 by multicast m2. The low-resolution image data transmitted from the network cameras C ₁ and C ₂ reaches the image processing apparatus 1 and also reaches the server 3.

Further, as shown in FIG. 2 (b), the network camera C ₁ and C _2, it is also possible to send multiple unicast packets. In the example shown in FIG. 2 (b), the network camera C ₁ and C _2, and transmits the high-resolution image data in unicast u1 to the network 2, transmits a low-resolution image data in unicast u2 to the network 2 To do. In general, since the unicast packet arrives at the particular device in the network 2, in the example shown in FIG. 2 (b), the captured image data transmitted from the network camera C ₁ and C ₂ an image processing apparatus as specified Reach only one.

Thus, the network camera C ₁ and C _2, multicast packet, or any of the may transmit the captured image data by using unicast packets.

Further, the network cameras C ₁ and C ₂ can also add time information at the time of transmission to the captured image data to be transmitted, regardless of whether multicast packets or unicast packets are used. Specifically, the network cameras C ₁ and C ₂ can add time information to the high-resolution captured image data, and also add time information to the low-resolution captured image data and transmit it to the network 2. it can. As the time information, for example, a RTP (Real-time Transport Protocol) time stamp described in RFC1889 is used.

Here, the timing at which the network cameras C ₁ and C ₂ transmit the captured image data is set for each network camera. For example, the network cameras C ₁ and C ₂ transmit the high-resolution captured image data at the time t 1 and the low resolution at the time t 2. The high-resolution photographic image data and the low-resolution photographic image data may be transmitted in order, as in the case of transmitting the photographic image data. You may make it transmit simultaneously.

The captured image data transmitted by the network cameras C ₁ and C ₂ does not always reach the image processing apparatus 1 in the order of transmission. This and size of the photographed image data to be transmitted, due to the load conditions of the network 2 such as, for example, network cameras C ₁ and C ₂ may transmit a high resolution photographic image data of the t1 time, the subsequent time t2 When the low-resolution captured image data is transmitted, the image processing apparatus 1 may receive the low-resolution captured image data at time t3 and then receive the high-resolution captured image data at time t4.

In this way, the image processing apparatus 1 receives the captured image data transmitted from the network cameras C ₁ and C ₂ via the network 2. Then, the image processing apparatus 1 that has received the captured image data performs an image temporary storage process and an image recording process, as will be described in detail below.

≪Temporary image storage process≫
FIG. 3 is a flowchart showing the procedure of the temporary image storage process in the image processing apparatus 1 according to the first embodiment of the present invention.

As shown in FIG. 3, the received image determination unit 11a of the image processing apparatus 1, the captured image data transmitted from the network camera C ₁ and C _2, via the network 2 and received by the network interface 12 obtains (step S101).

  Next, the received image determination unit 11a determines whether the received captured image data is high-resolution captured image data or low-resolution captured image data (step S102). Specifically, the port number for receiving high-resolution captured image data and the port number for receiving low-resolution captured image data are assigned as different port numbers, and depending on the destination port number of the received packet, It is determined whether the received captured image data is high-resolution captured image data or low-resolution captured image data.

  In step S102, if the received image determination unit 11a determines that the received captured image data is low-resolution captured image data, whether or not time information is added to the received low-resolution captured image data. Is determined (step S103).

  If it is determined in step 103 that the time information is added to the received low-resolution captured image data, the received image determination unit 11a receives the received low-resolution captured image data and the low-resolution captured image data. Is associated with the time information added to and stored in the low-resolution image storage means 13a of the memory 13 (step S105).

  On the other hand, if the received image determination unit 11a determines in step S103 that the time information is not added to the received low-resolution captured image data, the time when the received low-resolution captured image data is received is the time information. (Step S104), the generated time information and the received low-resolution captured image data are associated with each other and stored in the low-resolution image storage unit 13a of the memory 13 (step S105).

  If the received image determination unit 11a determines in step S102 that the received captured image data is high-resolution captured image data, the received high-resolution captured image data is the same as in step S103. It is determined whether or not time information is added to (step S106).

  In step S106, when the received image determination unit 11a determines that time information is added to the received high-resolution captured image data, the received high-resolution captured image data and the high-resolution captured image data Is associated with the time information added to and stored in the high-resolution image storage means 13b of the memory 13 (step S108).

  On the other hand, in step S106, when the received image determination unit 11a determines that time information is not added to the received high-resolution captured image data, this high-resolution captured image is the same as in step S107. The time when the data is received is generated as time information (step S107), and the generated time information is associated with the received high-resolution captured image data and stored in the high-resolution image storage unit 13b of the memory 13 (step S108). ).

≪Image recording process≫
FIG. 4 is a flowchart showing a processing procedure of image recording processing in the image processing apparatus 1 according to the first embodiment of the present invention.

  First, the image determination unit 11c of the CPU 11 determines whether or not low-resolution captured image data is newly stored by the storage control unit 11b (step S201).

  If it is determined in step S201 that low-resolution captured image data has been newly stored, the time information of the newly-stored low-resolution captured image data stored in the low-resolution image storage unit 13a is stored. It is determined whether or not high-resolution captured image data corresponding to time information before and after the indicated time is stored (step S202).

  Specifically, the image determination unit 11c reads time information associated with the low-resolution captured image data stored last from the low-resolution image storage unit 13a of the memory 13. Then, the image determination unit 11c reads the time information associated with the high-resolution captured image data from the high-resolution image storage unit 13b of the memory 13, and the low-resolution image stored last in the read time information. It is determined whether or not time information corresponding to the time before and after the time indicated by the time information associated with the captured image data exists. If this time information exists, the image determination unit 11c stores high-resolution captured image data corresponding to the time information before and after the time indicated by the time information of the low-resolution captured image data stored last. It is determined that

  If it is determined in step S202 that the corresponding high-resolution captured image data is stored, the abnormality detection unit 11d performs the image analysis to store the last stored low-resolution captured image data and the oldest low-resolution image data. The difference from the captured image data is calculated, and it is determined whether there is an abnormality based on the result (steps S203 and S204).

  Specifically, the abnormality detection unit 11d reads low-resolution captured image data associated with the oldest time information from the low-resolution image storage unit 13a of the memory 13. Then, the abnormality detection unit 11d divides the read low-resolution captured image data and the last stored low-resolution captured image data into predetermined image areas, and both captured image data for each of the divided image areas. The difference is calculated. Then, the abnormality detecting unit 11d cumulatively adds the calculated differences, and detects the abnormality when the accumulated difference total value exceeds a predetermined threshold value.

  Here, if this threshold is too high, it is determined that the two photographed image data are always similar, and it is impossible to detect an abnormality, and if it is too low, an abnormality is detected even when the difference between the two photographed image data is as small as, for example, noise. End up. For this reason, it is necessary for the provider or the like to calculate in advance an appropriate value based on the actual measurement, and for the provider or the user to set an appropriate value in advance.

  Next, when an abnormality is detected by the abnormality detection unit 11d, the image selection unit 11e indicates the time information of the low-resolution captured image data stored last from the high-resolution image storage unit 13b of the memory 13. High-resolution captured image data associated with time information indicating the time closest to the time is selected (step S205).

  Then, the output control unit 11f controls the optical disc drive 14 to record the high-resolution captured image data selected by the image selection unit 11e on the optical disc (step S206).

  Note that the output control unit 11f may store the selected high-resolution photographic image data in the hard disk 15 instead of storing the high-resolution photographic image data selected by the image selection unit 11e in the optical disc. Then, it may be stored in a hard disk built in the server 3 connected to the network 2 via the network interface 12.

  Next, the output control unit 11f deletes the high-resolution captured image data selected in step S205 from the high-resolution image storage unit 13b of the memory 13 together with the time information associated therewith (step S207).

  Further, the output control unit 11f outputs the high-resolution captured image data associated with the time information older than the time indicated by the time information of the high-resolution captured image data selected in step S205 together with the time information to the high resolution of the memory 13. It deletes from the image memory | storage means 13b (step S209).

  Next, the output control unit 11f deletes the low-resolution captured image data associated with the time information before the predetermined time from the low-resolution image storage unit 13a of the memory 13 together with the time information (step S210).

Here, if the predetermined time is too long, it is necessary to provide a large-capacity memory 13, and the cost of the apparatus increases. Further, when the predetermined time is too short, when the motion of the object network camera C ₁ and C ₂ is photographed is slow, it can not be detected abnormalities. For this reason, it is necessary for the provider or the like to calculate in advance an appropriate value based on the actual measurement, and for the provider or the user to set an appropriate value in advance.

  On the other hand, if no abnormality is detected by the abnormality detection unit 11d in step S204, the image selection unit 11e receives the last stored low resolution from the high resolution image storage unit 13b of the memory 13, as in the process of step S205. The high-resolution captured image data associated with the time information indicating the time closest to the time indicated by the time information of the captured image data is selected, and the process proceeds to step S209 (step S208).

  In steps S203 and S204, the abnormality detection unit 11d calculates the difference between the two captured image data for each divided image area, and the difference total value obtained by cumulatively adding the difference exceeds a predetermined threshold value. However, it is also possible to perform image analysis using higher-resolution captured image data and detect the abnormality based on the image analysis result.

  Specifically, the abnormality detection unit 11d extracts an image area having a high difference from the image areas in the divided low-resolution captured image data. Then, the abnormality detection unit 11d outputs the high-resolution captured image data associated with the time information indicating the time closest to the time indicated by the time information of the low-resolution captured image data stored last, to the high-resolution image storage unit 13b. And the high-resolution captured image data associated with the oldest time information is read from the high-resolution image storage unit 13b. Then, the abnormality detection unit 11d extracts a high-resolution image area corresponding to the extracted low-resolution image area having a high difference from the read high-resolution captured image data, and extracts the extracted high-resolution image data. The image area is further divided into predetermined second image areas, and the difference between the high-resolution image areas is calculated for each divided second image area. Then, the abnormality detecting unit 11d cumulatively adds the calculated differences, and when the accumulated second difference value exceeds a predetermined second threshold value, the abnormality detecting unit 11d detects the abnormality. Good.

  As described above, according to the image processing apparatus 1 that is Embodiment 1 of the present invention, it is determined whether or not an abnormality is detected based on low-resolution captured image data. Since the photographed image data is recorded on the optical disk, the high-resolution photographed image data in the vicinity where the abnormality is detected can be recorded without a process that places a heavy load on the apparatus such as image reduction.

  FIG. 5 shows an image temporary storage process and an image recording process when the low-resolution captured image data and the high-resolution captured image data arrive in the order of transmission in the image processing apparatus 1 that is Embodiment 1 of the present invention. It is a time chart. In addition, T1-T19 in FIG. 5 has shown the certain time point on the time-axis, and has shown the code | symbol in ascending order along progress of time.

As shown in FIG. 5, the image processing apparatus 1, in the T11 time, receives the low-resolution photographic image data of the time information indicating that the network camera C ₁ are transmitted to the T1 time it is added. Then, the image processing apparatus 1 stores the received low-resolution captured image data in the memory 13. Similarly, in the T12 time, receives the high-resolution captured image data of the time information is added to indicate that it has been transmitted from the network camera C ₁ to time T2 stored in the memory 13, at T13 the time, the network camera C It receives the low-resolution photographic image data of the time information is added to indicate that it has been transmitted from the ₁ to the time T4 stored in the memory 13, at T14 the time, that has been transmitted from the network camera C ₁ to T5 time High-resolution captured image data to which the time information shown is added is received and stored in the memory 13. Here, it is assumed that the time from the time T2 to the time T4 is longer than the time from the time T4 to the time T5.

  At time T13, new low-resolution captured image data is stored, and at time T14, time information T2 and time T5 are time information before and after the last stored low-resolution captured image data at time T4. Since the high-resolution captured image data is stored, at time T14, the abnormality detecting unit 11d starts image analysis based on the last stored low-resolution captured image data. At time T15, the abnormality detection unit 11d detects an abnormality as a result of the analysis.

  Next, since an abnormality has been detected, at time T16, the image selection unit 11e receives from the high resolution image storage unit 13b of the memory 13 the high resolution associated with the time information at time T5, which is the time closest to time T4. Then, the output control unit 11f controls the optical disk drive 14 to record the selected high-resolution captured image data at time T5 on the optical disk.

  At time T17, the output control unit 11f deletes the selected high-resolution captured image data at time T5 from the high-resolution image storage unit 13b of the memory 13 together with time information associated therewith.

  Next, at time T18, the output control unit 11f reads from the high-resolution image storage unit 13b of the memory 13 at the time T2 which is a time older than the time indicated by the time information of the selected high-resolution captured image data at time T5. The high-resolution captured image data is deleted together with the time information associated therewith.

  Further, at the time T19, the output control unit 11f deletes the low-resolution captured image data at the time T1 which is a time before a predetermined time from the low-resolution image storage unit 13a of the memory 13 together with this time information.

  FIG. 6 shows image temporary processing in the case where the order in which low-resolution captured image data and high-resolution captured image data are transmitted does not match the order of arrival in the image processing apparatus 1 that is Embodiment 1 of the present invention. 3 is a time chart showing a storage process and an image recording process. In addition, T1-T30 in FIG. 6 has shown the certain time point on the time-axis, and has shown the code | symbol in ascending order along progress of time.

As shown in FIG. 6, the image processing apparatus 1, in the T21 time, receives the high-resolution captured image data of the time information indicating that the network camera C ₁ are transmitted to the T1 time it is added. The image processing apparatus 1 stores the received high-resolution captured image data and time information in the memory 13 in association with each other. Similarly, in the T22 time, receives the low-resolution photographic image data of the time information is added to indicate that it has been transmitted from the network camera C ₁ to time T2 stored in the memory 13, at T23 the time, the network camera C _The high-resolution captured image data to which time information indicating transmission from ₁ to T3 is added is received and stored in the memory 13. At T24 time, time information indicating that it has been transmitted from the network camera C ₁ to T24 point receives the captured image data of the added high resolution stored in the memory 13. At T25 time, the time information indicating that the network camera C ₁ are transmitted to the T5 point receives the captured image data of the added low resolution stored in the memory 13. Since the network 2 does not always receive data in the order in which the data was transmitted, here, the captured image data transmitted at time T24 is received earlier than the captured image data transmitted at time T5. Yes. Here, it is assumed that the time from time T5 to time T24 is longer than the time from time T3 to time T5.

  Then, at time T25, low-resolution captured image data is newly stored, and high-resolution images at times T3 and T24, which are time information before and after the last-stored low-resolution captured image data at time T5, are stored. Since the captured image data is stored, at time T25, the abnormality detection unit 11d starts image analysis based on the last stored low-resolution captured image data. At time T26, the abnormality detection unit 11d detects an abnormality as a result of the analysis.

  Next, since an abnormality has been detected, at time T27, the image selection unit 11e receives from the high-resolution image storage unit 13b of the memory 13 the high resolution associated with the time information at time T3, which is the time closest to time T5. Then, the output control unit 11f controls the optical disk drive 14 to record the selected high-resolution captured image data at time T3 on the optical disk.

  At time T28, the output control unit 11f deletes the selected high-resolution captured image data at time T3 from the high-resolution image storage unit 13b of the memory 13 together with time information associated therewith.

  Next, at time T29, the output control means 11f reads from the high-resolution image storage means 13b of the memory 13 at the time T1 that is older than the time indicated by the time information of the selected high-resolution captured image data at time T3. The high-resolution captured image data is deleted together with the time information associated therewith.

  Further, at time T30, the output control unit 11f deletes the low-resolution captured image data at time T2, which is a time before a predetermined time, from the low-resolution image storage unit 13a of the memory 13 together with this time information.

  As described above, according to the image processing apparatus 1 that is Embodiment 1 of the present invention, even when the order in which the low-resolution captured image data and the high-resolution captured image data are transmitted and the arrival order do not match, Based on the low-resolution captured image data, it can be determined whether or not an abnormality has been detected. Since the high-resolution captured image data is stored on the optical disk only when an abnormality is detected, the abnormality is detected by image analysis without involving a process that places a heavy load on the apparatus such as image reduction. It is possible to record photographic image data of resolution.

Further, according to the image processing apparatus 1 according to a first embodiment of the present invention, based on the sum of differences, and detects an abnormality, if the network camera C ₁ and C _2, there is a change in the captured image In addition, it is possible to record high-resolution captured image data that has changed.

  The second embodiment of the present invention is applied to a surveillance camera system using a network camera, receives low-resolution captured image data and high-resolution captured image data captured by the network camera, and captures low-image captured image data. An example is an image processing device that detects an abnormality by determining whether or not a specific color is greater than or equal to a threshold value, and highlights high-resolution captured image data near the time when the abnormality is detected on the display unit. I will explain.

<Configuration of surveillance camera system>
FIG. 7 is a configuration diagram illustrating a configuration of a surveillance camera system to which an image processing apparatus that is Embodiment 2 of the present invention is applied.

As shown in FIG. 7, the surveillance camera system 101 includes an image processing device 10, network cameras C ₁ and C ₂ , a server 3, and a display device 4, each of which is connected via the network 2. .

The display device 4 is a liquid crystal display or the like, and receives and displays captured image data captured by the network cameras C ₁ and C _{2 according} to a signal from the image processing device 10. Further, in response to a request from the image processing apparatus 1, the high-resolution captured image data selected by the image selection unit 11 e of the CPU 11 of the image processing apparatus 1 is highlighted on the display unit 16.

The network cameras C ₁ and C ₂ and the server 3 shown in FIG. 7 are provided in the surveillance camera system 100 to which the image processing apparatus 1 according to the first embodiment of the present invention shown in FIG. 1 is applied. Since it has the same structure, description is abbreviate | omitted.

  The image processing apparatus 1 includes a CPU 11, a network interface 12, a memory 13, an optical disk drive 14, a hard disk 15, and a display unit 16, which are connected via a bus 20 as shown in FIG. The network interface 12, the memory 13, the optical disk drive 14, and the hard disk 15 are provided in the monitoring camera system 100 to which the image processing apparatus 1 according to the first embodiment of the present invention shown in FIG. 1 is applied. Since it is the same, description is abbreviate | omitted.

  The CPU 11 of the image processing apparatus 1 performs central control of the apparatus itself and executes an image analysis program, so that the received image determination unit 11a, the storage control unit 11b, the image determination unit 11c, An abnormality detection unit 11d, an image selection unit 11e, and an output control unit 11f are mounted.

  The abnormality detection unit 11d has the number of pixels indicating a predetermined color among the low-resolution captured image data stored last when the image determination unit 11c determines that the image is stored in the high-resolution image storage unit 13b. Is more than a predetermined threshold, it is detected as abnormal.

  The output control unit 11f highlights the high-resolution captured image data selected by the image selection unit 11e on the display unit 16.

  The received image determination unit 11a, the storage control unit 11b, the image determination unit 11c, and the image selection unit 11e are monitoring cameras to which the image processing apparatus 1 according to the first embodiment of the present invention shown in FIG. 1 is applied. Since it is the same as that provided in the system 100, a description thereof will be omitted.

The display unit 16 includes a liquid crystal display, a CRT display, and the like, and displays captured image data captured by the network cameras C ₁ and C _{2 in} accordance with instructions from the CPU 11. Further, the high-resolution photographic image data selected by the image selection unit 11e is highlighted on the display unit 16 according to an instruction from the output control unit 11f.

<Operation of Surveillance Camera System 100>
Next, the operation of the monitoring camera system 101 to which the image processing apparatus 10 according to the second embodiment of the present invention is applied will be described with reference to the drawings.

Similarly to the image processing apparatus 1 according to the first embodiment of the present invention, the image processing apparatus 10 according to the second embodiment of the present invention uses multicast packets, unicast packets, or the like from the network cameras C ₁ and C _2. Captured image data is received via the network 2.

  The image processing apparatus 10 that has received the captured image data performs image temporary storage processing and image enhancement display processing. Here, the image temporary storage process is the same as the image temporary storage process in the image processing apparatus 1 according to the first embodiment of the present invention, and thus the description thereof is omitted.

≪Image highlighting process≫
FIG. 8 is a flowchart showing a processing procedure of image enhancement display processing in the image processing apparatus 10 according to the second embodiment of the present invention.

  First, the image determination unit 11c of the CPU 11 determines whether or not low-resolution captured image data is newly stored by the storage control unit 11b (step S301).

  In step S301, when it is determined that the low-resolution captured image data is newly stored, the time information of the newly-stored low-resolution captured image data stored in the low-resolution image storage unit 13a is stored. It is determined whether or not high-resolution captured image data having time information before and after the indicated time is stored (step S302).

  Specifically, the image determination unit 11c reads time information associated with the low-resolution captured image data stored last from the low-resolution image storage unit 13a of the memory 13. Then, the image determination unit 11c reads the time information associated with the high-resolution captured image data from the high-resolution image storage unit 13b of the memory 13, and the low-resolution image stored last in the read time information. It is determined whether or not time information corresponding to the time before and after the time indicated by the time information associated with the captured image data exists. If this time information exists, the image determination unit 11c stores high-resolution captured image data having time information before and after the time indicated by the time information of the low-resolution captured image data stored last. It is determined that

  If it is determined in step S302 that the corresponding high-resolution captured image data is stored, the abnormality detection unit 11d performs image analysis of the last stored low-resolution captured image data to determine whether there is an abnormality. It is determined whether or not (step S303). Specifically, the abnormality detection unit 11d detects an abnormality when, for example, the number of pixels indicating skin color is equal to or greater than a predetermined threshold in the low-resolution captured image data stored last.

  Here, if this threshold is too high, it will not be possible to detect an abnormality even if a person is photographed, and if it is too low, an abnormality will be detected even when an object having a small skin color other than a person is photographed. End up. For this reason, it is necessary for the provider or the like to calculate in advance an appropriate value based on the actual measurement, and for the provider or the user to set an appropriate value in advance.

  Next, when an abnormality is detected by the abnormality detection unit 11d, the image selection unit 11e indicates the time information of the low-resolution captured image data stored last from the high-resolution image storage unit 13b of the memory 13. High-resolution captured image data associated with time information indicating the time closest to the time is selected (step S305).

  Next, the output control unit 11f highlights and displays the high-resolution captured image data selected by the image selection unit 11e on the display unit 16 (step S306). Specifically, the output control unit 11f displays the selected high-resolution captured image data on the display unit 16 and displays a red frame around the display area of the displayed high-resolution captured image data. Then, a red lamp picture is displayed on a part of the display area of the high-resolution captured image data.

  Note that the output control unit 11f is a display device connected to the network 2 via the network interface 12 instead of highlighting and displaying the high-resolution captured image data selected by the image selection unit 11e on the display unit 16. 4, the high-resolution captured image data selected by the image selection unit 11e may be highlighted on the display unit 16.

  Next, the output control unit 11f deletes the high-resolution captured image data selected in step S305 from the high-resolution image storage unit 13b of the memory 13 together with the time information associated therewith (step S307).

  Further, the output control unit 11f includes the high-resolution captured image data associated with the time information older than the time indicated by the time information of the high-resolution captured image data selected in step S305, together with the time information associated therewith. It deletes from the high-resolution image storage means 13b of the memory 13 (step S309).

  Next, the output control unit 11f deletes the low-resolution captured image data stored last together with the time information from the low-resolution image storage unit 13a of the memory 13 (step S310).

  On the other hand, if no abnormality is detected by the abnormality detection unit 11d in step S304, the image selection unit 11e receives the last stored low resolution from the high resolution image storage unit 13b of the memory 13 as in the process of step S305. The high-resolution captured image data associated with the time information indicating the time closest to the time indicated by the time information of the captured image data is selected, and the process proceeds to step S309 (step S308).

  In steps S303 and S304, the abnormality detection unit 11d detects an abnormality when the number of pixels indicating the skin color is equal to or greater than a predetermined threshold, but performs image analysis using higher-resolution captured image data. And the abnormality may be detected from the image analysis result.

  Specifically, the abnormality detection unit 11d outputs the high-resolution captured image data associated with the time information indicating the time closest to the time indicated by the time information of the low-resolution captured image data stored last. Read from the storage means 13b. Then, from the read high-resolution captured image data, extract a high-resolution image area corresponding to an image area including pixels indicating skin color in the low-resolution captured image data, and for the extracted high-resolution image area, It is determined whether or not it is similar to the color pattern of the human face, and if it is determined that the extracted high-resolution image area contains a human face image based on this determination result, it is detected as an abnormality. May be.

  As described above, according to the image processing apparatus 10 that is Embodiment 2 of the present invention, image analysis is performed based on low-resolution captured image data, and it is determined whether an abnormality is detected based on the image analysis result. Since the high-resolution captured image data is highlighted only when an abnormality is detected, the abnormality is detected by image analysis without involving processing that places a high load on the device such as image reduction, and the high-resolution image at the time of abnormality is detected. The captured image data can be highlighted.

Further, according to the image processing apparatus 10 that is the second embodiment of the present invention, an abnormality is detected based on the number of skin-colored pixels, so that when a person is photographed by the network cameras C ₁ and C ₂ , It is possible to highlight the high-resolution photographed image data in which is photographed.

1 is a configuration diagram illustrating a configuration of a monitoring camera system 100 to which an image processing apparatus 1 that is Embodiment 1 of the present invention is applied. FIG. It is the figure explaining transmission / reception of the picked-up image data in the surveillance camera system 100 to which the image processing apparatus 1 which is Example 1 of this invention is applied. 3 is a flowchart illustrating a processing procedure of image temporary storage processing in the image processing apparatus 1 that is Embodiment 1 of the present invention. 3 is a flowchart illustrating a processing procedure of image recording processing in the image processing apparatus 1 that is Embodiment 1 of the present invention. 5 is a time chart of image temporary storage processing and image recording processing when low-resolution captured image data and high-resolution captured image data arrive in the order of transmission in the image processing apparatus 1 that is Embodiment 1 of the present invention. . In the image processing apparatus 1 according to the first embodiment of the present invention, an image temporary storage process and an image in the case where the order in which the low-resolution captured image data and the high-resolution captured image data are transmitted and the arrival order do not match 6 is a time chart showing a recording process. It is the block diagram which showed the structure of the surveillance camera system 101 to which the image processing apparatus which is Example 2 of this invention is applied. It is the flowchart which showed the process sequence of the image emphasis display process in the image processing apparatus 10 which is Example 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,10 ... Image processing apparatus 2 ... Network 3 ... Server 4 ... Display apparatus 11 ... CPU
11a ... Received image determination means 11b ... Storage control means 11c ... Image determination means 11d ... Abnormality detection means 11e ... Image selection means 11f ... Output control means 13 ... Memory 13a ... Low resolution image storage means 13b ... High resolution image storage means 14 ... Optical disc drive (recording unit)
15 ... Hard disk (recording part)
16 ... Display unit 20 ... Bus 100, 101 ... Surveillance camera system

Claims (3)

Connected to the camera via a network, receive low-resolution captured image data and high-resolution captured image data captured by the camera, and based on the image analysis result of the received low-resolution captured image data, An image processing apparatus for recording or displaying received high-resolution captured image data,
Low-resolution captured image data received from the camera, and time information indicating the time when the low-resolution captured image data is received, or the time when the low-resolution captured image data is transmitted from the camera; Low-resolution image storage means for associating and storing
High-resolution captured image data received from the camera, and time information indicating the time when the high-resolution captured image data is received, or the time when the high-resolution captured image data is transmitted from the camera; High-resolution image storage means for storing
The high-resolution captured image data corresponding to the time information within a predetermined time before and after the time indicated by the time information of the low-resolution captured image data stored last in the low-resolution image storage unit is the high-resolution image storage unit. Image determining means for determining whether or not stored in
When the image determination unit determines that the image is stored in the high-resolution image storage unit, the image analysis is performed using the last-stored low-resolution captured image data, and based on the image analysis result. An anomaly detection means for detecting an anomaly on the image,
When an abnormality on the image is detected by the abnormality detection unit, the time closest to the time indicated by the time information corresponding to the last-stored low-resolution captured image data is stored from the high-resolution image storage unit. Image selection means for selecting the high-resolution captured image data corresponding to the time information shown;
Output control means for recording the selected high-resolution captured image data in a recording unit, or for highlighting the selected high-resolution captured image data on a display unit;
An image processing apparatus comprising: The abnormality detection means includes
When the image determination unit determines that the image is stored in the high-resolution image storage unit, the low-resolution captured image data corresponding to the oldest time information stored in the low-resolution image storage unit; If the difference from the last-stored low-resolution captured image data is greater than or equal to a predetermined threshold, it is determined that there is an abnormality on the image,
The output control means includes
The low-resolution captured image storage means deletes the low-resolution captured image data corresponding to time information before a predetermined time from the time information corresponding to the last stored low-resolution captured image data, and The image processing apparatus according to claim 1, wherein the selected high-resolution captured image data is recorded in a recording unit, or the selected high-resolution captured image data is highlighted on a display unit. The abnormality detection means includes
When the image determination unit determines that the image is stored in the high-resolution image storage unit, the number of pixels indicating a predetermined color in the last stored low-resolution captured image data is a predetermined threshold value. The image processing apparatus according to claim 1, wherein if it is above, it is determined that there is an abnormality.

Images