KR20090072109A - How to calibrate the brightness of video taken for video call - Google Patents

Abstract

The present invention relates to a method for correcting luminance of a captured image for a video call, the method comprising: detecting a face region of a user in the captured image; Receiving luminance values of a portion of the captured image corresponding to the face region; And generating a corrected image by correcting only luminance of a portion of the captured image corresponding to the face region based on the luminance values, a preset reference luminance value, and a preset correction factor value. The sharpness of the portion of the corrected image corresponding to the face area is based on the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken, compared to the sharpness of the portion corresponding to the face area of the captured image. It is increased regardless. According to the present invention, a luminance correction method of a video call photographed video includes a natural correction image including a clear face image regardless of the color of the user's clothes or the amount of light irradiated to the user from the surroundings at the time of photographing. It may be provided to the user, and the natural correction image that does not stand out the boundary of the face area can be provided to the user or the other party.

Description

Luminance correction method for photographing image for the use of the video calling}

The present invention relates to a method for correcting brightness of an image, and more particularly, to a method for correcting brightness of a captured video for video call.

In general, when a video telephone such as a VoIP over internet protocol phone or a portable telephone having a video call function communicates with another video telephone through an internet protocol (IP) network or a mobile communication network, these two images are used. The telephones transmit and receive voice and video data of users to and from each other through an IP network or a mobile communication network. Therefore, the user can make a voice and video call with the call counterpart via the video telephone. On the other hand, since the main purpose of the video call is to allow the user to check his or her face and the face of the call counterpart, it is very important that the video phone displays the user's face clearly. However, according to various environmental factors such as the brightness of the shooting location or the color of the clothes of the user, the image of the face part of the user captured by the photographing device may be distorted. That is, the luminance of a specific portion may be distorted in the image photographed by the photographing apparatus. The reason is that the automatic exposure operation of a photographing device such as a camera is greatly affected by the brightness of the surrounding environment. Here, the automatic exposure operation of the photographing apparatus is an operation in which the photographing apparatus automatically recognizes the entire brightness of the photographing area, and automatically adjusts the degree of exposure by appropriately operating the aperture according to the recognition result. For example, as shown in FIG. 1A, when the user wears black clothes, the average value of the overall luminance of the photographing area decreases due to the influence of the black clothes. As a result, in order to compensate for the average value of the brightness, the photographing apparatus photographs the corresponding area in a state where the degree of exposure is increased. In the photographed image, the image corresponding to the black clothes is clearly visible due to the increased brightness, but the part of the face that is relatively brighter than the black clothes is excessively increased, so that the brightness is excessively bright, as shown in FIG. 1A. It can be seen that. On the contrary, as shown in FIG. 1B, when the user wears white clothes, the average value of the overall luminance of the photographing area increases under the influence of the white clothes. As a result, the photographing apparatus photographs the region in a state in which the degree of exposure is reduced in order to compensate for the luminance average value. In the photographed image, the image corresponding to the white clothing decreases its brightness and looks clear, but the darker face portion of the face that is relatively darker than the white clothing decreases the brightness excessively, and thus, the image corresponding to the white clothing looks too dark. Can be. On the other hand, the phenomenon in which the photographed face image appears too bright or too dark may be caused not only by the color of the clothes of the user, but also by the ambient light irradiated to the user. For example, in FIG. 1A, when the amount of irradiation of upper light (eg, room light or sunlight), or left or right light (eg, light incident through the room light or a window) decreases, The average value of the overall luminance of the shooting area is further reduced by the influence of ambient light as well as black clothes. As a result, the phenomenon in which the face portion of the captured image appears too bright as compared to the actual face of the user appears more seriously. In addition, in FIG. 1B, when the irradiation amount of the upper light or the left or right light is increased, the average value of the overall luminance of the image to be photographed is further increased by the influence of the ambient light as well as the white clothes. As a result, the phenomenon that the face part of the captured image appears too dark appears more seriously.

On the other hand, as a technology for differently adjusting the brightness of the face region and the rest of the region except the face during a video call, a power-saving video communication terminal and its operation method is disclosed in Korean Patent Laid-Open No. 2003-0094746 have. However, according to the technique disclosed in Korean Patent Laid-Open Publication No. 2003-0094746, in order to save power, the brightness control unit lowers the brightness of the remaining area except the face in the captured image as compared to the face area, or the background switching unit performs the face in the captured image. The remaining area except for is replaced with a preset dark background image. Therefore, not only the image of the face region but also the image of the remaining region except for the face must be corrected, thereby increasing the throughput of the image data and its processing time. In addition, even when the face is taken too dark as well as when the image is taken too bright, the brightness control adjusts the brightness of the image of the face area relatively brighter than the image of the rest area except the face, It is difficult to provide a clear face image. In addition, due to the sharp difference between the brightness of the image of the remaining areas except for the face and the brightness of the face area, the entire image looks unnatural, thereby causing a visual burden on the user.

Accordingly, a technical problem to be solved by the present invention is to correct the luminance of a face region based on a preset reference luminance value, a luminance value of the face region, and a preset correction factor value in a captured image, thereby providing a color of a user's clothes. In another aspect, the present invention provides a method for correcting luminance of a captured image for a video call, which may provide the user or a call counterpart with a natural corrected image including a clear face image irrespective of the amount of light irradiated to the user from the surroundings. have.

According to an aspect of the present invention, there is provided a luminance correction method of a captured image for a video call, comprising: detecting a face region of a user from the captured image; Receiving luminance values of a portion of the captured image corresponding to the face region; And generating a corrected image by correcting only luminance of a portion of the captured image corresponding to the face region based on the luminance values, a preset reference luminance value, and a preset correction factor value. The sharpness of the portion of the corrected image corresponding to the face area is based on the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken, compared to the sharpness of the portion corresponding to the face area of the captured image. It is increased regardless.

According to another aspect of the present invention, there is provided a method of compensating luminance of a captured image for a video call, comprising: detecting a face region and at least one peripheral region surrounding an outside of the face region in the captured image; ; Receiving first luminance values of a portion of the captured image corresponding to the face region and second luminance values of the portion corresponding to the at least one peripheral region; Correcting luminance of a portion of the captured image corresponding to the face region based on the first luminance values, a preset reference luminance value, and a preset first correction factor value; And generating a corrected image by correcting the luminance of a portion of the captured image corresponding to the at least one peripheral region based on the second luminance values, the reference luminance value, and a preset second correction factor value. It includes. The face area and the at least one peripheral area correspond to a part of the captured image. The sharpness of the portion of the corrected image corresponding to the face area is based on the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken, compared to the sharpness of the portion corresponding to the face area of the captured image. It is increased regardless.

As described above, the brightness correction method of the captured video for video call according to the present invention, in the captured image, the brightness of the face area to the preset reference brightness value, the brightness value of the face area, and the preset correction factor value Since the correction is based on the user, a natural correction image including a clear face image may be provided to the user or the call counterpart regardless of the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken. In addition, the brightness correction method of the video call recording image according to the present invention is corrected so that the brightness of the face region and the peripheral region surrounding the outer edge of the face region in the captured image to be changed step by step, so that the boundary of the face region is not noticeable The natural correction image may be provided to the user or the call counterpart.

Hereinafter, with reference to the accompanying drawings will be described a preferred embodiment of the present invention. However, the present invention is not limited to the embodiments disclosed below, but can be implemented in various forms, and only the present embodiments are intended to complete the disclosure of the present invention and to those skilled in the art. It is provided for complete information.

2 is a schematic block diagram of a video telephone to which a brightness correction method of a captured video for a video call according to the present invention is applied. The video telephone 100 includes a camera 101, a camera interface 102, a central processing unit 103, a memory 104, an area detection tracking unit 105, a luminance correction unit 106, Microphone 107, microphone interface 108, media processor 109, communication unit 110, display interface 111, display device 112, audio interface 113, speaker 114, sub-bus bridge (sub bus bridge 115, a user register set (URS) 116, and an input unit 117. Camera interface 102, CPU 103, memory 104, area detection tracking unit 105, luminance correction unit 106, microphone interface 108, media processor 109, communication unit 110, display interface 111, the audio interface 113, and the sub bus bridge 115 are each connected to a main bus 118. In addition, the sub bus bridge 115, the URS 116, and the input unit 117 are connected to the sub bus 119, respectively.

The camera 101 photographs the user's state and outputs image data IDAT1 to IDATK (K is an integer) representing the captured image. The camera interface 102 receives the image data IDAT1 to IDATK from the camera 101 under the control of the CPU 103 and outputs it to the main bus 118. Here, the image data IDAT1 to IDATK are luminance values FY1 to FYM, PY1 to PYN, BY1 to BYQ (M, N, and Q are integers) of pixels constituting the captured image 200 (FIG. 8). Each). The luminance values FY1 to FYM represent luminances of a portion corresponding to the face area FA of the captured image 200, and the luminance values PY1 to PYN represent peripheral regions PA1 to PAJ of the captured image 200. (J is an integer). In addition, the luminance values BY1 to BYQ (not shown) indicate the luminance of the background area BA in the captured image 200 except for the face area FA and the peripheral areas PA1 to PAJ.

The CPU 103 stores the image data IDAT1 to IDATK on the main bus 118 in the memory 104. The CPU 103 includes a camera interface 102, a memory 104, an area detection tracking unit 105, a luminance correction unit 106, a microphone interface 108, a media processor 109, a communication unit 110, a display interface. 111 controls the operation of the audio interface 113 and the sub-bus bridge 115, respectively. In addition, the CPU 103 includes a camera interface 102, a memory 104, an area detection tracking unit 105, a luminance correction unit 106, a microphone interface 108, a media processor 109, a communication unit 110, Priority is determined according to a request for using the main bus 118 of the display interface 111, the audio interface 113, and the sub-bus bridge 115. The CPU 103 determines the camera interface 102, the memory 104, the area detection tracking unit 105, the luminance correction unit 106, the microphone interface 108, and the media processor 109 according to the result of the determination of the priority. ), At least one of the communication unit 110, the display interface 111, the audio interface 113, and the sub-bus bridge 115 is given a right to use the main bus 118. Optionally, the CPU 103 may include a program memory (not shown) in which an area detection tracking program corresponding to the operation of the area detection tracking unit 105 and a brightness correction program corresponding to the operation of the luminance correction unit 106 are stored. It may include. In this case, the video telephone 100 does not include the area detection tracking unit 105 and the brightness correction unit 106.

The area detection tracking unit 105 receives the image data IDAT1 to IDATK loaded on the main bus 118 under the control of the CPU 103, and based on the image data IDAT1 to IDATK, captures a captured image. In operation 200, the facial area FA is determined. Thereafter, the area detection tracking unit 105 tracks the movement of the face area FA, and outputs the face area information FAIF to the main bus 118 according to the tracking result. Further, the area detection tracking unit 105 further determines at least one peripheral area (at least one of PA1 to PAJ) surrounding the outside of the face area FA under the control of the CPU 103, and determines the peripheral area ( The movement of at least one of PA1 to PAJ can be tracked. In this case, the area detection tracking unit 105 may determine, as the peripheral area (at least one of PA1 to PAJ), an area corresponding to the number of pixels set in the outer direction from the boundary line of the face area FA. In this case, the area detection tracking unit 105 outputs peripheral area information (at least one of PAIF1 to PAIFJ) to the main bus 118. On the other hand, when the area detection tracking unit 105 detects and tracks the plurality of peripheral areas PA2 to PAJ, the area detection tracking unit 105 outputs the plurality of peripheral area information PAIF2 to PAIFJ on the main bus 118. Here, since the area detection and tracking operation by the area detection tracking unit 105 can be well understood by those skilled in the art, a detailed description thereof will be omitted.

The CPU 103 selects an image corresponding to the face area FA from the captured image 200 based on the face area information FAIF, and stores the luminance values FY1 to FYM of the selected image in the memory 104. Read from The luminance corrector 106 receives luminance values FY1 to FYM from the memory 104 under the control of the CPU 103. The luminance corrector 106 compares the luminance values FY1 to FYM with the preset reference luminance value YR, respectively. When there is at least one luminance value (for example, FY1) that matches the reference luminance value YR as a result of the comparison, the luminance correction unit 106 outputs the luminance value FY1 as the corrected luminance value FY1 'as it is. do. As a result, the luminance value FY1 stored in the memory 104 is maintained as it is.

In addition, as a result of the comparison, the luminance correction unit 106 determines luminance difference values FYD2 to FYDM between luminance values (for example, FY2 to FYM) that do not match the reference luminance value YR, and the reference luminance value YR. Calculate each). Thereafter, the luminance correction unit 106 is based on luminance values FY2 to FYM that do not coincide with the reference luminance value YR, luminance difference values FYD2 to FYDM, and a preset correction factor value α0. The corrected luminance values FY2 'to FYM' are calculated and output to the main bus 118. As a result, the CPU 103 substitutes and stores the luminance values FY2 to FYM stored in the memory 104 with correction luminance values FY2 'to FYM'. At this time, the CPU 103 deletes the luminance values FY2 to FYM stored in the memory 104 and then stores the corrected luminance values FY2 'to FYM' in the address area where the luminance values FY2 to FYM are stored. Can be.

As the luminance values FY2 to FYM are replaced with the correction luminance values FY2 'to FYM', the luminance of the portion of the captured image 200 corresponding to the face area FA is corrected, thereby correcting the image (eg, , 300b) (see FIG. 9B). Here, since the equations for calculating the correction luminance values FY1 'to FYM' are similar to each other, for the sake of simplicity, the following description will focus on the equations for calculating the correction luminance values FY1 '. The equation for calculating the correction luminance value FY1 'can be expressed as follows.

In Equation 1, for example, when FY2 is substituted for FY1, Equation 1 is changed to an equation for calculating the correction luminance value FY2 '. From Equation 1, the luminance difference between the corrected image and the captured image of the face area FA is determined by the relationship between the average value of the luminance values FY1 to FYM of the face area FA and the reference luminance value YR. It can be seen that it changes accordingly. For example, when the average value of the luminance values FY1 to FYM is larger than the reference luminance value YR, the luminance of the portion of the corrected image corresponding to the face area is higher than the luminance of the portion of the captured image corresponding to the face area. low. In addition, when the average value of the luminance values FY1 to FYM is smaller than the reference luminance value YR, the luminance of the portion of the corrected image corresponding to the face area is higher than the luminance of the portion of the captured image corresponding to the face area.

On the other hand, when the area detection tracking unit 105 outputs the peripheral area information PAIF1 to PAIFJ to the main bus 118, the CPU 103 uses the captured image based on the peripheral area information PAIF1 to PAIFJ. Among the 200, images corresponding to the peripheral areas PA1 to PAJ are selected, and luminance values PY1 to PYN of the selected images are read from the memory 104. The luminance corrector 106 receives luminance values PY1 to PYN from the memory 104 under the control of the CPU 103. The luminance corrector 106 compares the luminance values PY1 to PYN with the reference luminance value YR, respectively. When there is at least one luminance value (for example, PY1) that matches the reference luminance value YR as a result of the comparison, the luminance corrector 106 outputs the luminance value PY1 as the corrected luminance value PY1 'as it is. do. As a result, the luminance value PY1 stored in the memory 104 is maintained as it is.

In addition, as a result of the comparison, the luminance correction unit 106 determines luminance difference values PYD2 to PYDN between luminance values (for example, PY2 to PYN) that do not match the reference luminance value YR, and the reference luminance value YR. Calculate each). Thereafter, the luminance correction unit 106 is based on luminance values PY2 to PYN that do not coincide with the reference luminance value YR, luminance difference values PYD2 to PYDN, and preset correction factor values α1 to αJ. The correction luminance values PY2 'to PYN' are respectively calculated. Here, the equations for calculating the correction luminance values PY1 'to PYN' are similar to each other. For example, one of the luminance values corresponding to the peripheral area PA1 of the captured image 200 is PY2, one of the luminance values corresponding to the peripheral area PA2 is PY20, and corresponds to the peripheral area PAJ. Assuming that one of the luminance values is PYN, the equation for calculating the corrected luminance values PY2 ', PY20', and PYN 'can be expressed as follows.

As can be seen from Equation 2, correction factor values α1 to αJ are used to calculate correction luminance values corresponding to the peripheral areas PA1 to PAJ, respectively. For example, assuming that luminance values of images of the peripheral area PA1 and the peripheral area PA2 are 20, respectively, to calculate luminance correction values for the 20 luminance values for the peripheral area PA1, α1 is used. Further, α2 is used to calculate luminance correction values for the 20 luminance values of the peripheral area PA2.

The CPU 103 replaces and stores the luminance values PY2 to PYN stored in the memory 104 with the corrected luminance values PY2 'to PYN' calculated by the luminance correction unit 106, respectively. As the luminance values PY2 to PYN stored in the memory 104 are replaced with the correction luminance values PY2 'to PYN', respectively, the luminance of the portion corresponding to the peripheral areas PA1 to PAJ in the captured image 200. Are respectively corrected to generate a corrected image (not shown). In this case, the luminance of the corrected image gradually decreases or gradually increases from the face area FA to the peripheral area PAJ. For example, as in the photographed image 200a illustrated in FIG. 9A, when the face region is photographed too brightly, the luminance of the corrected image 300a is gradually increased from the face region FA to the peripheral region PAJ. do. As shown in the photographed image 200b of FIG. 9B, when the face region is photographed too dark, the luminance of the corrected image 300b gradually decreases from the face region FA to the peripheral region PAJ.

Meanwhile, the reference luminance value YR and the correction factor values α0 to αJ may be preset in the luminance corrector 106. The correction factor value α 0 may be set to any one of the prime numbers satisfying X, or 1, at 0 ≦ X ≦ 1. Each of the correction factor values α1 to αJ may be set to any one of the prime numbers satisfying X at 0 ≦ X ≦ 1. The correction factor value α 0 is larger than each of the correction factor values α 1 to α J. Further, among the correction factor values alpha 1 to alpha J, the correction factor value alpha 1 is the maximum value, and the correction factor value alpha J is the minimum value. That is, the correction factor value gradually decreases from α1 to αJ.

In order to set the reference luminance value YR and the correction factor values α0 to αJ in the luminance correction unit 106, the user operates the input unit 117 to register and store the setting information SETF in the URS 116. Can be. As a result, the CPU 103 receives the setting information SETF from the URS 116 via the sub-bus bridge 115, and based on the setting information SETF, the reference luminance value is given to the luminance correcting unit 106. (YR) and correction factor values α0 to αJ are set. The user may manipulate the input unit 117 to change the reference luminance value YR and the correction factor values α0 to αJ. Also, although not shown in detail in FIG. 2, the user may operate the input unit 117 to select a call connection or disconnection between his video phone 100 and the video phone (not shown) of the call counterpart.

The microphone 107 converts a user's voice into audio data AUD and outputs the audio. The microphone interface 108 outputs audio data AUD received from the microphone 107 to the main bus 118 under the control of the CPU 103. The media processor 109 controls the image data CDAT1 to CDATK (K is an integer) representing the corrected image received from the memory 104 and the audio received from the microphone interface 108 under the control of the CPU 103. The data AUD is encoded, and the encoded image data EDAT1 to EDATK and the encoded audio data EAUD are output. Here, the CDAT1 to CDATK are used only for distinguishing from the image data IDAT1 to IDATK representing the photographed image. That is, some of the image data CDAT1 to CDATK representing the corrected image (that is, image data whose luminance value is not corrected) are actually the same as some of the image data IDAT1 to IDATK.

The communication unit 110 controls a data packet DPK suitable for a preset communication protocol based on the encoded image data EDAT1 to EDATK and the encoded audio data EAUD under the control of the CPU 103. And a data packet (DPK) is transmitted to the video telephone of the called party. The display interface 111 outputs the image data CDAT1 to CDATK received from the memory 104 to the display device 112 under the control of the CPU 103. The display apparatus 112 displays the corrected image based on the image data CDAT1 to CDATK. Here, the sharpness of the portion of the corrected image corresponding to the face area is increased regardless of the color of the user's clothes or the amount of light irradiated to the user from the surroundings at the time of shooting, compared to the sharpness of the portion corresponding to the face area of the captured image. do. The audio interface 113 outputs the voice of the other party to the speaker 114 based on the decoded audio data DAUD under the control of the CPU 103. Here, the decoded audio data DAUD is obtained by decoding the encoded audio data (not shown) from the video telephone of the call counterpart received by the communication unit 110 by the media processor 109. The sub bus bridge 115 outputs the setting information SETF from the URS 116 received through the sub bus 119 to the main bus 118 under the control of the CPU 103. In addition, the sub-bus bridge 115 receives signals (not shown) from the input unit 117 or the URS 116 received through the sub-bus 119 under the control of the CPU 103. Output to. The sub bus bridge 115 may output signals (not shown) received through the main bus 118 to the sub bus 119 under the control of the CPU 103.

Next, a detailed operation process of the video telephone 100 will be described with reference to FIGS. 3 to 7. For convenience of description, descriptions of parts related to audio data AUD or decoded audio data DAUD of the video telephone 100 will be omitted.

3 is a flowchart illustrating a luminance correction process 1000 of a captured image for a video call according to an embodiment of the present invention. First, the video phone 100 receives image data IDAT1 to IDATK corresponding to the captured image from the camera 101 (step 1100). At this time, the CPU 103 may store the image data IDAT1 to IDATK in the memory 104. Meanwhile, the area detection tracking unit 105 receives the image data IDAT1 to IDATK loaded on the main bus 118, and based on the image data IDAT1 to IDATK, the area detection tracking unit 105 receives a face region ( FA) (step 1200). At this time, the area detection tracking unit 105 tracks the movement of the face area FA and outputs face area information FAIF. The CPU 103 selects an image corresponding to the face area FA from the captured image 200 based on the face area information FAIF, and stores the luminance values FY1 to FYM of the selected image in the memory 104. Read from As a result, the brightness correction unit 106 receives the brightness values FY1 to FYM from the memory 104 under the control of the CPU 103 (step 1300). Thereafter, the luminance correction unit 106 outputs correction luminance values FY1 'to FYM' based on the luminance values FY1 to FYM, the preset reference luminance value YR, and the correction factor value α0. do. The CPU 103 replaces and stores the luminance values FY1 to FYM stored in the memory 104 with correction luminance values FY1 'to FYM', respectively. As a result, a corrected image in which the luminance value of the face area FA is corrected is generated (step 1400). On the other hand, the CPU 103 reads the image data CDAT1 to CDATK representing the corrected image from the memory 104 in order to transmit the corrected image to the video telephone of the call counterpart. The media processor 109 receives and encodes the image data CDAT1 to CDATK under the control of the CPU 103 (step 1500). Thereafter, the communication unit 110 transmits the encoded image data EDAT1 to EDATK to the video telephone of the call counterpart under the control of the CPU 103 (step 1600). More specifically, the communication unit 110 generates a data packet DPK based on the encoded image data EDAT1 to EDATK and the encoded audio data EAUD, and converts the data packet DPK to the call counterpart. Send to video phone. In addition, the display interface 111 outputs the image data CDAT1 to CDATK to the display apparatus 112 under the control of the CPU 103. As a result, the display apparatus 112 displays the corrected image in which the face area FA is corrected in luminance based on the image data CDAT1 to CDATK (step 1700).

Referring to FIG. 4, the operation of the video telephone 100 of step 1400 of generating the corrected image will be described in more detail as follows.

First, the luminance correction unit 106 compares one of the luminance values FY1 to FYM (for example, FY1) with a reference luminance value YR (step 1401), so that the luminance value FY1 is a reference luminance value. It is determined whether it is different from YR (step 1402). In operation 1402, when the corresponding luminance value FY1 is different from the reference luminance value YR, the luminance corrector 106 calculates a luminance difference value FYD1 between the corresponding luminance value FY1 and the reference luminance value YR. (Step 1403). Thereafter, the luminance correction unit 106 substitutes the luminance value FY1, the luminance difference value FYD1, and the correction factor value α0 into [Equation 1] to calculate the correction luminance value FY1 '. (Step 1404). The CPU 103 substitutes and stores the luminance value FY1 stored in the memory 104 with the correction luminance value FY1 'outputted from the luminance correction unit 106 (step 1405). On the other hand, in step 1402, when the luminance value FY1 is equal to the reference luminance value YR, the luminance correction unit 106 outputs the luminance value FY1 as the correction luminance value FY1 'as it is. As a result, the luminance value FY1 stored in the memory 104 is maintained as it is (step 1406). Here, although the operation of the video telephone 100 has been described in the order of steps 1405 and 1406, the operations of the video telephone 100 of steps 1405 and 1406 can be executed simultaneously. That is, when the luminance value is equal to the reference luminance value YR, the luminance correction unit 106 outputs the luminance value as a correction luminance value as it is, and when the luminance value is different from the reference luminance value YR, The calculated luminance value is calculated and output using Equation 1. The brightness correction unit 106 repeatedly outputs the brightness value as the corrected brightness value or calculates the corrected brightness value from the brightness value FY1 to the brightness value FYM. Thereafter, when the luminance correcting unit 106 outputs the corrected luminance values FY1 'to FYM', the CPU 103 adjusts the luminance values FY1 to FYM stored in the memory 104 at the corrected luminance values FY1 'to FYM ') and save each. As a result, the luminance value output as it is without being calculated by the luminance correction unit 106 is maintained in the memory 104 as it is.

After step 1405 or 1406, the brightness compensator 106 determines whether brightness compensation of the captured image 200 (that is, the face region portion of the captured image 200) is completed (step 1407). In this case, the luminance corrector 106 may determine whether the luminance correction is completed according to whether or not the correction luminance values FY1 ′ to FYM ′ for the luminance values FY1 to FYM are all calculated. In operation 1407, when the luminance correction of the captured image 200 is not completed, the video telephone 100 repeats the operations of operations 1401 to 1407. Referring to FIG. 9A, the brightness of the face portion of the photographed image 200a photographed too brightly is corrected by the above-described brightness correction process 1000, so that the brightness of the face portion of the corrected image 300a is slightly reduced, thereby It is confirmed that the area becomes clearer. Referring to FIG. 9B, the luminance of the face portion of the photographed image 200a photographed too dark is corrected by the above-described luminance correction process 1000, so that the luminance of the face portion of the corrected image 300b is slightly increased, thereby It is confirmed that the area becomes clearer.

The above-described brightness correction process 1000 is an operation process of the video phone 100 when only the brightness of the face area FA of the captured image 200 is corrected. However, when only the luminance of the face area FA is corrected in the captured image 200, due to the sudden difference in luminance between the image of the face area FA and the image of the background part, the boundary of the face area FA of the corrected image is changed. As it becomes noticeable, the correction image may look unnatural overall. Accordingly, in order to reduce the phenomenon in which the boundary of the face area FA is prominently displayed in the corrected image, the video phone 100 not only faces the face area FA of the captured image 200 but also the outside of the face area FA. Luminance of the surrounding peripheral areas PA1 to PAJ can be further corrected. In this case, in the corrected image, the brightness is gradually gradientd from the face area FA to the peripheral area PAJ, so that a natural corrected image is obtained in which the boundary of the face area FA is not prominent. The operation of the video phone 100 related to this will be described with reference to FIGS. 5 to 7 as follows.

5 is a flowchart illustrating a luminance correction process 2000 of a captured image for a video call according to another exemplary embodiment of the present invention. The luminance correction process 2000 is similar to the luminance correction process 1000 described above, with a few differences. Therefore, in the present embodiment, for the sake of simplicity, the following description will focus on differences between the luminance correction processes 2000 and 1000. The difference between the luminance correction processes 2000 and 1000 is that the operation of the videophone 100 in steps 2200 to 2500 is slightly different from the operation of the videophone 100 in steps 1200 to 1400.

First, in operation 2200, the area detection tracking unit 105 further detects not only the facial area FA but also the peripheral areas PA1 to PAJ in the captured image 200 based on the image data IDAT1 to IDATK. It is. For convenience of explanation, the case where the area detection tracking unit 105 detects the peripheral area PA1 will be described by way of example. In this case, the detection tracking unit 105 outputs the face area information FAIF and the peripheral area information PAIF1. In this case, the area detection tracking unit 105 detects, as the peripheral area PA1, an area corresponding to the number of pixels set in the outer direction from the boundary line of the face area FA.

The CPU 103 selects an image corresponding to the face area FA and the surrounding area PA1 from the captured image 200 based on the face area information FAIF and the surrounding area information PAIF1, and selects the selected image. Luminance values FY1 to FYM (i.e., first luminance values) and luminance values (e.g., PY1 to PY20) (i.e., second luminance values) are read from the memory 104. Here, the luminance values FY1 to FYM represent the luminance of the face area FA, and the luminance values PY1 to PY20 represent the luminance of the peripheral area PA1.

In operation 2300, the luminance corrector 106 receives luminance values FY1 to FYM and PY1 to PY20 from the memory 104 under the control of the CPU 103. Thereafter, in step 2400, the luminance corrector 106 adjusts the luminance values FY1 ′ to FYM based on the luminance values FY1 to FYM, the preset reference luminance value YR, and the correction factor value α0. Output ') In operation 2400, the CPU 103 replaces and stores the luminance values FY1 to FYM stored in the memory 104 with the corrected luminance values FY1 ′ to FYM ′, respectively. As a result, the luminance value of the face area FA is corrected. Then, in step 2500, the luminance correction unit 106 adjusts the luminance values PY1 ′ to PY20 based on the luminance values PY1 to PY20, the reference luminance value YR, and the correction factor value α1. Output ') In step 2500, the CPU 103 replaces and stores the luminance values PY1 to PY20 stored in the memory 104 with the corrected luminance values PY1 ′ to PY20 ′, respectively. As a result, a corrected image in which the luminance values of the face area FA and the peripheral area PA1 are corrected is generated.

In step 2200, the area detection tracking unit 105 detects the face area FA and the peripheral area PA1 as an example. However, the area detection tracking unit 105 may further include additional peripheral areas (eg, PA2, PA3) can be further detected. In this case, the area detection tracking unit 105 detects, as the peripheral area PA2, an area corresponding to the number of pixels set in the outer direction from the boundary line of the peripheral area PA1, and is set in the outer direction from the boundary line of the peripheral area PA2. An area equal to the number of pixels is detected as the peripheral area PA3. If the peripheral areas PA2 and PA3 are further detected, in step 2500, the luminance corrector 106 determines the luminance values (eg, PY1 to PY (N / 3)), the reference luminance value YR, And correction luminance values PY1 'to PY (N / 3)' based on the correction factor value alpha1. In addition, the luminance correction unit 106 corrects the luminance based on the luminance values PY (N / 3 + 1) to PY (2N / 3), the reference luminance value YR, and the correction factor value α2. The values PY (N / 3 +1) 'to PY (2N / 3)' are output. In addition, the luminance correction unit 106 is based on the luminance values PY (2N / 3 + 1) to PYN, the reference luminance value YR, and the correction factor value α3, and the luminance correction values PY ( 2N / 3 +1) 'to PYN').

FIG. 6 is a detailed flowchart of step 2400 shown in FIG. 5. Since the operation of the video telephone 100 of step 2400 is similar to that of the video telephone 100 of step 1400 described above with reference to FIG. 4, for simplicity of explanation, the operation description of the video telephone 100 of step 2400 will be described. It is omitted. FIG. 7 is a detailed flowchart of step 2500 shown in FIG. 5. Operation of the video phone 100 of step 2500 is similar to that of the video phone 100 of step 1400 described above with reference to FIG. 4 except for one difference. Therefore, for the sake of simplicity, the description of the operation of the video telephone 100 of step 2500 except for the difference is omitted. The difference is that, in step 2504, the luminance corrector 106 of the video telephone 100 uses [Equation 2] to calculate the corrected luminance value (for example, PY1 '). That is, the luminance correction unit 106 substitutes the luminance value (for example, PY1), the luminance difference value PYD1, and the correction factor value α1 into [Equation 2], thereby correcting the luminance value PY1 '. Calculate

The above embodiments are for explaining the present invention, and the present invention is not limited to these embodiments, and various embodiments are possible within the scope of the present invention. In addition, although not described, equivalent means will also be referred to as incorporated in the present invention. Therefore, the true scope of the present invention will be defined by the claims below.

1A and 1B are diagrams illustrating a conventional captured image for a video call.

2 is a schematic block diagram of a video telephone to which a brightness correction method of a captured video for a video call according to the present invention is applied.

3 is a flowchart illustrating a luminance correction process of a captured image for a video call according to an embodiment of the present invention.

4 is a detailed flowchart of a step of generating a corrected image illustrated in FIG. 3.

5 is a flowchart illustrating a luminance correction process of a captured image for a video call according to another embodiment of the present invention.

FIG. 6 is a detailed flowchart of step 2400 shown in FIG. 5.

FIG. 7 is a detailed flowchart of step 2500 shown in FIG. 5.

FIG. 8 is a diagram illustrating a captured image for explaining a detection step illustrated in FIGS. 3 and 5.

9A and 9B are diagrams illustrating corrected images and captured images corrected by a brightness correction method of a captured image for a video call according to the present invention.

<Explanation of symbols for main parts of drawing>

100: video phone 101: camera

102: camera interface 103: CPU

104: memory 105: area detection tracking unit

106: luminance correction unit 107: microphone

108: microphone interface 109: media processor

110: communication unit 111: display interface

112: display device 113: audio interface

114: Speaker 115: Subbus Bridge

116: URS 117: input unit

118: main bus 119: sub bus

Claims (18)

Detecting a face area of the user from the captured image; Receiving luminance values of a portion of the captured image corresponding to the face region; And Generating a corrected image by correcting only luminance of a portion of the captured image corresponding to the face region based on the luminance values, a preset reference luminance value, and a preset correction factor value; The sharpness of the portion of the corrected image corresponding to the face area is based on the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken, compared to the sharpness of the portion corresponding to the face area of the captured image. How to calibrate the brightness of the recorded video regardless of increased video call. The method of claim 1, When the average value of the luminance values is larger than the reference luminance value, the luminance of the portion of the corrected image corresponding to the face area is lower than the luminance of the portion of the captured image corresponding to the face area, When the average value of the luminance values is smaller than the reference luminance value, the luminance of the portion of the corrected image corresponding to the face area is higher than the luminance of the portion of the captured image corresponding to the face area. Brightness correction method. The method of claim 1, Encoding image data representing the corrected image; And And transmitting the encoded video data to a video telephone of a call counterpart. The method of claim 1, And displaying the corrected image on the video telephone of the user. The method of claim 1, And storing image data representing the captured image or the corrected image. The method of claim 1, wherein the detecting of the face area comprises: Determining the face area in the captured image; Tracking the movement of the face area; And And outputting face region information according to the tracking result. The method of claim 6, wherein the receiving step, Selecting an image corresponding to the face region from the photographed image based on the face region information; And And receiving the luminance values of the selected image. The method of claim 1, The value of the correction factor, at 0≤X≤1, any one of the decimal number that satisfies X, or 1, the brightness correction method of the video call image taken. The method of claim 1, wherein the generating of the corrected image comprises: Comparing the luminance values with the reference luminance values, respectively; Maintaining the at least one luminance value when there is at least one luminance value matching the reference luminance value among the luminance values as a result of the comparison: As a result of the comparison, calculating luminance difference values between luminance values not matching the reference luminance value and the reference luminance value among the luminance values; Calculating correction luminance values based on luminance values that do not match the reference luminance value, the luminance difference values, and the correction factor value; And And substituting luminance values that do not match the reference luminance values with the corrected luminance values, respectively. Detecting a face region and at least one peripheral region surrounding the outside of the face region in the captured image; Receiving first luminance values of a portion of the captured image corresponding to the face region and second luminance values of the portion corresponding to the at least one peripheral region; Correcting luminance of a portion of the captured image corresponding to the face region based on the first luminance values, a preset reference luminance value, and a preset first correction factor value; And Generating a corrected image by correcting a luminance of a portion of the photographed image corresponding to the at least one peripheral region based on the second luminance values, the reference luminance value, and a preset second correction factor value; Including, The face area and the at least one peripheral area correspond to a part of the captured image, The sharpness of the portion of the corrected image corresponding to the face area is based on the color of the clothes of the user or the amount of light irradiated to the user from the surroundings when the image is taken, compared to the sharpness of the portion corresponding to the face area of the captured image. How to calibrate the brightness of the recorded video regardless of increased video call. The method of claim 10, And the first correction factor value is greater than the second correction factor value. The method of claim 10, In the detecting step, when a plurality of peripheral regions are detected, in the generating of the corrected image, in order to correct luminance of portions of the photographed image corresponding to the plurality of peripheral regions, respectively, the plurality of peripheral regions The luminance correction method of the video call recording image, each of which uses a plurality of preset correction factor values corresponding to. The method of claim 12, The plurality of correction factor values are different from each other, Of the plurality of peripheral areas, one of the plurality of correction factor values, which are preset corresponding to the peripheral area closest to the face area, is a maximum correction factor value and corresponds in advance to the peripheral area farthest from the face area. Another one of the plurality of correction factor values that is set is a minimum correction factor value, The luminance of the corrected image is gradually reduced or gradually increased from the face region to the peripheral region farthest away from the face region. The method of claim 10, When the average value of the first luminance values is greater than the reference luminance value, the luminance of the portion of the corrected image corresponding to the face area is lower than the luminance of the portion of the captured image corresponding to the face area, When the average value of one luminance value is smaller than the reference luminance value, the luminance of the portion of the corrected image corresponding to the face area is higher than the luminance of the portion of the captured image corresponding to the face area, The luminance of the portion of the corrected image corresponding to the face region is higher or lower than the luminance of the portion corresponding to the at least one peripheral region of the corrected image. The method of claim 10, Encoding image data representing the corrected image; And And transmitting the encoded video data to a video telephone of a call counterpart. The method of claim 10, And displaying the corrected image on the video telephone of the user. The method of claim 10, The first correction factor value is any one of decimals satisfying X, or 1, at 0 ≦ X ≦ 1, And the second correction factor value is any one of prime numbers satisfying X, where 0 ≦ X ≦ 1. The method of claim 10, Correcting the luminance of the portion of the captured image corresponding to the face area, A first comparing step of comparing the first luminance values with the reference luminance values, respectively; Maintaining the at least one first luminance value when there is at least one first luminance value that matches the reference luminance value among the first luminance values as a result of the first comparison: Calculating first luminance difference values between the first luminance values and the reference luminance values that do not coincide with the reference luminance values among the first luminance values as a result of the first comparison; Calculating first correction luminance values based on first luminance values, the first luminance difference values, and the first correction factor value that do not coincide with the reference luminance value; And Replacing first luminance values that do not match the reference luminance value with the first corrected luminance values,  Generating the corrected image, A second comparing step of comparing the second luminance values with the reference luminance value, respectively; Maintaining the at least one second luminance value when there is at least one second luminance value that matches the reference luminance value among the second luminance values as a result of the second comparison; Calculating second luminance difference values between the reference luminance values and second luminance values that do not match the reference luminance values, among the second luminance values as a result of the second comparison; Calculating second correction luminance values based on second luminance values, the second luminance difference values, and the second correction factor value that do not coincide with the reference luminance value; And And substituting second luminance values which do not coincide with the reference luminance values with the second correction luminance values.

Images