WO2006013510A1

WO2006013510A1 - De-interlacing

Info

Publication number: WO2006013510A1
Application number: PCT/IB2005/052431
Authority: WO
Inventors: Robert H. Gelderblom; Lambertus A. Van Eggelen; Marco K. Bosma; Alexander H. W. Van Eeuwijk
Original assignee: Koninklijke Philips Electronics NV
Current assignee: Koninklijke Philips NV
Priority date: 2004-07-29
Filing date: 2005-07-20
Publication date: 2006-02-09
Anticipated expiration: 2007-01-29
Also published as: TW200627951A

Abstract

A motion based de-interlacing unit (200) for computing an image on basis of a sequence of video fields is disclosed. The de-interlacing unit comprises: motion determining means (201) for determining a motion indicator for a particular group of pixels (108) of the image; and interpolation means (208) for computing the values of the particular group of pixels (108) on basis of the sequence of video fields and on basis of the motion indicator. The de-interlacing unit further comprises border detection means (230) for detecting whether the particular group of pixels is located at a horizontal border of an object in the image. The interpolation means is forced to perform a non-motion compensated inter-field interpolation if the border detection means (230) determine that the particular group of pixels is located at a horizontal border of an object.

Description

De-interlacing

The invention relates to motion based de-interlacing unit for computing an image on basis of a sequence of video fields, the de-interlacing unit comprising: motion determining means for determining a motion indicator for a particular group of pixels of the image; and - interpolation means for computing the values of the particular group of pixels on basis of the sequence of video fields and on basis of the motion indicator.

The invention further relates to an image processing apparatus comprising such a motion based de-interlacing unit.

The invention further relates to a computer program product to be loaded by a computer arrangement, comprising instructions for de-interlacing a sequence of video fields into an image, the computer arrangement comprising processing means and a memory, the computer program product, after being loaded, providing said processing means with the capability to carry out: determining a motion indicator for a particular group of pixels of the image; and computing the values of the particular group of pixels on basis of the sequence of video fields and on basis of the motion indicator.

The invention further relates to a method of motion based de- interlacing comprising: - determining a motion indicator for a particular group of pixels of the image; and computing the values of the particular group of pixels on basis of the sequence of video fields and on basis of the motion indicator.

An embodiment of the de-interlacing unit of the kind described in the opening paragraph is known from European patent application EP 0395263 A2. This patent application describes a motion adaptive progressive scan conversion. Progressive scan conversion can be carried out in a number of ways, such as by previous field replacement, median filtering in which three spatially consecutive lines are examined (temporally these three lines will come from two consecutive fields), or a motion compensated technique which utilizes multi gradient motion detection followed by multidirectional linear interpolation. In the cited patent application is disclosed to use inter-field interpolation in wholly static picture areas to retain as much vertical information as possible, and to use intra-fϊeld interpolation when significant motion is present.

To detect whether there is motion, groups of pixels from consecutive images are compared in order to compute respective match errors. These match errors are compared with a motion threshold. The motion threshold is not constant but adapted to the local frequency content of the picture. If the frequency content is relatively high then also the motion threshold is relatively high.

Unfortunately, false detections occur, i.e. motion is detected although there is no motion and vice versa. Consequently, an inappropriate interpolation is selected for the corresponding part of the picture resulting in a reduced quality.

It is an object of the invention to provide a de- interlacing unit of the kind described in the opening paragraph which provides output images with a relatively high quality. This object of the invention is achieved in that the de- interlacing unit further comprises border detection means for detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation means is forced to perform a non-motion compensated inter-field interpolation if the border detection means determine that the particular group of pixels is located at a horizontal border of an object.

With a horizontal border is meant that there is a first region in the image which is located above a second region in the image and that a relatively strong transition in color and/or grey value is noticeable between the first region and the second region. This transition corresponds to the border between the first region and the second region. The size of the first region and the second region may be substantially different. For instance the first region corresponds to only a limited number of horizontal lines such as a part of a ticker tape, i.e. a banner, while the second region corresponds to the rest of the image.

Human observers are relatively sensitive for horizontal and vertical structures in images. Disturbances in displaying these structures are noticed relatively well. The inventors have discovered that it is preferred to apply a non motion compensated inter-field interpolation in the event of presence of horizontal structures, i.e. horizontal borders. One can imagine that if intra- field interpolation is inappropriately selected in the event of a horizontal border like a ticker tape, that this would result in a flickering appearance of the ticker tape. The same holds for wrongly selecting a motion compensated inter field interpolation.

In other words, an achievement of an embodiment of the de-interlacing unit according to the invention is that it is arranged to protect horizontal borders, by overriding the decision or output of the motion determining means. Even if the horizontal border is directly adjacent to a moving object, the de-interlacing unit according to the invention will use the interpolation which is suitable for still objects, i.e. no motion. Preferably, the interpolation of the pixels of the moving object is such that mixing of pixels from both sides of the border is prevented. That means that for the selection of input pixels for the interpolation only pixels are selected which are located on a predetermined location relative to the detected border. An embodiment of the motion based de-interlacing unit is a motion adaptive de-interlacing unit, whereby: the motion determining means comprises a motion decision means for computing the motion indicator which represents whether there is motion for the particular group of pixels of the image; and whereby - the interpolation means is arranged to perform:

- inter-field interpolation if the motion indicator indicates that there is no motion for the particular group of pixels or if the border detection means determine that the particular group of pixels is located at a horizontal border of an object; and

- intra-field interpolation if the motion indicator indicates that there is motion for the particular group of pixels and if the border detection means determine that the particular group of pixels is not located at a horizontal border of an object.

An embodiment of the motion based de- interlacing unit is a motion compensated de-interlacing unit, whereby: the motion determining means comprises a motion vector computing means for computing the motion indicator which represents a motion vector for the particular group of pixels of the image; and the interpolation means is arranged to perform: - motion compensated inter-field interpolation on basis of the motion vector if the border detection means determine that the particular group of pixels is not located at a horizontal border of an object; and

- non-motion compensated inter-field interpolation if the border detection means determine that the particular group of pixels is located at a horizontal border of an object.

In an embodiment of the motion based de-interlacing unit according to the invention, the border detection means comprises vertical inconsistency computing means for computing a vertical inconsistency measure which is based on differences between respective pixels of a first group of pixels and a second group of pixels, the first group of pixels being vertically neighboring the second group of pixels. By computing differences between pixel values a transition in pixel values is computed. A relatively big difference corresponds to a relatively strong transition in vertical direction. To determine whether there is a relatively strong transition in vertical direction the vertical inconsistency measure has to be compared to a second measure.

In an embodiment of the motion based de- interlacing unit according to the invention, the border detection means comprises comparing means for comparing the vertical inconsistency measure with a predetermined threshold, which is arranged to determine that the particular group of pixels is located at a horizontal border of an object if the vertical inconsistency measure is larger than the predetermined threshold.

In another embodiment of the motion based de- interlacing unit according to the invention, the border detection means further comprises: horizontal inconsistency computing means for computing a horizontal inconsistency measure which is based on further differences between respective pixels of a third group of pixels and a fourth group of pixels, the third group of pixels and the fourth group of pixels belonging to a previous field which is temporally preceding the image or belonging to a next field which is temporally succeeding the image, the third group of pixels, the fourth group of pixels and the particular group of pixels having the same vertical coordinate; and - comparing means for comparing the horizontal inconsistency measure with the vertical inconsistency measure and being arranged to determine that the particular group of pixels is located at a horizontal border of an object if the vertical inconsistency measure is relatively large compared with the horizontal inconsistency measure. An advantage of comparing the vertical inconsistency measure with the horizontal inconsistency measure instead of comparing with a predetermined threshold, is robustness.

In an embodiment of the motion based de- interlacing unit according to the invention the vertical inconsistency measure is computed by selecting the first group of pixels out of a set of groups of pixels, the selecting based on evaluation of the set of groups of pixels. That means that the selection of the first group of pixels is not pre-determined but based on evaluation of a number of candidate groups of pixels. Preferably, the evaluation is based on maximizing the differences between pixel values. In an embodiment of the motion based de- interlacing unit according to the invention, the horizontal inconsistency measure is computed by selecting the third group of pixels out of a further set of groups of pixels, the selecting based on evaluation of the further set of groups of pixels. That means that the selection of the third group of pixels is not pre¬ determined but based on evaluation of a number of candidate groups of pixels. Preferably, the evaluation is based on maximizing the differences between pixel values.

In an embodiment of the motion based de- interlacing unit according to the : invention, the vertical inconsistency computing means is arranged to compute the vertical inconsistency measure by calculating a sum of differences between respective pixel values of the first group of pixels and the second group of pixels. For example, the inconsistency measure might be the Sum of Absolute Difference (SAD). Preferred the computation of the horizontal inconsistency measure is also based on calculating a sum of absolute differences. It is a further object of the invention to provide an image processing apparatus of the kind described in the opening paragraph which provides output images with a relatively high quality. This object of the invention is achieved in that the de- interlacing unit further comprises border detection means for detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation means is forced to perform a non-motion compensated inter-field interpolation if the border detection means determine that the particular group of pixels is located at a horizontal border of an object.

The image processing apparatus may comprise additional components, e.g. a display device for displaying the output images. The image processing unit might support one or more of the following types of image processing: Video compression, i.e. encoding or decoding, e.g. according to the MPEG standard.

Image rate conversion: From a series of original input images a larger series of output images is calculated. Output images are temporally located between two original input images; and

Temporal noise reduction. This can also involve spatial processing, resulting in spatial-temporal noise reduction.

The image processing apparatus might e.g. be a TV, a set top box, a VCR (Video Cassette Recorder) player, a satellite tuner, a DVD (Digital Versatile Disk) player or recorder.

It is a further object of the invention to provide a computer program product of the kind described in the opening paragraph which provides output images with a relatively high quality.

This object of the invention is achieved by further detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation is a non-motion compensated inter-field interpolation if the border detection indicates that the particular group of pixels is located at a horizontal border of an object.

It is a further object of the invention to provide a method of the kind described in the opening paragraph which provides output images with a relatively high quality. This object of the invention is achieved by further detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation is a non-motion compensated inter-field interpolation if the border detection indicates that the particular group of pixels is located at a horizontal border of an object.

Modifications of the de-interlacing unit and variations thereof may correspond to modifications and variations thereof of the image processing apparatus, the method and the computer program product, being described.

These and other aspects of the de-interlacing unit, of the image processing apparatus, of the method and of the computer program product, according to the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:

Fig. 1 schematically shows portions of three consecutive video fields; Fig. 2 schematically shows an embodiment of the de-interlacing unit according to the invention;

Fig. 3 A schematically shows an embodiment of a motion adaptive de- interlacing unit according to the invention; Fig. 3B schematically shows an other embodiment of a motion adaptive de- interlacing unit according to the invention;

Fig. 3 C schematically shows an other embodiment of a motion adaptive de- interlacing unit according to the invention, having a controllable match error computing unit;

Fig. 4 schematically shows an embodiment of a motion compensated de- interlacing unit according to the invention;

Fig. 5A schematically shows a sequence of output images which has not been generated by means of a de-interlacing unit according to the prior art;

Fig. 5B schematically shows a sequence of output images which has been generated by means of an embodiment of the de-interlacing unit according to the invention; Fig. 6A schematically shows an embodiment of the border detection unit according to the invention;

Fig. 6B the schematically shows an alternative embodiment of the border detection unit according to the invention; and

Fig. 7 schematically shows an image processing apparatus according to the invention.

Same reference numerals are used to denote similar parts throughout the figures.

Interlacing is the common video broadcast procedure for transmitting the odd or even numbered video frame lines alternately, i.e. field by field. De-interlacing attempts to restore the full vertical resolution, i.e. make odd and even lines available simultaneously for each video frame. In this specification with an image is meant either a field or a frame. There are several types of de-interlacing. The focus in this document is on motion based de-interlacing. Motion based de-interlacing may be motion adaptive de- interlacing whereby for a particular group of pixels it is determined whether there is motion or not. The type of interpolation is based on this determination. An alternative motion based de-interlacing is motion compensated de-interlacing whereby for a particular group of pixels the amount of motion and the direction of motion is estimated, i.e. a motion vector is computed. On basis of the computed motion vector a motion compensated interpolation is performed.

But besides determining the motion for the particular pixel or a group of pixels it is also important to determine whether the particular pixel is located at a border. According to the invention, the method of interpolation is pre-determined in the event of a detected border.

First a preferred method of determining whether there is motion will be explained and secondly embodiments of border detection are explained according to the invention. The combination of both results in a motion adaptive de-interlacing unit providing high-quality output images.

In connection with Fig. 4 a motion compensated the interlacing unit according to the invention is briefly described.

Fig. 1 schematically shows portions of three consecutive video fields 100, 102 and 106. Fig. 1 also shows an output image 104. The blocks, e.g. 108-114 which are depicted in Fig. 1 correspond to groups of pixels. Each group of pixels comprises e.g. 8 pixels. A group of pixels might also comprise only one pixel. Notice that a part of the output image 104 and one of the three consecutive video fields 102 are the same. The blocks which are depicted with solid lines correspond to pixel values which are actually present in the video signal as received, i.e. the input video fields. The blocks which are depicted with dashed lines correspond to pixel values which have to be computed on basis of the pixel values which are actually present in the video signal as received. The combination of actually received pixel values and computed pixel values together form the output video image 104.

The odd video lines y-3, y-1, y+1, y+3 of the output image 104 are copies from the video lines of the current video field 102. The depicted portions of the even video lines y-2, y, y+2, y +4 are based on corresponding video lines from one or more other video fields if no motion has been detected for these portions. This type of interpolation is called inter- field interpolation. However if motion has been detected, then also these even video lines y-2, y, y+2, y +4 are based on the video lines of the current video field 102. Preferably, the pixel values of the portions of the even video lines of the output image 104 are computed by means of edge dependent, i.e. orientation dependent interpolation. This latter type of interpolation is called intra-field interpolation.

In order to decide which type of interpolation, i.e. inter-field interpolation or intra field interpolation, has to be selected for a particular pixel or group of pixels, it is important to determine whether there is motion for the particular pixel or the group of pixels. Below a preferred method of determining whether there is motion will be explained for a particular group of pixels 108. The particular group of pixels 108 is located at a particular spatial position x,y.

The method of determining whether there is motion comprises the following steps: a match error E(x,y) is computed on basis of comparing a first group of pixels 114 of a first video field 100 with a second group of pixels 112 of a second video field 106. The spatial position of the first group of pixels 114, the spatial position of the second group of pixels 112 and the spatial position of the particular group of pixels 108 are mutually equal, i.e. x,y. The match error E(x,y) is computed by means of calculating a sum of absolute differences between respective pixel values of the first group of pixels 114 and pixel values of the second group of pixels 112. Alternatively, first a low pass filtering is performed; a motion threshold T(x,y) is computed for the particular group of pixels 108. Preferably, the motion threshold T(x,y) is based on a number of components: a predetermined constant, a noise dependent component, a contrast dependent component and an environment dependent component. Most of these components are optional but the latter one is obligatory. The different optional components are explained in more detail in connection with Fig. 3 A. The environment dependent component is explained in more detail below; - a motion indicator M(x,y) which represents whether there is motion for the particular group of pixels 108 is computed by means of comparing the match error E(x,y) with the motion threshold T(x,y). If the match error E(x,y) is larger than the motion threshold T(x,y) then it is assumed that there is motion for the particular group of pixels 108. So, the motion indicator M(x,y) has a value corresponding with motion, e.g. M(x,y) =1. However, if the match error E(x,y) is lower than the motion threshold T(x,y), then it is assumed that there is no motion for the particular group of pixels 108. Consequently the motion indicator M(x,y) has a value corresponding with no motion, e.g. M(x,y)=0.

The values of the pixels of the particular group of pixels are computed by means of interpolation. The type of interpolation depends on the computed value of the motion indicator M(x,y), as explained above.

As said, the motion threshold T(x,y) comprises an environment dependent component. That means that the values of the motion indicators M(x,y) being computed for other groups of pixels, e.g. a fourth group of pixels 110, are taken into account to compute the current motion threshold T(x,y) belonging to the particular group of pixels 108. The basic rule is that the values of a predetermined number of motion indicators in the spatial and/or temporal environment of the particular group of pixels 108 are evaluated. For instance the values of the following motion indicators are evaluated: M(x-3, y), M(x-1, y), M(x, y-2), M(x+2, y-2), M(x, y+3). If the number of motion indicators in this set having a value corresponding to motion is relatively high, then the motion threshold T(x,y) for the particular group of pixels 108 is relatively low. If the number of motion indicators in this set having a value corresponding to no motion is relatively high, then the motion threshold T(x,y) for the particular group of pixels 108 is relatively high.

It should be noted that, although only the motion indicators of a limited set are directly taken into account to determine the motion threshold T(x,y) for the particular group of pixels 108, the actual influence of previous motion indicators is much bigger. This is because of recursion, since the previous motion indicators are on their turn based on other motion indicators which have been determined before.

Fig. 2 schematically shows an embodiment of the de-interlacing unit 200 according to the invention. The motion based de- interlacing unit 200 comprises: a border detection unit 230 for detecting whether the particular group of pixels 108 is located at a horizontal border of an object. This border detection unit will be explained in more detail in connection with Fig. 6 A and Fig 6B. The border detection unit 230 as may be provided with three inputs, image data from a previous field, a current field and a next field. Depending on the type of border detection some of these inputs are optional. a motion determining unit 201 for computing a motion indicator which represents motion for a group of pixels, for which the values have to be computed by means of interpolation. The motion determining unit may be a motion decision unit 206 as described in connection with Figs. 3A-3C or may be a motion vector computing unit for computing a motion vector for the particular group of pixels 108 of the image; and an interpolation unit 208 for computing pixel values. The interpolation unit 208 is controlled by the motion determining unit 201 and the border detection unit 230. The interpolation unit 208 is provided with input video fields and provides video frames at its output connector 216. The interpolation unit 208 is arranged to switch per pixel or group of pixels between multiple types of interpolation on basis of the values of the provided motion indicators and the output of the border detection unit 230. The interpolation unit 208 is forced to perform a non motion compensated inter-field interpolation if the border detection unit 230 determines that the particular group of pixels is located at a horizontal border of an object. The border detection unit 230, the motion determining unit 201, and the interpolation unit 208 may be implemented using one processor. Normally, these functions are performed under control of a software program product. During execution, normally the software program product is loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, like a ROM, hard disk, or magnetical and/or optical storage, or may be loaded via a network like Internet. Optionally an application specific integrated circuit provides the disclosed functionality.

The de-interlacing unit 200 according to the invention is provided with consecutive video fields at its respective input connectors 210-214. The image processing apparatus in which the de-interlacing unit 200 is comprised, has a storage device for temporary storage of a number of video fields. This storage device is not depicted in Fig. 2.

Fig. 3 A schematically shows an embodiment of the de-interlacing unit 300 according to the invention. The motion adaptive de-interlacing unit 300 comprises: a border detection unit 230 for detecting whether the particular group of pixels 108 is located at a horizontal border of an object; a match error computing unit 202 for computing match errors on basis of comparing groups of pixels of different video fields; a motion threshold computing unit 204 for computing motion thresholds; a motion decision unit 206 for computing motion indicators which represents whether there is motion for a group of pixels, for which the values have to be computed by means of interpolation. The motion decision unit 206 is provided with the match errors being computed by the match error computing unit 202 and is provided with the motion thresholds being computed by the motion threshold computing unit 204. The motion decision unit 206 is arranged to compute the motion indicators by means of comparing the match errors with the corresponding motion thresholds; and an interpolation unit 208 for computing pixel values. The interpolation unit 208 is arranged to switch per pixel or group of pixels between inter-field interpolation and intra field interpolation on basis of the values of the provided motion indicators and the output of the border detection unit 230. The interpolation unit 208 is forced to perform inter- field interpolation if the border detection unit 230 determines that the particular group of pixels is located at a horizontal border of an object.

The match error computing unit 202, the motion threshold computing unit 204, the motion decision unit 206, the interpolation unit 208 and the border detection unit 230 may be implemented using one processor. Normally, these functions are performed under control of a software program product. During execution, normally the software program product is loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, like a ROM, hard disk, or magnetical and/or optical storage, or may be loaded via a network like Internet. Optionally an application specific integrated circuit provides the disclosed functionality.

The match error computing unit 202 optionally comprises two horizontal low pass filters 218 and 220 for low pass filtering the input data. That means that low pass filtered pixel values of a first video field, typically the previous video field, are compared with low pass filtered pixel values of the second video field, typically the next video field. Preferably, comparing is based on computing differences. The reason for low pass filtering is to compensate for sync jitter.

The motion threshold computing unit 204 comprises processing means 222- 226 for computing the different components of the motion thresholds. The motion threshold computing unit 204 comprises an environment evaluation unit 226 to evaluate the motion indicators in the environment, as described in connection with Fig. 1. Notice the feedback connection 228 by means of which values of motion indicators are provided from the motion decision unit 206 to the motion threshold computing unit, in particular the environment evaluation unit 226. The de-interlacing unit 200 comprises not depicted storage means for temporary storage of motion indicators. The motion threshold computing unit 204 preferably comprises a contrast computing unit 222 for computing local contrast. Preferably, mutual differences between pixel values within groups of pixels of multiple video fields are applied to compute the contrast values. Only a limited number of groups of pixels in the environment of the particular group of pixels under consideration are used for the calculation of the contrast value. If the contrast value is relatively high, then the corresponding motion threshold is also relatively high. If the contrast value is relatively low, then the corresponding motion threshold is also relatively low.

The motion threshold computing unit 204 optionally comprises a noise computing unit 224 for computing a global noise value. Mutual differences between pixel values within groups of pixels are used to compute the global noise value. A limited number of groups of pixels being located at a number of locations within the video field are used. These groups of pixels are not necessarily located in a direct environment of the particular group of pixels. Preferably, the groups of pixels are located in regions where there is no motion. If the noise value is relatively high, then the motion thresholds are also relatively high. If the noise value is relatively low, then the motion thresholds are also relatively low. Fig. 3B schematically shows another embodiment of the de-interlacing unit

301 according to the invention. This embodiment of the de-interlacing unit 301 is almost equal to the de-interlacing unit 200 which is described in connection with Fig. 3A. The only difference is the final decision unit 302. The control of the interpolation unit 208 is not only based on the current motion indicator M(x,y) for a particular group of pixels 108 being located at the spatial position (x,y) and output of the border detection unit 230 but is also based on other motion indicators in the environment. Preferably, the final motion indicator F(x,y) is based on motion indicators of adjacent groups of pixels, e.g. F(x,y) = M(x, y) * M(x-1, y) * M(x+1, y) * M(x, y-1) * M(x, y+1) where a "1" indicates motion and a "0" indicates no motion, i.e. still.

Fig. 3 C schematically shows another embodiment of the de-interlacing unit

302 according to the invention, having a controllable match error computing unit 202. This embodiment of the de-interlacing unit 302 is relatively similar to the de-interlacing unit 301 which is described in connection with Fig. 3A. The major difference is the deployment and working of the environment evaluation unit 226. The match error computing unit 202 of the embodiment of the de- interlacing unit 302 comprises the environment evaluation unit 226.

Notice the feedback connection by means of which values of motion indicators are provided from the motion decision unit 206 to the match error computing unit 202, in particular the environment evaluation unit 226. The de-interlacing unit 302 comprises not depicted storage means for temporary storage of motion indicators. The environment evaluation unit 226 is arranged to evaluate the motion indicators in the environment. However now the output of the environment interpolation unit 226 is opposite to the output as in the case of the de- interlacing unit 200 as described in connection with Fig. 3 A.

The match error E(x,y) comprises a local component and an environment dependent component. The local component is based on pixel value differences of pixels being located at spatial position x,y. The environment dependent component is based on values of motion indicators being computed for other groups of pixels. That means that the values of the motion indicators being computed for other groups of pixels, e.g. a fourth group of pixels 110, are taken into account to compute the current match error E(x,y) belonging to the particular group of pixels 108. The basic rule is that the values of a predetermined number of motion indicators in the spatial and/or temporal environment of the particular group of pixels 108 are evaluated. For instance the values of the following motion indicators are evaluated: M(x-3, y), M(x-1, y), M(x, y-2), M(x+2, y-2), M(x, y+3). If the number of motion indicators in this set having a value corresponding to motion is relatively high, then the match error E(x,y) for the particular group of pixels 108 is relatively high. If the number of motion indicators in this set having a value corresponding to no motion is relatively high, then the match error E(x,y) for the particular group of pixels 108 is relatively low.

It should be noted that, although only the motion indicators of a limited set are directly taken into account to determine the match error E(x,y) for the particular group of pixels 108, the actual influence of previous motion indicators is much bigger. This is because of recursion, since the previous motion indicators are on their turn based on other motion indicators which have been determined before.

Fig. 4 schematically shows an embodiment of a motion compensated de- interlacing unit 400 according to the invention. The motion based de- interlacing unit 400 comprises: a border detection unit 230 for detecting whether the particular group of pixels .108 is located at a horizontal border of an object; a motion vector computing unit 201 for computing a motion indicator which represents motion for a group of pixels, for which the values have to be computed by means of interpolation. The motion vector computing unit is arranged to compute a motion vector for the particular group of pixels 108 of the image; and - an interpolation unit 208 for computing pixel values. The interpolation unit

208 is controlled by the motion determining unit 201 and the border detection unit 230. The interpolation unit 208 is provided with input video fields and provides output images at its output connector 216. The interpolation unit 208 is arranged to switch per pixel or group of pixels between multiple types of interpolation on basis of the values of the provided motion indicators and the output of the border detection unit 230. The interpolation unit 208 is arranged to perform motion compensated inter-field interpolation on basis of the motion vector if the border detection means 230 determine that the particular group of pixels is not located at a horizontal border of an object; and is arranged to perform non-motion compensated inter-field interpolation if the border detection means 230 determine that the particular group of pixels is located at a horizontal border of an object.

Non motion compensated inter-field interpolation may be performed by field insertion, i.e. without taking care of motion or by inter-field interpolation using a motion vector which is equal to the null vector. In the first case the values of the particular group of pixels 108 are copies of corresponding values of another group of pixels from the previous or the next field, whereby the spatial coordinates of the particular group of pixels and of the said another group of pixels are mutually equal. In the second case the values of the particular group of pixels are based on combining the values of a first group of pixels from the previous field and a second group of pixels from the next field, whereby the spatial coordinates of the first group of pixels, the second group of pixels and of the particular group of pixels are mutually equal.

Motion vector computing unit 201 comprises: a motion vector candidate generating unit 402 for generating a set of motion vector candidates; a motion vector candidate evaluation unit 404 for evaluating the motion vector candidates of the set; and a motion vector selection unit 406 for selecting the motion vector for the particular group of pixels on basis of the evaluation of the motion vector candidates.

The border detection unit 230, the motion determining unit 201, and the interpolation unit 208 may be implemented using one processor.

Next the importance of the border detection will be explained. This is done by comparing two sequences of output images. Fig. 5 A schematically shows a sequence of output images 502-506 which has been generated by means of an embodiment of the de- interlacing unit according to the prior art, while Fig. 5B schematically shows a sequence of output images 514-518 which has been generated by means of an embodiment of the de- interlacing unit according to the invention.

The sequence of output images 502-506 originate from a television broadcast. The upper part 508 of the images 502-506 show a driving car, moving from the right to the left. The lower part 510 correspond to a banner, e.g. for displaying prizes at the stock exchange. The banner comprises a gray region on which characters move from the left to the right. There is a distinct border between the upper part 508 and the lower part 510. In this case the banner comprises a graphics line 512. However, because of inappropriate de- interlacing the graphics line 512 does not appear in each of the output images 502-506. In fact the graphics line 512 is visible in only half of the images. That means the graphics line is visible, is not visible, is visible, is not visible, et cetera. The effect of this is a flickering which is perceived as very annoying. It is caused by applying intra- field interpolation.

Fig. 5B schematically shows a sequence of output images 514-518 which is based on the same sequence of input fields as the sequence shown in Fig. 5 A. However, in the sequence 514-518 the graphics line 512 is continuously visible. This is achieved by inter- field interpolation. The selection of inter-field interpolation instead of intra field interpolation is based on the detection of the horizontal border between the upper part 508 and the lower part 510 of the images.

Fig. 6A schematically shows an embodiment of the border detection unit 230 according to the invention. The border detection unit 230 comprises: a vertical inconsistency computing unit 602 for computing a vertical inconsistency measure which is based on differences between respective pixels of a first group of pixels and a second group of pixels, the first group of pixels being vertically neighboring the second group of pixels; - a comparing unit 604 for comparing the vertical inconsistency measure with a predetermined border threshold. The comparing unit 604 is arranged to determine that the particular group of pixels 108 is located at a horizontal border of an object if the vertical inconsistency measure is larger than the predetermined border threshold; an input connector 608 for providing the border detection unit 230 with input data, i.e. a video field 100, 102, 106; an output connector 610 for outputting a value indicating whether there is a border detected or not; and a further input connector 606 for providing the border detection unit 230 with the predetermined threshold. The predetermined threshold may be based on image content. The working of the border detection unit 230 as depicted in Fig. 6A is as follows. See also Fig. 1. Suppose that for a particular group of pixels 108 in the current field it has to be determined whether there is a border or not.

A first way to do this is based on computing the vertical inconsistency measure V(x,y) with pixel values of the previous field P_p , e.g. V(x,y) = SAD(P_p(x,y),P_p(x,y- 2)) (1) or with

V(x,y) = SAD(P_p(x,y),P_p(x,y + 2)), (2) whereby SAD(P_p (x,y),P_p(x,y — 2)) means the sum of absolute differences between the respective pixels of the groups of pixels located at spatial position x,y and x,y-2, respectively. Alternatively the maximum of the values of Equation 1 and 2 is used:

V(x,y) = mvL(SAD(P_p(x,y),P_p(x,y-2)),SAD(P_p(x,y),P_p(x,y + 2))) (3) A second way to do this is based on computing the vertical inconsistency measure V(x,y) with pixel values of the current field P_c , e.g.

V(x,y) = SAD(P_c(x, y-l),P_c(x,y +I)) (4)

A third way to do this is based on computing the vertical inconsistency measure V(x, y) with pixel values of the next field P_n , e.g.

V(x,y) = SAD(P_fl(x,y),P_n(x,y-2)) (5) or with

V(x,y) = SAD(P_n(x,y),P_n(x,y + 2)) (6)

Alternatively the maximum of the values of Equation 5 and 6 is used: V(x,y) = m_Sκ(SAD(P_n(x,y),P_n(x,y- 2)),SAD(P_n(x,y),P_n(x,y + 2))) (7)

After having computed the vertical inconsistency measure the border detection proceeds with comparing the vertical inconsistency measure with the predetermined border threshold B , if V(x,y) ≥ B then it is assumed that there is a border for the particular group of pixels 108, else it is assumed that there is no border for the particular group of pixels 108. Fig. 6B the schematically shows an alternative embodiment of the border detection unit 230 according to the invention. The border detection unit 230 comprises: a vertical inconsistency computing unit 602 for computing a vertical inconsistency measure which is based on differences between respective pixels of a first group 114 of pixels and a second group of pixels, the first group of pixels being vertically neighboring the second group of pixels; horizontal inconsistency computing means 614 for computing a horizontal inconsistency measure which is based on further differences between respective pixels of a third group of pixels 114 and a fourth group of pixels 112, the third group of pixels 114 and the fourth group of pixels 112 belonging to a previous field which is temporally preceding the image or belonging to a next field 100 which is temporally succeeding the image 104, the third group of pixels 114, the fourth group of pixels 112 and the particular group of pixels 108 having the same vertical coordinate y; a comparing unit 604 for comparing the horizontal inconsistency measure with the vertical inconsistency measure and being arranged to determine that the particular group of pixels is located at a horizontal border of an object if the vertical inconsistency measure is relatively large compared with the horizontal inconsistency measure; an input connector 608 for providing the border detection unit 230 with input data, i.e. a video field 100, 102, 106; and an output connector 610 for outputting a value indicating whether there is a border detected or not. The working of the border detection unit 230 as depicted in Fig. 6B is as follows. See also Fig. 1. Suppose that for a particular group of pixels 108 in the current field it has to be determined whether there is a border or not.

The vertical inconsistency measure can be computed with one of the Equations 1-7 The horizontal inconsistency measure can be computed with one of the following Equations:

H(x,y) = SAD(P_p(x,y),P_n(x,y)) (8) or

H(x,y) = SAD(P_p(x-l,y),P_p(x + l,y)) (9) or

H(x,y) = SAD(P_p(x-l,y),P_p(x,y)) (10) or

H(x,y) = SAD(P_p(x,y),P_p(x + l,y)) (11) or H(X,;^;) = SAD(P_n(x - l,y),P_n(x + hy)) (12) or

H(x,y) = SAD(P_n(x-l,y),P_n(x,y)) (13) or

H(x,y) = SAD(P,χx,y),P_n(x + l,y)) (14) Alternative ways of computing the horizontal inconsistency measure are also possible.

Preferably the maximum of the values of Equations 8-14 is used.

After having computed the vertical inconsistency measure V(x,y) and the horizontal inconsistency measure H(x,y) , the border detection proceeds with comparing the vertical inconsistency measure with the horizontal inconsistency measure H(x,y) . If V(x, y) > C* H(x, y) +D then it is assumed that there is a border for the particular group of pixels 108, else it is assumed that there is no border for the particular group of pixels 108. Hereby are C and D constants. Typical values are 6 and 128. Typical number of pixels of a group of pixels equals 8.

Fig. 7 schematically shows an image processing apparatus 700 according to the invention, comprising: - receiving means 702 for receiving a signal representing input images; the de-interlacing unit 704 as described in connection with any of the Figs. 2 or 3 or 4; and an optional display device 706.

The signal may be a broadcast signal received via an antenna or cable but may also be a signal from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk (DVD). The signal is provided at the input connector 708. The image processing apparatus 700 might e.g. be a TV or a personal computer comprising a video signal receiving module. Alternatively the image processing apparatus 700 does not comprise the optional display device 706 but provides the output images to an apparatus that does comprise a display device 706. Then the image processing apparatus 700 might be e.g. a set top box, a satellite-tuner, a VCR player, a DVD player or recorder. Optionally the image processing apparatus 700 comprises storage means, like a hard-disk or means for storage on removable media, e.g. optical disks. The image processing apparatus 700 might also be a system being applied by a film-studio or broadcaster.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitable programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words are to be interpreted as names.

Claims

CLAIMS:

1. A motion based de-interlacing unit (200) for computing an image on basis of a sequence of video fields, the de- interlacing unit comprising: motion determining means (201) for determining a motion indicator for a particular group of pixels (108) of the image; and - interpolation means (208) for computing the values of the particular group of pixels (108) on basis of the sequence of video fields and on basis of the motion indicator, characterized in that the de-interlacing unit further comprises border detection means (230) for detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation means is forced to perform a non-motion compensated inter-field interpolation if the border detection means (230) determine that the particular group of pixels is located at a horizontal border of an object.

2. A motion based de- interlacing unit (200) as claimed in claim 1, being a motion adaptive de-interlacing unit, whereby: - the motion determining means (201) comprises a motion decision means (206) for computing the motion indicator which represents whether there is motion for the particular group of pixels (108) of the image; and whereby the interpolation means is arranged to perform:

- inter- field interpolation if the motion indicator indicates that there is no motion for the particular group of pixels (108) or if the border detection means (230) determine that the particular group of pixels is located at a horizontal border of an object; and

- intra- field interpolation if the motion indicator indicates that there is motion for the particular group of pixels (108) and if the border detection means (230) determine that the particular group of pixels is not located at a horizontal border of an object.

3. A motion based de-interlacing unit (200) as claimed in claim 1, being a motion compensated de-interlacing unit, whereby: the motion determining means (201) comprises a motion vector computing means for computing the motion indicator which represents a motion vector for the particular group of pixels (108) of the image; and the interpolation means is arranged to perform:

- motion compensated inter-field interpolation on basis of the motion vector if the border detection means (230) determine that the particular group of pixels is not located at a horizontal border of an object; and

- non-motion compensated inter-field interpolation if the border detection means (230) determine that the particular group of pixels is located at a horizontal border of an object.

4. A motion based de-interlacing unit (200) as claimed in any of the Claims 1-3, whereby the border detection means (230) comprises vertical inconsistency computing means (602) for computing a vertical inconsistency measure which is based on differences between respective pixels of a first group of pixels and a second group of pixels, the first group of pixels being vertically neighboring the second group of pixels.

5. A motion based de-interlacing unit (200) as claimed in Claim 4, whereby the border detection means (230) comprises comparing means (604) for comparing the vertical inconsistency measure with a predetermined threshold, which is arranged to determine that the particular group of pixels is located at a horizontal border of an object if the vertical inconsistency measure is larger than the predetermined threshold.

6. A motion based de-interlacing unit (200) as claimed in Claim 4, whereby the border detection means (230) further comprises: horizontal inconsistency computing means (614) for computing a horizontal inconsistency measure which is based on further differences between respective pixels of a third group of pixels and a fourth group of pixels, the third group of pixels and the fourth group of pixels belonging to a previous field which is temporally preceding the image or belonging to a next field which is temporally succeeding the image, the third group of pixels, the fourth group of pixels and the particular group of pixels having the same vertical coordinate; and comparing means (604) for comparing the horizontal inconsistency measure with the vertical inconsistency measure and being arranged to determine that the particular group of pixels is located at a horizontal border of an object if the vertical inconsistency measure is relatively large compared with the horizontal inconsistency measure.

7. A motion based de-interlacing unit (200) as claimed in Claim 4, whereby the vertical inconsistency measure is computed by selecting the first group of pixels out of a set of groups of pixels, the selecting based on evaluation of the set of groups of pixels.

8. A motion based de-interlacing unit (200) as claimed in Claim 7, whereby the evaluation is based on maximizing the differences.

9. A motion based de-interlacing unit (200) as claimed in Claim 6, whereby the horizontal inconsistency measure is computed by selecting the third group of pixels out of a further set of groups of pixels, the selecting based on evaluation of the further set of groups ofpixels.

10. A motion based de- interlacing unit (200) as claimed in Claim 4, whereby the vertical inconsistency computing means (602) is arranged to compute the vertical inconsistency measure by calculating a sum of differences between respective pixel values of the first group ofpixels and the second group ofpixels.

11. An image processing apparatus (700) comprising: - receiving means (702) for receiving a signal corresponding to a sequence of video fields; and a motion based de- interlacing unit (704) for calculating an image on basis of the sequence of video fields, the motion based de-interlacing unit (704) as claimed in Claim

1.

12. A computer program product to be loaded by a computer arrangement, comprising instructions for de-interlacing a sequence of video fields into an image, the computer arrangement comprising processing means and a memory, the computer program product, after being loaded, providing said processing means with the capability to carry out: - determining a motion indicator for a particular group ofpixels (108) of the image; and computing the values of the particular group ofpixels (108) on basis of the sequence of video fields and on basis of the motion indicator, characterized in further detecting whether the particular group ofpixels is located at a horizontal border of an object in the image and that the interpolation is a non-motion compensated inter- field interpolation if the border detection indicates that the particular group of pixels is located at a horizontal border of an object.

13. A method of motion based de-interlacing a sequence of video fields into an image comprising: determining a motion indicator for a particular group of pixels (108) of the image; and computing the values of the particular group of pixels (108) on basis of the sequence of video fields and on basis of the motion indicator, characterized in further detecting whether the particular group of pixels is located at a horizontal border of an object in the image and that the interpolation is a non-motion compensated inter-field interpolation if the border detection indicates that the particular group of pixels is located at a horizontal border of an object.