[go: up one dir, main page]

WO1999043161A1 - Determining visually noticeable differences between two images - Google Patents

Determining visually noticeable differences between two images Download PDF

Info

Publication number
WO1999043161A1
WO1999043161A1 PCT/GB1999/000487 GB9900487W WO9943161A1 WO 1999043161 A1 WO1999043161 A1 WO 1999043161A1 GB 9900487 W GB9900487 W GB 9900487W WO 9943161 A1 WO9943161 A1 WO 9943161A1
Authority
WO
WIPO (PCT)
Prior art keywords
input image
image
contrast
input
pixels
Prior art date
Application number
PCT/GB1999/000487
Other languages
French (fr)
Inventor
Ping Wu
Original Assignee
Tandberg Television Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tandberg Television Limited filed Critical Tandberg Television Limited
Priority to AU26299/99A priority Critical patent/AU2629999A/en
Publication of WO1999043161A1 publication Critical patent/WO1999043161A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding

Definitions

  • the present invention relates to determining visually noticeable differences between two images.
  • the invention has particular application to the encoding and later decoding of an image in such fields as television signal generation, compression and transmission.
  • When encoding an original image for television transmission and display at a receiver it is important to know how well the system for transmitting and receiving will perform in reproducing the original image.
  • the performance of the system is related to the visually noticeable differences between the original image and the displayed image.
  • a number of models have already been proposed to predict whether a human observer will be able to discriminate between two images.
  • One such visual discrimination model is known as the Sarnoff visual discrimination model.
  • a visual discrimination model may take, as input, a pair of images and provide an output which is a map showing the probability, as a function of position on the images, that an observer would be able to detect differences between the images.
  • the map is referred to as a JND map showing Just Noticeable Differences. This JND map can itself be presented as an image, with higher grey levels corresponding to higher probabilities of discrimination.
  • the image fidelity should be directly proportional to the quantisation sampling density. This assumption may be erroneous with the result that encoding bits are squandered where they have little impact on the fidelity of reproduction of an image.
  • a better method of controlling quantisation parameters is to base control on a JND map to obtain either more uniform image fidelity for a fixed bit rate of encoding or better bit rates for a desired level of fidelity.
  • a method of identifying visually noticeable differences between a first input image and a second input image comprising the steps of: generating a first, reference contrast image from the first input image; generating a second contrast image from the second input image; forming a contrast difference image from the first and second contrast images; and selecting, from the contrast difference image, each pixel which has an intensity exceeding a threshold determined by the intensity of the corresponding pixel in the reference contrast image.
  • apparatus for identifying visually noticeable differences between a first input image and a second input image
  • the apparatus comprising: a generator to generate a first, reference contrast image from the first input image and a second contrast image from the second input image; a difference forming circuit to form a contrast difference image from the first and second contrast images; and a selector to select from the contrast difference image each pixel, which has an intensity exceeding a threshold, determined by the intensity of the corresponding pixel in the reference contrast image.
  • the selected pixels which exceed the threshold value are indicative of the visually noticeable differences between the first input image and the second input image.
  • the selected pixels will indicate where noticeable differences occur between them as a result of the compression.
  • the degree of compression can thus be increased in those areas of each image where visually noticeable differences do not appear.
  • Figure 1 shows in block diagram form a video signal transmission apparatus embodying the present invention
  • Figure 2 shows a video signal frame transmitted by the apparatus of Figure 1 ;
  • Figure 3 shows a control circuit included in the apparatus of Figure 1 ;
  • Figure 4 shows a block of pixels that are processed by the circuit of Figure 3;
  • Figure 5 shows a block diagram of steps used in processing signals in the circuit of Figure 3 to provide a map of just noticeable differences between two image frames;
  • Figure 6 shows a graph of threshold values applicable to the process steps in Figure 5;
  • Figure 7 shows a conventional television signal encoder apparatus
  • Figure 8 shows the apparatus of Figure 7 modified according to the invention to add a control circuit
  • FIG. 8A shows details of the control circuit added into the apparatus of Figure 8.
  • Figure 9 shows a further television signal encoder modified to incorporate the control circuit of Figure 8A.
  • Figure 10 shows a statistical multiplexer apparatus incorporating a plurality of the control circuits of Figure 8A.
  • Figure 1 shows a video signal transmission apparatus including an analogue to digital converter 10 to receive an input analogue television signal 11 for conversion to a digital signal.
  • the input digital signal is encoded and compressed in an encoder 12 and applied to a transmission circuit 13.
  • the transmission circuit 13 broadcasts the compressed signal in conventional manner for reception by one or more receiver circuits 14.
  • the receiver circuit 14 applies the received signal to a decoder 15 for decoding and supply to a display 16.
  • Figure 2 shows a frame 10 included in the digital signal output from the encoder 12.
  • the digital television signal includes a sequence of frames 10.
  • Each frame consists of a matrix of pixels divided into M x N blocks of pixels.
  • a control circuit 30 is incorporated into the encoder 12 to control the compression of the television signal so as to reduce the effect of the unwanted noise.
  • the control circuit 30 is shown in further detail in Figure 3.
  • the digital television signal from the analogue to digital converter 10 is applied to a compression circuit 31 for compression and supply to an output terminal 38 connected to the transmission circuit 13.
  • the digital television signal is also supplied to a contrast image generator 33.
  • the generator 33 generates a contrast image from the pixels of each frame and applies the contrast image to a difference circuit 35.
  • the compressed signal at the output terminal 38 is supplied to a decompression circuit 34 where the signal is subject to decompression and the resulting decompressed signal is passed to a 1 - frame store 32.
  • the output of the 1 - frame store 32 is applied to a second contrast image generator 36.
  • the difference circuit 35 receives the contrast images from the two contrast image generators and produces a contrast difference image.
  • the contrast difference image is mapped in a JND Oust noticeable difference) map circuit 37.
  • the map circuit 37 applies an output signal to control the compression circuit 31.
  • Each frame of the digital television signal applied to the contrast image generator 33 and the compression circuit 37 includes the blocks of pixels already mentioned and shown in Figure 2.
  • each of the pixels is processed in relation to a 3 x 3 neighbourhood area as shown in Figure 4.
  • Each pixel a, ,j is included in a 3 x 3 area extending from the pixel a ⁇ to the pixel a, + ⁇ ⁇ J+ ⁇
  • the image generator 33 generates a contrast value c,,, for each pixel a according to the relation;
  • Lmax is the maximum luminance value of the pixels in the 3 x 3 neighbourhood area and L min is the minimum luminance value of the pixels in the 3 x 3 neighbourhood area.
  • a signal frame I of the digital television signal is received by the contrast image generator 33 in step 50 at the same time that a signal frame II is received by the contrast image generator 36 in step 51.
  • the signal frame I is converted in step 52 into a contrast image I as already described above with reference to Figure 3.
  • the signal frame II is a reconstruction of the television signal frame which precedes the signal frame I.
  • the signal frame II is derived by the decompression circuit 34 and supplied to the 1 - frame store 32.
  • the signal frame II, supplied in step 51 is converted in step 53 into a contrast image II by the contrast image generator 36.
  • the contrast image generator 36 uses the same process as the contrast image generator 33 to generate its contrast image.
  • the difference circuit 35 compares the contrast value of each pixel in the contrast image I with the contrast value of each pixel in the contrast image II.
  • the result is a contrast difference image in which the contrast image differences for each frame are mapped in comparison with the preceding frame in the television signal.
  • the contrast image I is applied, in step 54, to the JND map circuit 37.
  • the contrast difference image is applied, in step 55, to the JND map circuit 37.
  • the contrast image I is used as a reference for each pixel of the contrast difference image so as to select pixels from the contrast difference image in each frame as will be explained with reference to Figure 6.
  • FIG. 6 there is shown a threshold curve Ct plotted against a difference value ⁇ c along one axis and a reference value C along the other axis. Each axis has a logarithmic scale.
  • the value ⁇ c is the difference value of a pixel in the contrast difference image supplied by the difference circuit 35.
  • the reference value C is the pixel value of the corresponding pixel in the reference contrast image supplied by the contrast image generator 33. It will be observed that the threshold curve Ct includes a constant portion for which ⁇ C approximates to 1 for values of log C between 0 and 3. For values of log C in excess of 3, the curve Ct has a gradient of 0.87.
  • a control signal output is supplied to an output terminal 47 from the JND map circuit 37.
  • the control signals are supplied to the compression circuit 31 to control the compression of the television signal from the analogue to digital converter 10.
  • the signal compression can thereby be varied in a selective manner to achieve a better image fidelity for a fixed bit rate of encoding in the compression circuit 31 or better bit rates for a desired level of fidelity.
  • a conventional video encoder comprises an input buffer 70 to receive a digital video signal.
  • the buffer 70 applies sliced and reordered picture frames to a summer 71.
  • the summer 71 has an output connected to a discrete cosine transform circuit 72 in which a discrete cosine transform is performed on the output from the summer 71.
  • the transform circuit 72 is connected to a quantisation circuit 73, which feeds a variable length coder 74.
  • the variable length coder 74 supplies an output buffer 75.
  • the output from the buffer 75 is an MPEG compressed bit stream representing the digital video signal applied to the input buffer 70.
  • a reverse quantisation circuit 76 receives an input from the quantisation circuit 73.
  • the circuit 76 is connected to feed a reverse transform circuit 77, which performs a reverse discrete cosine transformation.
  • Each frame of the input digital video signal is decompressed by the circuits 76 and 77 and applied to a frame store 78.
  • a forward motion estimation circuit 79 derives forward motion estimation signals by comparing each frame in the store 78 with the frame at the output from the buffer 70.
  • a forward motion compensation circuit 80 receives the frame stored in the frame store 78 and the motion estimation signals from the circuit 79. The motion compensation circuit 80 applies motion compensation to the video signals passing through the summer 71.
  • the apparatus of Figure 7 is shown adapted to include a control circuit 82.
  • the control circuit 82 receives at input 84 the decompressed and motion compensated signal generated by the circuit 80.
  • the circuit 82 also receives at input 86 the picture information from the buffer 70 in respect of the next succeeding frame.
  • the circuit 82 has an output terminal 88 connected to the transform circuit 72.
  • Figure 8A shows further detail of the control circuit 82.
  • the input terminal 84 is connected to a first contrast image generator 85 and the input terminal 86 is connected to a second contrast image generator 87.
  • the two contrast image generators 85 and 87 correspond respectively to the two contrast image generators 33 and 36 of Figure 3.
  • the contrast image generators 85 and 87 generate contrast images in the same manner as the corresponding contrast image generators 33 and 36 of Figure 3.
  • the contrast images generated by the generators 85 and 87 are applied to a difference circuit 89, which corresponds to the difference circuit 35 in Figure 3.
  • the output from the difference circuit 89 is applied to a JND map circuit 90 corresponding to the map circuit 37 of Figure 3.
  • control circuit 82 operate in the same manner as the corresponding elements in the control circuit of Figure 3 to generate control signals at the output terminal 88.
  • the control signals are applied to control the transform circuit 72 such that DCT values are sent when the control signal exceeds the threshold Ct but are not sent when the control signal is below the threshold Ct.
  • FIG 9 a second adaptation of the apparatus of Figure 7 is shown.
  • the quantisation circuit 73 receives the output from the transform circuit 72 by way of a delay circuit 91.
  • the control circuit 82 has its input terminal 84 connected to a frame store 92, its input terminal 86 connected to the input buffer 70 and its output terminal 88 connected to the quantisation circuit 73 and the inverse quantisation circuit 76.
  • a quantisation circuit 94 receives the output from the transform circuit 72 and supplies a quantised signal to an inverse quantisation circuit 96.
  • the output from the inverse quantisation circuit is subject to an inverse DCT transformation in an inverse transform circuit 98 and is supplied to the frame store 92.
  • the control circuit 82 generates a control signal on the terminal 88 to vary the step size of the quantisation applied by the quantisation circuit 73.
  • the threshold Ct determines the quantisation parameters and hence the image fidelity of the MPEG bitstream supplied from the output buffer 75.
  • Each of the encoders 110 quantises and encodes the respective input digital signal for supply to a statistical multiplexer 112. Only two encoders are shown in Figure 10 but it will be appreciated by those skilled in the art that multiple encoders may be linked to a common statistical multiplexer 112.
  • the bit rate of each encoder is a function of the input digital signal and each encoder places a varying demand on the bit rate capacity of the statistical multiplexer 112.
  • the bit rate capacity of the multiplexer 112 is shared between the encoders 110a to 110n in a known manner.
  • Control circuits 114a to 114n are provided for controlling the encoders 110a to 110n.
  • Each of the control circuits 114a to 114n has the same configuration and function as the control circuit shown in Figure 3 and described above.
  • the control circuits 114a to 114n each control the quantisation of the respective encoder to achieve a higher bit rate for a given image fidelity.
  • the number of encoders 110, which share the bit rate capacity of the statistical multiplexer 112, may thus be increased.
  • the invention provides a method and apparatus which is able to identify areas in an image where visually noticeable differences occur between an original image and a reproduction of that image.
  • the performance of the system that provides the reproduction can be derived by means of the invention and adjustment made to the system to enable the requisite image fidelity to be achieved.
  • the degree of compression can be adjusted to reduce or eliminate the usually noticeable differences.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present invention relates to a method and apparatus for determining visually noticeable differences between two input images. A first contrast image is generated from the first input image and a second contrast image is generate from the second input image. A contrast difference image is then formed from the two contrast images. Pixels in the contrast difference image are selected which differ in intensity from corresponding pixels in the first contrast image by a threshold value which is referenced to the intensity of the corresponding pixels in the first contrast image. The selected pixels may be applied to control the performance of television signal compression circuits so as to control the image fidelity.

Description

Determining Visually Noticeable Differences Between Two Images.
The present invention relates to determining visually noticeable differences between two images. The invention has particular application to the encoding and later decoding of an image in such fields as television signal generation, compression and transmission. When encoding an original image for television transmission and display at a receiver, it is important to know how well the system for transmitting and receiving will perform in reproducing the original image. The performance of the system is related to the visually noticeable differences between the original image and the displayed image.
A number of models have already been proposed to predict whether a human observer will be able to discriminate between two images. One such visual discrimination model is known as the Sarnoff visual discrimination model. A visual discrimination model may take, as input, a pair of images and provide an output which is a map showing the probability, as a function of position on the images, that an observer would be able to detect differences between the images. The map is referred to as a JND map showing Just Noticeable Differences. This JND map can itself be presented as an image, with higher grey levels corresponding to higher probabilities of discrimination.
In an encoding algorithm to encode picture information using quantisation methods, the image fidelity should be directly proportional to the quantisation sampling density. This assumption may be erroneous with the result that encoding bits are squandered where they have little impact on the fidelity of reproduction of an image. A better method of controlling quantisation parameters is to base control on a JND map to obtain either more uniform image fidelity for a fixed bit rate of encoding or better bit rates for a desired level of fidelity.
It is an object of the present invention to improve the identification of noticeable differences between two images. According to the present invention there is now provided a method of identifying visually noticeable differences between a first input image and a second input image, the method comprising the steps of: generating a first, reference contrast image from the first input image; generating a second contrast image from the second input image; forming a contrast difference image from the first and second contrast images; and selecting, from the contrast difference image, each pixel which has an intensity exceeding a threshold determined by the intensity of the corresponding pixel in the reference contrast image.
Further according to the present invention, there is provided apparatus for identifying visually noticeable differences between a first input image and a second input image, the apparatus comprising: a generator to generate a first, reference contrast image from the first input image and a second contrast image from the second input image; a difference forming circuit to form a contrast difference image from the first and second contrast images; and a selector to select from the contrast difference image each pixel, which has an intensity exceeding a threshold, determined by the intensity of the corresponding pixel in the reference contrast image.
The selected pixels which exceed the threshold value are indicative of the visually noticeable differences between the first input image and the second input image. Where the second input image is a compressed form of the first input image, the selected pixels will indicate where noticeable differences occur between them as a result of the compression. The degree of compression can thus be increased in those areas of each image where visually noticeable differences do not appear.
The invention will now be described, by way of example, with reference to the accompanying drawings in which: Figure 1 shows in block diagram form a video signal transmission apparatus embodying the present invention;
Figure 2 shows a video signal frame transmitted by the apparatus of Figure 1 ;
Figure 3 shows a control circuit included in the apparatus of Figure 1 ;
Figure 4 shows a block of pixels that are processed by the circuit of Figure 3;
Figure 5 shows a block diagram of steps used in processing signals in the circuit of Figure 3 to provide a map of just noticeable differences between two image frames;
Figure 6 shows a graph of threshold values applicable to the process steps in Figure 5;
Figure 7 shows a conventional television signal encoder apparatus;
Figure 8 shows the apparatus of Figure 7 modified according to the invention to add a control circuit;
Figure 8A shows details of the control circuit added into the apparatus of Figure 8;
Figure 9 shows a further television signal encoder modified to incorporate the control circuit of Figure 8A; and
Figure 10 shows a statistical multiplexer apparatus incorporating a plurality of the control circuits of Figure 8A.
Figure 1 shows a video signal transmission apparatus including an analogue to digital converter 10 to receive an input analogue television signal 11 for conversion to a digital signal. The input digital signal is encoded and compressed in an encoder 12 and applied to a transmission circuit 13. The transmission circuit 13 broadcasts the compressed signal in conventional manner for reception by one or more receiver circuits 14. The receiver circuit 14 applies the received signal to a decoder 15 for decoding and supply to a display 16.
Figure 2 shows a frame 10 included in the digital signal output from the encoder 12. As is well understood in the digital television art, the digital television signal includes a sequence of frames 10. Each frame consists of a matrix of pixels divided into M x N blocks of pixels.
The processes of encoding, compression, transmission and display all introduce some degree of unwanted noise into the picture information displayed by the television display 16. At the display 16, the noise may be more noticeable in some picture blocks of the image than in others. A control circuit 30 is incorporated into the encoder 12 to control the compression of the television signal so as to reduce the effect of the unwanted noise. The control circuit 30 is shown in further detail in Figure 3.
In Figure 3, the digital television signal from the analogue to digital converter 10 is applied to a compression circuit 31 for compression and supply to an output terminal 38 connected to the transmission circuit 13. The digital television signal is also supplied to a contrast image generator 33. The generator 33 generates a contrast image from the pixels of each frame and applies the contrast image to a difference circuit 35.
The compressed signal at the output terminal 38 is supplied to a decompression circuit 34 where the signal is subject to decompression and the resulting decompressed signal is passed to a 1 - frame store 32. The output of the 1 - frame store 32 is applied to a second contrast image generator 36. The difference circuit 35 receives the contrast images from the two contrast image generators and produces a contrast difference image. The contrast difference image is mapped in a JND Oust noticeable difference) map circuit 37. The map circuit 37 applies an output signal to control the compression circuit 31.
The operation of the control circuit shown in Figure 3 will now be described with reference to Figures 2, 4, 5 and 6. Each frame of the digital television signal applied to the contrast image generator 33 and the compression circuit 37 includes the blocks of pixels already mentioned and shown in Figure 2. In the contrast image generator 33, each of the pixels is processed in relation to a 3 x 3 neighbourhood area as shown in Figure 4. Each pixel a,,j is included in a 3 x 3 area extending from the pixel a^^ to the pixel a,+ιιJ+ι The image generator 33 generates a contrast value c,,, for each pixel a according to the relation;
Cι,j = 100 * Lmax - Lmin Lmax + Lmin
Where Lmax is the maximum luminance value of the pixels in the 3 x 3 neighbourhood area and L min is the minimum luminance value of the pixels in the 3 x 3 neighbourhood area.
Referring to Figure 5, a signal frame I of the digital television signal is received by the contrast image generator 33 in step 50 at the same time that a signal frame II is received by the contrast image generator 36 in step 51. The signal frame I is converted in step 52 into a contrast image I as already described above with reference to Figure 3. The signal frame II is a reconstruction of the television signal frame which precedes the signal frame I. The signal frame II is derived by the decompression circuit 34 and supplied to the 1 - frame store 32. The signal frame II, supplied in step 51 , is converted in step 53 into a contrast image II by the contrast image generator 36. The contrast image generator 36 uses the same process as the contrast image generator 33 to generate its contrast image.
The difference circuit 35 compares the contrast value of each pixel in the contrast image I with the contrast value of each pixel in the contrast image II. The result is a contrast difference image in which the contrast image differences for each frame are mapped in comparison with the preceding frame in the television signal.
The contrast image I is applied, in step 54, to the JND map circuit 37. The contrast difference image is applied, in step 55, to the JND map circuit 37. The contrast image I is used as a reference for each pixel of the contrast difference image so as to select pixels from the contrast difference image in each frame as will be explained with reference to Figure 6.
In Figure 6 there is shown a threshold curve Ct plotted against a difference value Δc along one axis and a reference value C along the other axis. Each axis has a logarithmic scale.
The value Δc is the difference value of a pixel in the contrast difference image supplied by the difference circuit 35. The reference value C is the pixel value of the corresponding pixel in the reference contrast image supplied by the contrast image generator 33. It will be observed that the threshold curve Ct includes a constant portion for which ΔC approximates to 1 for values of log C between 0 and 3. For values of log C in excess of 3, the curve Ct has a gradient of 0.87.
For each pixel where the value of log ΔC exceeds the threshold Ct, a control signal output is supplied to an output terminal 47 from the JND map circuit 37. The control signals are supplied to the compression circuit 31 to control the compression of the television signal from the analogue to digital converter 10. The signal compression can thereby be varied in a selective manner to achieve a better image fidelity for a fixed bit rate of encoding in the compression circuit 31 or better bit rates for a desired level of fidelity.
Further details regarding the manner in which the compression may be selectively varied will now be described with reference to Figures 7 and 8. In Figure 7, a conventional video encoder comprises an input buffer 70 to receive a digital video signal. The buffer 70 applies sliced and reordered picture frames to a summer 71. The summer 71 has an output connected to a discrete cosine transform circuit 72 in which a discrete cosine transform is performed on the output from the summer 71. The transform circuit 72 is connected to a quantisation circuit 73, which feeds a variable length coder 74. The variable length coder 74 supplies an output buffer 75. The output from the buffer 75 is an MPEG compressed bit stream representing the digital video signal applied to the input buffer 70.
A reverse quantisation circuit 76 receives an input from the quantisation circuit 73. The circuit 76 is connected to feed a reverse transform circuit 77, which performs a reverse discrete cosine transformation. Each frame of the input digital video signal is decompressed by the circuits 76 and 77 and applied to a frame store 78. A forward motion estimation circuit 79 derives forward motion estimation signals by comparing each frame in the store 78 with the frame at the output from the buffer 70.
A forward motion compensation circuit 80 receives the frame stored in the frame store 78 and the motion estimation signals from the circuit 79. The motion compensation circuit 80 applies motion compensation to the video signals passing through the summer 71.
In Figure 8, the apparatus of Figure 7 is shown adapted to include a control circuit 82. The control circuit 82 receives at input 84 the decompressed and motion compensated signal generated by the circuit 80. The circuit 82 also receives at input 86 the picture information from the buffer 70 in respect of the next succeeding frame. The circuit 82 has an output terminal 88 connected to the transform circuit 72.
Figure 8A shows further detail of the control circuit 82. As seen in Figure 8A, the input terminal 84 is connected to a first contrast image generator 85 and the input terminal 86 is connected to a second contrast image generator 87. The two contrast image generators 85 and 87 correspond respectively to the two contrast image generators 33 and 36 of Figure 3. The contrast image generators 85 and 87 generate contrast images in the same manner as the corresponding contrast image generators 33 and 36 of Figure 3. The contrast images generated by the generators 85 and 87 are applied to a difference circuit 89, which corresponds to the difference circuit 35 in Figure 3. The output from the difference circuit 89 is applied to a JND map circuit 90 corresponding to the map circuit 37 of Figure 3.
The elements of the control circuit 82 operate in the same manner as the corresponding elements in the control circuit of Figure 3 to generate control signals at the output terminal 88. The control signals are applied to control the transform circuit 72 such that DCT values are sent when the control signal exceeds the threshold Ct but are not sent when the control signal is below the threshold Ct.
In Figure 9, a second adaptation of the apparatus of Figure 7 is shown. In this second adaptation, the quantisation circuit 73 receives the output from the transform circuit 72 by way of a delay circuit 91. The control circuit 82 has its input terminal 84 connected to a frame store 92, its input terminal 86 connected to the input buffer 70 and its output terminal 88 connected to the quantisation circuit 73 and the inverse quantisation circuit 76. A quantisation circuit 94 receives the output from the transform circuit 72 and supplies a quantised signal to an inverse quantisation circuit 96. The output from the inverse quantisation circuit is subject to an inverse DCT transformation in an inverse transform circuit 98 and is supplied to the frame store 92.
The control circuit 82 generates a control signal on the terminal 88 to vary the step size of the quantisation applied by the quantisation circuit 73. The threshold Ct determines the quantisation parameters and hence the image fidelity of the MPEG bitstream supplied from the output buffer 75.
In Figure 10, a series of encoders 110a to 11 On each receives a digital video signal at a respective input terminal 111a to 111 n. Each of the encoders 110 quantises and encodes the respective input digital signal for supply to a statistical multiplexer 112. Only two encoders are shown in Figure 10 but it will be appreciated by those skilled in the art that multiple encoders may be linked to a common statistical multiplexer 112. The bit rate of each encoder is a function of the input digital signal and each encoder places a varying demand on the bit rate capacity of the statistical multiplexer 112. The bit rate capacity of the multiplexer 112 is shared between the encoders 110a to 110n in a known manner.
Control circuits 114a to 114n are provided for controlling the encoders 110a to 110n. Each of the control circuits 114a to 114n has the same configuration and function as the control circuit shown in Figure 3 and described above. The control circuits 114a to 114n each control the quantisation of the respective encoder to achieve a higher bit rate for a given image fidelity. The number of encoders 110, which share the bit rate capacity of the statistical multiplexer 112, may thus be increased.
Whilst the invention has been described in relation to the control of television signals compressed by discrete cosine transform methods and by quantisation methods, it will be apparent that the invention may also be applied to the control of television signals compressed by other techniques such as wavelet compression.
It will be seen from the above description that the invention provides a method and apparatus which is able to identify areas in an image where visually noticeable differences occur between an original image and a reproduction of that image. The performance of the system that provides the reproduction can be derived by means of the invention and adjustment made to the system to enable the requisite image fidelity to be achieved. Thus, where the system compresses the original image, the degree of compression can be adjusted to reduce or eliminate the usually noticeable differences.

Claims

1. A method of identifying visually noticeable differences between a first input image and a second input image, the method comprising the steps of: generating a first, reference contrast image from the first input image, generating a second contast image from the second input image; forming a contrast difference image from the first and second contrast images; and selecting, from the contrast difference image, each pixel which has an intensity exceeding a threshold determined by the intensity of the corresponding pixel in the reference contrast image.
2. A method as claimed in claim 1 , wherein the first and second input images each consist of a matrix of pixels within a television signal frame.
3. A method as claimed claim 2, wherein the first and second contrast images are generated by processing each of the pixels in the respective frame to produce a contrast value c, for each pixel a,,, according to the relation;
Cij = 100 * Lmax - Lmin Lmax + Lmin
Where Lmax is the maximum luminance value of the pixels in the 3 x 3 neighbourhood area adjoining the pixel and Lmin is the minimum luminance value of the pixels in the 3 x 3 neighbourhood area.
4. A method as claimed in claim 1 , 2 or 3, including the further step of deriving the second input image from the first input image by first compressing and subsequently decompressing the first input image.
A method as claimed in claim 4, wherein the step of deriving the second input image from the first input image comprises compressing the first input image by a discrete cosine transformation step and decompressing the compressed first input image by an inverse discrete cosine transformation step.
A method as claimed in claim 4, wherein the step of deriving the second input image from the first input image comprises compressing the first input image by a digital quantisation step and decompressing the compressed first input image by an inverse digital quantisation step.
A method as claimed in claim 5 or 6, comprising the further step of controlling the degree of compression by reference to the pixels selected from the contrast difference image, so as to limit the number of visually noticeable differences between the first and second input images.
Apparatus for identifying visually noticeable differences between a first input image and a second input image, the apparatus comprising: a generator to generate a first, reference contrast image from the first input image and a second contrast image from the second input image; a difference forming circuit to form a contrast difference image from the first and second contrast images; and a selector to select from the contrast difference image each pixel, which has an intensity exceeding a threshold, determined by the intensity of the corresponding pixel in the reference contrast image.
Apparatus as claimed in claim 8, wherein the generator is adapted to generate the first and second contrast images from first and second input images each consisting of a matrix of pixels within a television signal frame.
10. Apparatus as claimed in claim 9, wherein the generator is adapted to generate the first and second contrast images by processing each of the pixels in the respective frame to produce a contrast value cy for each pixel ay according to the relation;
Cij = 100 * Lmax - Lmin Lmax + Lmin
Where Lmax is the maximum luminance value of the pixels in the 3 x 3 neighbourhood area adjoining the pixel and Lmin is the minimum luminance value of the pixels in the 3 x 3 neighbourhood area.
11. Apparatus as claimed in claim 8, 9 or 10, further including compression and decompression circuits adapted to derive the second input image from the first input image by first compressing and subsequently decompressing the first input image.
12. Apparatus as claimed in claim 11 , wherein the compression circuit comprises means to compress the first input image by discrete cosine transformation and the decompression circuit comprises means to decompress the compressed first input image by inverse discrete cosine transformation.
13. Apparatus as claimed in claim 11 , wherein the compression circuit comprises means to compress the first input image by digital quantisation and the decompression circuit comprises means to decompress the compressed first input image by inverse digital quantisation.
14. Apparatus as claimed in claim 11 or 12, further comprising control means to control the compression in the compression circuit in response to the pixels selected from the contrast difference image, so as to limit the number of visually noticeable differences between the first and second input images.
15. Signal encoding and multiplexing apparatus comprising a plurality of signal encoders for encoding and compressing a repetitive plurality of input digital television signals to produce a corresponding plurality of encoded, compressed television signals and a statistical multiplexer to multiplex the encoded television signals, wherein each of the signal encoders includes a control circuit to control the degree of compression applied by the signal encoder, the control circuit being as claimed in claim 14.
16. A method of identifying visually noticeable differences between a first input image, the method being substantially as herein before described with reference to Figure 1 , 3, 8, 9, or 10 of the accompanying drawings.
17. Apparatus for identifying visually noticeable differences between a first input image and a second input image, the apparatus being substantially as herein before described with reference to Figure 1 , 3, 8, 9, or 10 of the accompanying drawings.
PCT/GB1999/000487 1998-02-21 1999-02-17 Determining visually noticeable differences between two images WO1999043161A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU26299/99A AU2629999A (en) 1998-02-21 1999-02-17 Determining visually noticeable differences between two images

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GBGB9803580.1A GB9803580D0 (en) 1998-02-21 1998-02-21 Determining visually noticeable differences between two images
GB9803580.1 1998-02-21

Publications (1)

Publication Number Publication Date
WO1999043161A1 true WO1999043161A1 (en) 1999-08-26

Family

ID=10827300

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB1999/000487 WO1999043161A1 (en) 1998-02-21 1999-02-17 Determining visually noticeable differences between two images

Country Status (4)

Country Link
AU (1) AU2629999A (en)
GB (1) GB9803580D0 (en)
WO (1) WO1999043161A1 (en)
ZA (1) ZA991308B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1100278A3 (en) * 1999-11-11 2003-01-22 Tektronix, Inc. Realtime human vision system behavioral modeling
EP1289318A1 (en) * 2001-08-13 2003-03-05 Nokia Corporation Method and system for measuring perceptual distortion in images
EP1294199A1 (en) * 2001-09-18 2003-03-19 Tektronix, Inc. Temporal processing for realtime human vision system behaviour modeling
EP1677252A1 (en) * 2005-01-03 2006-07-05 Thomson Licensing Method and apparatus for calculating a lambda value controlling the coding mode in a video encoder.
WO2006104710A1 (en) * 2005-03-25 2006-10-05 Siemens Medical Solutions Usa, Inc. Unified visual measurement of blur and noise distortions in digital images
EP3764647A1 (en) * 2019-07-11 2021-01-13 Google LLC Compressing a frame based on errors of previous frames

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0526201A2 (en) * 1991-07-31 1993-02-03 Sony Corporation Video signal transmission apparatus and method
EP0555061A1 (en) * 1992-02-07 1993-08-11 Hudson Soft Co., Ltd. Method and apparatus for compressing and extending an image
EP0577085A2 (en) * 1992-06-30 1994-01-05 Eastman Kodak Company Method and apparatus for determining visually perceptible differences between images
US5329379A (en) * 1992-10-22 1994-07-12 International Business Machines Corporation System and method of measuring fidelity of decompressed video signals and images
WO1997037322A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Apparatus and method for optimizing encoding and performing automated steerable image compression using a perceptual metric
WO1997037324A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Apparatus for assessing the visibility of differences between two image sequences
WO1997037326A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Method and apparatus for decomposing an image stream into units of local contrast
WO1997039417A2 (en) * 1996-03-29 1997-10-23 Sarnoff Corporation Method and apparatus for training a neural network to use a fidelity metric
WO1997046019A1 (en) * 1996-05-29 1997-12-04 Nds Limited Method and apparatus for processing a digital signal

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0526201A2 (en) * 1991-07-31 1993-02-03 Sony Corporation Video signal transmission apparatus and method
EP0555061A1 (en) * 1992-02-07 1993-08-11 Hudson Soft Co., Ltd. Method and apparatus for compressing and extending an image
EP0577085A2 (en) * 1992-06-30 1994-01-05 Eastman Kodak Company Method and apparatus for determining visually perceptible differences between images
US5329379A (en) * 1992-10-22 1994-07-12 International Business Machines Corporation System and method of measuring fidelity of decompressed video signals and images
WO1997037322A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Apparatus and method for optimizing encoding and performing automated steerable image compression using a perceptual metric
WO1997037324A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Apparatus for assessing the visibility of differences between two image sequences
WO1997037326A1 (en) * 1996-03-29 1997-10-09 Sarnoff Corporation Method and apparatus for decomposing an image stream into units of local contrast
WO1997039417A2 (en) * 1996-03-29 1997-10-23 Sarnoff Corporation Method and apparatus for training a neural network to use a fidelity metric
WO1997046019A1 (en) * 1996-05-29 1997-12-04 Nds Limited Method and apparatus for processing a digital signal

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1100278A3 (en) * 1999-11-11 2003-01-22 Tektronix, Inc. Realtime human vision system behavioral modeling
US6678424B1 (en) 1999-11-11 2004-01-13 Tektronix, Inc. Real time human vision system behavioral modeling
EP1289318A1 (en) * 2001-08-13 2003-03-05 Nokia Corporation Method and system for measuring perceptual distortion in images
EP1294199A1 (en) * 2001-09-18 2003-03-19 Tektronix, Inc. Temporal processing for realtime human vision system behaviour modeling
US6941017B2 (en) 2001-09-18 2005-09-06 Tektronix, Inc. Temporal processing for realtime human vision system behavior modeling
EP1677252A1 (en) * 2005-01-03 2006-07-05 Thomson Licensing Method and apparatus for calculating a lambda value controlling the coding mode in a video encoder.
WO2006104710A1 (en) * 2005-03-25 2006-10-05 Siemens Medical Solutions Usa, Inc. Unified visual measurement of blur and noise distortions in digital images
US7715646B2 (en) 2005-03-25 2010-05-11 Siemens Medical Solutions Usa, Inc. Unified visual measurement of blur and noise distortions in digital images
EP3764647A1 (en) * 2019-07-11 2021-01-13 Google LLC Compressing a frame based on errors of previous frames
US11228786B2 (en) 2019-07-11 2022-01-18 Google Llc Compressing frame based on errors of previous frames

Also Published As

Publication number Publication date
ZA991308B (en) 1999-08-20
GB9803580D0 (en) 1998-04-15
AU2629999A (en) 1999-09-06

Similar Documents

Publication Publication Date Title
JP3442028B2 (en) Data decoding method and apparatus
JP3888597B2 (en) Motion compensation coding apparatus and motion compensation coding / decoding method
US7860159B2 (en) Quantization matrix for still and moving picture coding
JP3109854B2 (en) Image coding method and apparatus
US5333013A (en) Image encoding apparatus
US7082221B1 (en) Bandwidth determination for multiple layer digital video
US7472151B2 (en) System and method for accelerating arithmetic decoding of video data
US6912318B2 (en) Method and system for compressing motion image information
JP2891253B1 (en) Image compression processor
JPH0686261A (en) Statistical multiplexer for multichannel image compression system
JPH1093966A (en) Image coding device
EP0986264B1 (en) Determining visually noticeable differences between two images
JPH09200758A (en) Image encoder
US5793428A (en) Self-encoded deltas for digital video data transmission
WO1999043161A1 (en) Determining visually noticeable differences between two images
US6631161B1 (en) Method and system for compressing motion image information
KR20020026189A (en) Efficient video data access using fixed ratio compression
KR100220582B1 (en) Image encoder having adaptive coding
JP2843024B2 (en) Method and apparatus for selecting transform coefficient of transform coding system
EP1170956A2 (en) Method and system for compressing motion image information
US20090060363A1 (en) System and method for image compression
KR900004962B1 (en) Image transmission system
JP4212127B2 (en) Encoding device, decoding device, re-encoding device, encoding method, decoding method, and re-encoding method
JPH08130737A (en) Image data encoding / decoding device
JPH07131794A (en) Image data encoding method and apparatus

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AU BR CA CN JP NZ US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WA Withdrawal of international application