[go: up one dir, main page]

EP3893504A1 - Systèmes et procédés pour codecs de compression de données hybrides à faible complexité, presque sans perte et à débit fixe - Google Patents

Systèmes et procédés pour codecs de compression de données hybrides à faible complexité, presque sans perte et à débit fixe Download PDF

Info

Publication number
EP3893504A1
EP3893504A1 EP21164402.6A EP21164402A EP3893504A1 EP 3893504 A1 EP3893504 A1 EP 3893504A1 EP 21164402 A EP21164402 A EP 21164402A EP 3893504 A1 EP3893504 A1 EP 3893504A1
Authority
EP
European Patent Office
Prior art keywords
codec
lossy
frame
data
lossless
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP21164402.6A
Other languages
German (de)
English (en)
Other versions
EP3893504B1 (fr
Inventor
Vijayaraghavan Thirumalai
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Display Co Ltd
Original Assignee
Samsung Display Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Display Co Ltd filed Critical Samsung Display Co Ltd
Publication of EP3893504A1 publication Critical patent/EP3893504A1/fr
Application granted granted Critical
Publication of EP3893504B1 publication Critical patent/EP3893504B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/34—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
    • G09G3/36—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source using liquid crystals
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3002—Conversion to or from differential modulation
    • H03M7/3044—Conversion to or from differential modulation with several bits only, i.e. the difference between successive samples being coded by more than one bit, e.g. differential pulse code modulation [DPCM]
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3059—Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/60—General implementation details not specific to a particular type of compression
    • H03M7/6017—Methods or arrangements to increase the throughput
    • H03M7/6023—Parallelization
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/60—General implementation details not specific to a particular type of compression
    • H03M7/6064—Selection of Compressor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/18—Use of a frame buffer in a display terminal, inclusive of the display panel

Definitions

  • One or more aspects of embodiments of the present disclosure relate to systems and methods for low-complexity near lossless fixed-rate hybrid data compression codecs.
  • Data compression is growing in importance for transferring and storing data in variety of fields. For example, as display resolutions increase, the amount of data stored and transmitted for each frame of image data increases. Compressing this data reduces the amount of data that must be transmitted to the display panel and can reduce the amount of data that is stored in memory to be displayed on display panel.
  • aspects of embodiments of the present disclosure are directed to systems and methods for low-complexity near lossless fixed-rate hybrid data compression codecs.
  • Some aspects of embodiments of the present disclosure relate to a fixed-rate codec that: provides mathematically lossless performance on representative input data (e.g., data typically observed in an application in which the codec is designed to operate); meets a bitrate constraint on non-representative input data (e.g., input data having higher entropy than representative input data); and can operate without an additional internal memory buffer (e.g., without an additional line or compressed line buffer in the case where the input data is image data).
  • aspects of embodiments of the present disclosure relate to systems and methods for near lossless compression of image data for storage in a fixed size buffer and subsequent decompression of the stored compressed image data.
  • aspects of embodiments of the present disclosure also relate to the compression of image data for transmission over a data link and subsequent decompression of the compressed image data received over the data link.
  • a method for encoding data includes: receiving a frame of data including a plurality of subunits; initializing a remaining bit budget to a bit budget for the frame of data; setting a current codec to a lossless codec having a lossless codec bitrate; and sequentially encoding the subunits of the frame of data, including: encoding a first subunit of the subunits using the current codec, set to the lossless codec, to compute a first encoded subunit; subtracting a length of the first encoded subunit from the remaining bit budget; determining whether the remaining bit budget exceeds a first lossy bits required to encode a plurality of remaining subunits of the frame of data using a first lossy codec having a first lossy codec bitrate lower than the lossless codec bitrate; in response to determining that the remaining bit budget is less than or equal to the first lossy bits required, setting the current codec to the first lossy codec; and encoding a second subunit of
  • the bit budget may include W - lossyBpp - 1 extra margin bits, where W is the length of a longest binary codeword of a symbol in the lossless codec, and where lossyBpp is the first lossy codec bitrate.
  • the frame of data may be a frame of image data including a plurality of pixels.
  • the lossless codec may be a differential pulse coded modulation (DPCM) codec without quantization
  • the first lossy codec may be a DPCM codec with quantization
  • the bit budget may be set based on a size of a buffer in a display panel for storing the frame of image data for an overdrive algorithm.
  • Each of the subunits may be a pixel of the frame of image data.
  • Each of the subunits may be a line of pixels of the frame of image data.
  • Each of the subunits may be a block of pixels of the frame of image data.
  • the sequentially encoding the subunits of the frame of data may further include, in a state where the current codec is set to the first lossy codec: determining whether the remaining bit budget exceeds a second lossy bits required to encode the remaining subunits of the frame of data using a second lossy codec having a second lossy codec bitrate lower than the first lossy codec bitrate; in response to determining that the remaining bit budget is less than or equal to the second lossy bits required setting the current codec to the second lossy codec; and encoding a third subunit of the subunits using the current codec, set to the second lossy codec, to compute a third encoded subunit.
  • the bit budget may include: W - lossyBpp 1 - 1 extra margin bits, where W is the length of a longest binary codeword of a symbol in the lossless codec, and where lossyBpp 1 is the first lossy codec bitrate, and W 1 - lossyBpp 2 - 1 extra margin bits, where W 1 is the length of a longest binary codeword of a symbol in the first lossy codec, and where lossyBpp 2 is the second lossy codec bitrate.
  • the sequentially encoding the subunits of the frame of data may further include, in the state where the current codec is set to a codec other than the lossless codec: determining whether the remaining bit budget exceeds an expected number of bits to encode the remaining subunits of the frame of data using the lossless codec; in response to determining that the remaining bit budget exceeds the expected number of bits to encode the remaining subunits of the frame of data using the lossless codec, setting the current codec to the lossless codec; and encoding a fourth subunit
  • an encoder includes: one or more processing circuits configured to: receive a frame of data comprising a plurality of subunits; initialize a remaining bit budget to a bit budget for the frame of data; set a current codec to a lossless codec having a lossless codec bitrate; encode a first subunit of the plurality of subunits using the current codec, set to the lossless codec, to compute a first encoded subunit; subtract a length of the first encoded subunit from the remaining bit budget; determine whether the remaining bit budget exceeds a first lossy bits required to encode a plurality of remaining subunits of the frame of data using a first lossy codec having a first lossy codec bitrate lower than the lossless codec bitrate; in response to a determination that the remaining bit budget is less than or equal to first lossy bits required set, the current codec to the first lossy codec; and encode a second subunit of the subunits using the current codec, set to the first lossy
  • the bit budget may include W - lossyBpp - 1 extra margin bits, where W is the length of a longest binary codeword of a symbol in the lossless codec, and where lossyBpp is the first lossy codec bitrate.
  • the frame of data may be a frame of image data including a plurality of pixels.
  • the lossless codec may be a differential pulse coded modulation (DPCM) codec without quantization
  • the first lossy codec may be a DPCM codec with quantization
  • the bit budget may be set based on a size of a buffer in a display panel for storing the frame of image data for an overdrive algorithm.
  • Each of the subunits may be a pixel of the frame of image data.
  • Each of the subunits may be a line of pixels of the frame of image data.
  • Each of the subunits may be a block of pixels of the frame of image data.
  • the one or more processing circuits may be further configured to, in a state where the current codec is set to the first lossy codec: determine whether the remaining bit budget exceeds a second lossy bits required to encode the remaining subunits of the frame of data using a second lossy codec having a second lossy codec bitrate lower than the first lossy codec bitrate; in response to a determination that the remaining bit budget is less than or equal to the second lossy bits required, set the current codec to the second lossy codec; and encode a third subunit of the subunits using the current codec, set to the second lossy codec, to compute a third encoded subunit.
  • the bit budget may include: W - lossyBpp 1 - 1 extra margin bits, where W is the length of a longest binary codeword of a symbol in the lossless codec, and where lossyBpp 1 is the first lossy codec bitrate, and W 1 - lossyBpp 2 - 1 extra margin bits, where W 1 is the length of a longest binary codeword of a symbol in the first lossy codec, and where lossyBpp 2 is the second lossy codec bitrate.
  • the one or more processing circuits may be further configured to, in the state where the current codec is set to a codec other than the lossless codec: determine whether the remaining bit budget exceeds an expected number of bits to encode the remaining subunits of the frame of data using the lossless codec; in response to a determination that the remaining bit budget exceeds the expected number of bits to encode the remaining subunits of the frame of data using the lossless codec, set the current codec to the lossless codec; and encode a fourth subunit of the subunits using the current
  • Data compression generally relates to encoding information using fewer bits than its original representation.
  • Certain applications demand mathematically lossless compression-compression in which no information is lost and the original representation can be decoded from the encoded representation.
  • some applications may place limits on the size of the encoded representation (e.g., compressed data). For example, a fixed budget in terms of storage space or bitrate may be allotted for each unit or frame of data.
  • a fixed budget in terms of storage space or bitrate may be allotted for each unit or frame of data.
  • an application may store compressed data in a fixed size buffer or use the compressed data in a constant bitrate (CBR) codec.
  • CBR constant bitrate
  • some aspects of embodiments of the present disclosure relate to a fixed-rate mathematically lossless (or nearly lossless) compression system to meet these requirements.
  • embodiments of the present disclosure will be described herein in the context of nearly lossless compression of frames of image data.
  • embodiments of the present disclosure are not limited thereto, and may be applied to the compression of other types of digital data in contexts that are tolerant of some degree of lossy compression (e.g., text, audio, video, and image data).
  • FIG. 1 is a schematic block diagram of encoder and decoder hardware according to one embodiment of the present disclosure.
  • image content 10 is supplied an encoder 100, which encodes the supplied image content 10 to compute data 30 (or an encoded or compressed bitstream 30).
  • the image content 10 may correspond to an image frame (e.g., a bitmap representation of a single image) or a portion of a single image or frame (e.g., a single channel of a frame, a slice or partition of a frame, etc.
  • a slice is a spatially distinct region of an image frame and may be encoded separately from other slices in the same frame.).
  • the encoded data 30 may be transmitted to a decoder 200 over a physical medium 50.
  • the encoder 100 and the decoder 200 may be implemented using one or more processing circuits (e.g., a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and/or a graphics processing unit (GPU)).
  • processing circuits e.g., a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and/or a graphics processing unit (GPU)
  • the encoder 100 and the decoder 200 may be implemented using the same physical processing circuits (e.g., a processing circuit configured to implement both the encoder 100 and the decoder 200), or using different processing circuits (e.g., a first processing circuit configured to implement the encoder 100 and a second processing circuit configured to implement the decoder 200).
  • transmitting the encoded data 30 to a decoder 200 over a physical medium 50 includes storing the encoded data 30 on a physical storage medium 50 such as buffer memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), flash memory such as NAND flash memory, optical media such as compact discs, digital versatile discs, Blue-ray discs, magnetic media such as hard disk drives, or the like).
  • a physical storage medium 50 such as buffer memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), flash memory such as NAND flash memory, optical media such as compact discs, digital versatile discs, Blue-ray discs, magnetic media such as hard disk drives, or the like).
  • the encoder 100, the decoder 200, and the storage medium 50 are all contained within a same physical device, such as within a display driver integrated circuit (DDIC) of a display device.
  • DDIC display driver integrated circuit
  • the encoded (or compressed) bitstream 30 is transferred over a wired or wireless connection. In some embodiments discussed herein, a transfer over a wired connection is described, but the techniques described herein may also be applicable in implementations that involve transfer over a wireless connection. In some embodiments, the encoded (or compressed) bitstream 30 is transferred over a physical medium 50 (e.g., over a wire such as a data bus or a cable or other connector or over a wireless connection) to a display driver integrated circuit (DDIC) of a display device (e.g., an external monitor, a television, or an integrated display panel of a smartphone, tablet, or laptop computer).
  • DDIC display driver integrated circuit
  • the encoded data 30 is transferred over a computer network such as a local area network (LAN) or a wide area network (WAN) such as the Internet or other telecommunications network such as a cellular wireless data network and combinations of different types of networks.
  • a computer network such as a local area network (LAN) or a wide area network (WAN) such as the Internet or other telecommunications network such as a cellular wireless data network and combinations of different types of networks.
  • the decoder 200 is configured to decode the encoded (or compressed) bitstream 30 received from the physical medium 50 into a decompressed representation 18.
  • the decompressed representation 18 is the same (or substantially the same) as the original representation of the image content 10.
  • the decompressed representation 18 may be substantially similar (e.g., visually similar) to the original representation of the image content 10 such that the resulting data appears visually lossless.
  • the decompressed representation 18 may be considered substantially similar to the original representation of the image content 10 in response to a peak-signal-to-nose ratio (PSNR) metric associated with the representations, a structural similarity (SSIM) metric associated with the representations, another metric associated with the representations, or a combination thereof satisfying one or more thresholds.
  • PSNR peak-signal-to-nose ratio
  • SSIM structural similarity
  • the decoder 200 may then supply the decoded representation 18 to other hardware or software modules for further use by a hardware or software application 20.
  • the decoded representation 18 may be image data that is to be displayed on a display panel, such as by supplying driving waveforms to the display panel to control the luminance of individual pixels of the display panel in accordance with the decoded representation 18 of the image content 10.
  • a liquid-crystal display (LCD) panel may exhibit motion blur or other visual artifacts due to slow response times of liquid crystals within the LCD panel when transitioning between different states. This may be particularly evident when there is a large change in gray level from one frame to the next and when operating the LCD panel at high refresh rates (e.g., at refresh rates above 60 Hz such as 144 Hz).
  • Overdrive is one technique to shorten the transition time between liquid-crystal states by driving the liquid crystals at higher voltages.
  • the higher driving voltage can cause overshoot effects that persist into the period for displaying the following image frame (e.g., at a refresh rate of 60 Hz, each image frame is displayed for approximately 16.67 milliseconds, but the effects of overdrive may persist for longer than 16.67 milliseconds). Accordingly, information from the previously image frame is used to modify the driving signals for the display of the current frame, thereby reducing or removing visual artifacts due to overdrive.
  • FIG. 2A is a schematic block diagram illustrating a display panel system implementing overdrive.
  • a current image frame e.g., n-th image frame
  • a previous image frame e.g., an (n-1)-th image frame
  • an overdrive algorithm module 310 which is configured to generate a compensated current frame, which compensates for the effect of using overdrive when displaying the previous ((n-1)-th) frame.
  • the current frame or n-th frame is also supplied to a delay module 320, which delays the signal associated with the current frame such that it is supplied to the overdrive algorithm module 310 together with the next frame (or (n+1)-th frame, e.g., the frame after the current frame or n-th frame).
  • the delay 320 may delay the output of the data for approximately 16.67 milliseconds.
  • the delay 320 may include a memory for storing the current frame during the delay period.
  • storing the current frame in the memory of the delay 320 can raise various design tradeoffs.
  • the data supplied by the delay 320 should be identical to the data that was supplied to the delay 320.
  • One way to do so would be to save the current frame data, in its entirety, to the memory.
  • doing so may result in high power consumption and/or high input/output bandwidth requirements in order to write and read all of the image frame data within one frame period (e.g., about 16.67 milliseconds).
  • a large amount of memory may significantly increase the hardware costs associated with a display device.
  • FIG. 2B is a schematic block diagram illustrating an application of an encoder and a decoder according to embodiments of the present disclosure in a display panel system for implementing overdrive.
  • an encoder 100 and a decoder 200 may be used with a storage medium 50 to implement a delay 320.
  • the encoder 100 encodes (or compresses) the current frame (or n-th frame), and stores or buffers the compressed n-th frame in the storage medium 50 for approximately the period of one frame.
  • the decoder 200 then reconstructs the n-th frame from the stored data and supplies the reconstructed n-th frame to the overdrive algorithm module 310, which combines the reconstructed n-th frame with the (n+1)-th frame to compute a compensated (n+1)-th frame.
  • Using the encoder 100 to compress the current frame can reduce the size of the data (e.g., number of bits) to be stored in the storage medium 50 during the delay period (e.g., for 16.67 milliseconds in the case of a refresh rate of 60 Hz).
  • This data compression can therefore accelerate processing and reduce power consumption by reducing the amount of data to be written and read, and also reduces the cost of the device by reducing the amount of physical memory that is allocated within the delay 320 for storing the image frame.
  • the encoder 100 and the decoder 200 implement fast and simple codecs, the encoding and decoding of the image frames can be performed at a lower cost (e.g., in power and hardware) than storing the uncompressed image frame.
  • the codec applied to compress the image frames is a fixed-rate codec-e.g., the encoded data 30 is constrained to a bit budget set by the size of the storage medium 50, and the codec cannot generate a frame of encoded data 30 that is larger than the bit budget.
  • good reconstruction quality and/or mathematically lossless reconstruction based on the previous frame results in more accurate overdrive compensation.
  • an encoder may encode an input frame of data into an encoded representation that is larger than the bitrate constraint (e.g., an encoded representation that is larger than the bit budget for the codec).
  • some aspects of embodiments of the present disclosure are directed to a fixed-rate codec that provides mathematically lossless performance on representative (e.g., typical) inputs, while also meeting the bitrate constraint when provided with non-representative (e.g., atypical) inputs, and that can also do so with reduced or minimized memory usage (e.g., memory buffers in the encoder).
  • representative e.g., typical
  • non-representative e.g., atypical
  • some aspects of embodiments of the present disclosure relate to applying a lossless (or near-lossless) codec to encode frames of input data and falling back to a lossy codec when the bit budget associated with the fixed-rate codec will be exceeded.
  • One approach that uses both a lossless codec and a lossy codec is to first apply the lossless codec on the entire frame of data. If the losslessly encoded data fit within the bit budget, then that losslessly encoded data would be output by the codec. However, in circumstances where the lossless codec generates encoded data that exceeds the bit budget, then the lossy codec is applied to the frame of input data to generate lossily encoded data.
  • the frame of input data must be buffered (e.g., so that the lossy codec can read the buffered copy of the input data in circumstances where the output of the lossless codec exceeded the bit budget), and in embodiments where the different codecs are applied in parallel, a compressed frame buffer may be needed to store the lossily encoded output of the lossy codec, to be used instead of the output of the lossless codec in circumstances where the lossless codec exceeds the bit budget.
  • One approach to reduce the amount of buffering is to apply a subunit-based approach in which subunits of a frame of data are encoded one at a time.
  • the subunit may be a line (or row) of the image data.
  • the serial or parallel approaches to applying lossless and lossy codecs may be applied to each subunit of the frame of data instead of the whole frame of data, where each subunit may be allocated by an equal portion of the overall bit budget for the frame of data. This allows the size of the input buffer (or the compressed buffer) to be reduced to the size of the subunit (or encoded subunit).
  • FIG. 3 is a schematic block diagram illustrating an encoder 100a implementing a hybrid data compression codec according to one embodiment of the present disclosure.
  • the encoder 100a shown in the embodiment of FIG. 3 is one embodiment of the encoder 100 (e.g., as shown in FIG.
  • the controller 190 configured to receive the encoded data from a codec (e.g., the lossless codec 170 or the lossy codec 180) and transmit the encoded data for use by an application.
  • the controller 190 stores the encoded data 30 in a storage medium 50 accessible by the application.
  • Embodiments of the present disclosure are agnostic to the particular lossless codec 170 and lossy codec 180 used.
  • the lossless codec 170 guarantees mathematically lossless performance-JPEG lossless or in general predictive coding schemes (such as differential pulse coded modulation or DPCM) are non-limiting examples of codecs that can provide such mathematically lossless performance.
  • the lossless codec 170 omits a quantization operation and/or omits a transform operation such as discrete cosine transform (DCT) or wavelet transform.
  • DCT discrete cosine transform
  • the lossy codec 180 guarantees fixed-rate behavior-predictive coding schemes, with or without transform, and with or without quantization, are non-limiting examples of codecs that can provide fixed-rate behavior.
  • the lossy codec 180 can be pixel-based or can be block-based (e.g., encoding blocks of multiple neighboring pixels or samples).
  • a hybrid data compression codec compresses input data such that an encoded representation of the input data fits within a target bit budget (TargetBitBudget).
  • the bit budget may be the total number of pixels in the input image ( N pixels ) multiplied by the target bits-per-pixel ( targetBpp ) , or N pixels ⁇ targetBpp.
  • the targetBpp and the lossless codec 170 are designed such that representative data (e.g., representative images) can be encoded by the lossless codec 170 within the bit budget ( targetBpp may be based on a source bit depth of the image content 10 and a desired compression ratio.
  • representative images may correspond to screenshots of typical user interactions such as productivity applications, web browsing, video playback, and games, whereas non-representative images may correspond to atypical images having high entropy, such as random noise (e.g., video "snow").
  • the lossy codec 180 has a bitrate ( lossyBpp ) lower than the target bitrate ( lossyBpp ⁇ targetBpp and/or lossyBpp ⁇ losslessBpp ) to serve as a fallback for circumstances where the lossless codec 170 may not be able to encode the entire input frame within the bit budget.
  • the bitrate of the lossy codec 180 may be referred to herein as the lossy codec bitrate. Accordingly, some aspects embodiments of the present disclosure relate to encoding some portions of the input frame of data using a lossless codec 170 and encoding other portions of the input frame of data using a lossy codec 180 in order to encode the entire frame within the bit budget.
  • FIG. 4 is a schematic depiction of a process for determining whether to switch from a higher bitrate codec to a lower bitrate codec in accordance with one aspect of embodiments of the present disclosure.
  • a first portion 410 of the frame 400 is already encoded (or previously encoded) by the encoder 100a, and a second portion 420 of the frame 400 is yet to be encoded ( RemainingPixels ) .
  • a current pixel 430 (or current value or symbol) is the pixel being encoded by the encoder 100a.
  • rate controller 110 updates its state and determines whether the current pixel 430 and the second portion 420 should be encoded using the lossless codec 170 or the lossy codec 180.
  • An input frame of data 10 may be divided into a plurality of sub-units 11.
  • FIG. 3 depicts the compression of a frame of image data 10.
  • the input frame of image data 10 includes a plurality of pixels arranged in a grid or two-dimensional array, where the grid of pixels may be divided into sub-units 11 such as rows (or lines), blocks (e.g., groups of adjacent pixels or groups of adjacent values) or individual pixels (e.g., individual values).
  • the frame 10 of image data may correspond to, for example, a single channel of image data, such as, in the case of RGB data, a red channel, a green channel, or a blue channel, and each value may correspond to a gray level of the channel (as another example, the image data may be YUV data and the channels correspond to a Y channel, a U channel, and a V channel).
  • FIG. 5A is a flowchart of a method 500 for encoding image data using hybrid data compression according to one embodiment of the present disclosure that may be performed by the encoder 100a of FIG. 3 .
  • a fixed-rate codec may be desired (e.g., where constant bitrate codecs are used to meet quality of service constraints based on available bandwidth).
  • FIGS. 3 , 4 , 5A , and 5B will be described below in the context of encoding image data.
  • the rate controller 110 of the encoder 100a initializes a current codec setting to identify the lossless codec 170, initializes a remaining bit budget (RemBits) value to a target bit budget for the input frame of data 10, and initializes a lossy bits required (LossyBitsReq) value to the product of the total number of pixels (or samples) in the input frame of data 10 and the lossy bitrate ( lossyBpp ) of the lossy codec 180.
  • the rate controller 110 also initializes a number of coded pixels ( codedPixels ) (e.g., number of coded symbols of the frame of data 10) to zero.
  • the encoder 100a determines, in operation 530, if it has finished encoding the current frame of data 10. For example, the encoder 100a may compare the number of coded pixels to the total number of pixels ( totalPixels ) (or the number of coded symbols to the total number of symbols) in the input frame of data 10. If the number of coded pixels is less than the total pixels, then there are more pixels to encode, otherwise the encoding process for the frame is done (and the encoder 100a proceeds to operation 510 to encode the next frame of data). As noted above, while FIG.
  • the rate controller may, for example, track the progress of the encoding process based on sub-units, e.g., codedSubunits and totalSubunits.
  • the rate controller 110 proceeds to operation 580.
  • the controller 190 may store the data in a storage medium 50 or transmit the data (e.g., supply the data to a display controller or a network controller).
  • the encoder 100a then returns to operation 530 to determine whether there are more pixels to encode and to continue encoding or to finish the encoding of the current frame accordingly.
  • the lossless codec 170 is used as long as there are more than enough remaining bits ( RemBits ) in the bit budget (or enough remaining bit budget) to perform a lossy compression on the remaining pixels, and when the rate controller 110 determines that the remaining bits in the bit budget are less than or equal to the number of bits required to perform a lossy compression on the remaining pixels (LossyBitsReq), the rate controller 110 switches to lossy compression for the remaining pixels.
  • a different threshold number of remaining bits in the bit budget may be set for switching between lossless and lossy compression.
  • FIG. 5B is a flowchart depicting a method for updating a state of a rate controller according to one embodiment of the present disclosure.
  • the rate controller 110 determines whether the remaining values (e.g., pixels) in the frame of data 10 are expected to consume less than the remaining bit budget using the lossless codec 170 ( RemBits > LossyBitsReq ) and whether the current codec is not the lossy codec 180 ( current ⁇ lossy ) .
  • the rate controller 110 In response to the rate controller 110 determining that both conditions are met, the rate controller 110 maintains the current codec as the lossless codec 170 in operation 575. Otherwise (e.g., when the remaining bit budget (RemBits) does not exceed the bits required to encode the remaining data of the input frame using the lossy codec), the rate controller 110 sets the current codec to the lossy codec 180 in operation 577 (without allowing return to the lossless codec, such that the remaining pixels are encoded using the lossy codec 180). After updating the rate control in operation 570 (e.g., in operations 575 or 577), the rate controller 110 proceeds to operation 580, as shown in FIG. 5A .
  • RemBits remaining bit budget
  • LossyBitsRequired ( RemainingPixels + 1) ⁇ lossyBpp, because the current value (or current pixel) is encoded using the lossy codec 180 (not the lossless codec 170) and therefore extra margin bits ( ExtraBits ) are further included to account for the potential difference in the length of the encoded current pixel using the lossless codec and using the lossy codec.
  • the encoder 100a may be configured to reserve bits in the buffer for the margin bits or may be configured to expand the buffer based on the margin bits.
  • the extra margin bits are included by initializing, in operation 510, the remaining bit budget (RemBits) to the target bit budget (TargetBitBudget) and adding the extra bits to the buffer in the memory (e.g., storage medium 50), such that the total size of the buffer in the memory is TargetBitBudget + ExtraBits (e.g., such that the data is compressed to a size less than or equal to TargetBitBudget + ExtraBits ) .
  • ExtraBits is aligned or rounded up to the nearest 8-bit or 16-bit boundary.
  • the number of margin bits may be rounded up to the nearest multiple of 8-bits or 16-bits (e.g., in accordance with the word size of the architecture of the storage medium), or rounded up in accordance with other physical constraints of the physical medium (e.g., payload size of a data transfer protocol or page size of a memory module).
  • Implementing the rate controller includes the use of two counters (one for each of RemBits and LossyBitsReq ) and one comparator for comparing the values of the two counters.
  • the comparator and counters may be implemented in hardware or software. According to some embodiments of the present disclosure, no line buffer (or buffer for storing some other subunit of the frame) or frame buffer is required. Accordingly, some aspects of embodiments of the present disclosure relate to a hybrid encoder having a low hardware cost, where the rate controller 110 is implemented using only two counters and one comparator.
  • the rate controller 110 evaluates whether to switch between codecs on a pixel-by-pixel (or sample-by-sample) basis (or pixel level of granularity).
  • a pixel-by-pixel (or sample-by-sample) basis or pixel level of granularity
  • embodiments of the present disclosure are not limited thereto.
  • evaluating whether to switch between codecs may be made on a row-by-row (or line-by-line) basis or a block-by-block basis (or block level of granularity, where a block represents a group of adjacent pixels or samples).
  • ExtraBits the appropriate number of extra margin bits
  • the number of margin bits may be rounded up to the nearest multiple of 8-bits or 16-bits (e.g., in accordance with the word size of the architecture of the storage medium), or rounded up in accordance with other physical constraints of the physical medium (e.g., payload size of a data transfer protocol or page size of a memory module).
  • the decoder 200 further includes a corresponding rate controller that implements substantially the same method described above with respect to FIG. 5B to determine when the encoder 100a would have switched between the different codecs and to automatically switch between a lossless decoder and a lossy decoder according to the determination made by the decoder rate controller.
  • some aspects of embodiments of the present disclosure relate to a hybrid encoder that initially encodes a frame of data using a lossless encoder and then switches to a lossy encoder (without the possibility of returning to the lossless encoder in the encoding of the frame of data).
  • FIGS. 6 , 7A , and 7B depict an embodiment in which a lossless codec and two lossy codecs are used in a hybrid data compression codec.
  • embodiments of the present disclosure are not limited thereto and may also include embodiments that use more than two lossy codecs (e.g., three or more lossy codecs) in conjunction with the lossless codec.
  • FIG. 6 is a schematic block diagram illustrating an encoder implementing hybrid data compression using multiple lossy codecs according to one embodiment of the present disclosure.
  • FIG. 6 is similar to FIG. 3 , and like reference numerals used in describing FIG. 6 are used to refer to like parts described with respect to FIG. 3 .
  • the encoder 100b shown in the embodiment of FIG. 6 is another embodiment of the encoder 100 (e.g., shown in FIG. 1 ) and includes a rate controller 110, a codec selector 120 that directs input data 10 to a currently selected codec, a lossless codec 170, a first lossy codec 181, and a second lossy codec 182.
  • the lossless codec 170 is configured to losslessly encode the input data 10, and the first and second lossy codecs 181 and 182 are configured to lossily encode the input data 10 at different bitrates.
  • the encoder 100b further includes a controller 190 configured to receive the encoded data from a codec (e.g., the lossless codec 170, the first lossy codec 181, or the second lossy codec 182) and transmit the data for use by an application, such as for storage on a storage medium 50.
  • a codec e.g., the lossless codec 170, the first lossy codec 181, or the second lossy codec 182
  • the first lossy codec 181 has a bitrate ( lossyBpp 1 ) that is lower than the bitrate ( losslessBpp or targetBpp ) of the lossless codec 170 ( lossyBpp 1 ⁇ losslessBpp ) .
  • the second lossy codec 182 has a bitrate ( lossyBpp 2 ) that is lower than the bitrate of the first lossy codec 181, such that lossyBpp 2 ⁇ lossyBpp 1 ⁇ targetBpp, so that the magnitude of the error (e.g., loss of accuracy in the reconstructed or decompressed data) increases with progression through the various lossy encoders.
  • FIG. 7A is a flowchart of a method 700 for encoding image data using hybrid data compression with multiple lossy codecs according to one embodiment of the present disclosure that may be performed by the encoder 100b depicted in FIG. 6 .
  • FIG. 7B is a flowchart depicting a method for updating a state of a rate controller of a hybrid data compression codec using multiple lossy codecs according to one embodiment of the present disclosure.
  • FIGS. 7A and 7B are similar to FIGS. 5A and 5B , respectively, and like reference numerals used in describing FIGS. 7A and 7B are used to refer to like operations described with respect to FIGS. 5A and 5B . Accordingly, the description of various similar parts and/or operations will not be repeated in detail herein.
  • the rate controller 110 of the encoder 100b initializes a current codec setting to identify a lossless codec 170, initializes a remaining bit budget (RemBits) value to a target bit budget for the frame, and initializes a lossy bits required (LossyBitsReq) value to the product of the total number of pixels (or samples) in the input frame of data and the bitrate ( lossyBpp 1 ) of a first lossy codec 181.
  • RemBits remaining bit budget
  • LosyBitsReq lossy bits required
  • the rate controller 110 also initializes a number of coded pixels ( codedPixels ) (e.g., number of coded symbols of the frame of data) to zero.
  • codedPixels e.g., number of coded symbols of the frame of data
  • the encoder 100b determines, in operation 730, if it has finished encoding the current frame of data 10, in a manner similar to that described above with respect to operation 530 of FIG. 5 .
  • the encoder 100b determines which codec is set as the current codec (e.g., the lossless codec 170, the first lossy codec 181, or the second lossy codec 182). In operation 750, the encoder applies the lossless codec 170, the first lossy codec 181, or the second lossy codec 182 based on the codec identified by the current codec state. In operation 760, the encoder 100b determines whether the current codec is the lowest bitrate codec (e.g., the second lossy codec 182 in the particular example described with respect to FIG. 6 ). If the current codec is not the lowest bitrate codec, then in operation 770, the encoder 100b updates the rate controller 110 to determine whether to change the current codec, as described in more detail below with respect to FIG. 7B .
  • the encoder 100b updates the rate controller 110 to determine whether to change the current codec, as described in more detail below with respect to FIG. 7B .
  • the controller 190 may store the data in a storage medium 50 or transmit the data internally (e.g., store the data in a location in memory associated with the controller).
  • the encoder 100b then returns to operation 730 to determine if there are more pixels to encode and to continue encoding or to finish the encoding of the current frame.
  • updating the rate controller 110 to determine whether to change from the current codec (e.g., the lossless codec 170 or the first lossy codec 181) to the next lower bitrate codec (e.g., the first lossy codec 181 or the second lossy codec 182, respectively) is performed in a substantially similar manner to the decision to switch from the lossless codec 170 to the lossy codec 180 as described above with respect to FIG. 5B .
  • the nextLossyCodec is, similarly, the next lossy codec that has a lower bitrate than the bitrate of the current codec (e.g., the codec having a bitrate that is lower than, and closest to, the bitrate of the current codec).
  • the lowest bitrate codec e.g., the codec among the plurality of lossy codecs of the hybrid encoder 100b having the lowest bitrate
  • the rate controller 110 determines if the number of remaining bits (RemBits) in the bit budget is less than or equal to the lossy bits required (LossyBitsReq) (e.g., determines whether RemBits ⁇ LossyBitsReq). If not, then the current codec (current) is maintained in its current state (represented in FIG. 7B as operation 775, although, in some embodiments, no action is taken by the rate controller 110 to maintain the state of the current codec).
  • RemBits the number of remaining bits in the bit budget is less than or equal to the lossy bits required (LossyBitsReq) (e.g., determines whether RemBits ⁇ LossyBitsReq). If not, then the current codec (current) is maintained in its current state (represented in FIG. 7B as operation 775, although, in some embodiments, no action is taken by the rate controller 110 to maintain the state of the current codec).
  • nextLossyCodec nextLossyCodec
  • the encoder 100b further reserves enough extra margin bits to perform the lossy compression using the plurality of fallback codecs.
  • These margin bits may be reserved in a buffer of the memory (e.g., the storage medium 50) or the buffer may be expanded based on the margin bits.
  • the encoder 100b may be configured to reserve bits in the buffer for the margin bits or may be configured to expand the buffer based on the margin bits.
  • the extra margin bits are included by initializing, in operation 710, the remaining bit budget (RemBits) to the target bit budget (TargetBitBudget) and adding the extra bits to the buffer in the memory (e.g., storage medium 50), such that the total size of the buffer in the memory is TargetBitBudget + ExtraBits.
  • the extra margin bits ExtraBits 1 allocated for transitioning from the lossless codec 170 to the first lossy codec 181 is similar to the extra margin bits described above with respect to FIGS.
  • ExtraBits 1 W - lossyBpp 1 - 1, where W represents the length of the longest binary codeword of a symbol in the lossless codec 170 .
  • ExtraBits 2 W 1 - lossyBpp 2 - 1, where W 1 represents the length of the longest binary codeword of a symbol in the first lossy codec 181.
  • the encoder 100b is configured to switch from a lossless codec to a lossy codec without the possibility of returning to the lossless codec for a given frame. Similarly, in some embodiments of the present disclosure, the encoder 100b is configured to switch from a higher bitrate codec to a lower bitrate codec without the possibility of returning to encoding using a higher bitrate coder later on in the encoding of the frame of data 10 for a given frame. In these embodiments, the encoder 100b is configured to switch back to the lossless codec for a subsequent frame. However, embodiments of the present disclosure are not limited thereto.
  • some embodiments of the present disclosure relate to an encoder that is configured to allow a return to a higher bitrate codec from a lower bitrate codec (e.g., from the first lossy codec 181 to the lossless codec 170) upon reaching one or more positions within the frame of data 10.
  • FIG. 8 is a schematic depiction of a process for determining whether to switch between different codecs (e.g., to a higher bitrate codec) in accordance with one aspect of embodiments of the present disclosure.
  • FIG. 8 will be described in the context of the encoder 100b having the lossless codec 170, the first lossy codec 181, and the second lossy codec 182, as shown in the embodiment of FIG. 6 .
  • FIG. 9A is a flowchart of a method 900 for encoding image data using hybrid data compression with multiple lossy codecs with return to lossless according to one embodiment of the present disclosure.
  • FIG. 9A is substantially similar to FIG. 7A , where like numerals refer to like operations of FIG. 7A , but differs in that FIG. 9A does not include a decision operation (e.g., corresponding to operation 760 of FIG. 7A ) based on whether the current codec is the lowest bitrate codec, and instead operation 970 to update the rate control is performed per subunit (e.g., per pixel in the embodiment shown in FIG. 9A ) of the frame of data.
  • a decision operation e.g., corresponding to operation 760 of FIG. 7A
  • operation 970 to update the rate control is performed per subunit (e.g., per pixel in the embodiment shown in FIG. 9A ) of the frame of data.
  • a first portion 810 of a frame of data 800 (e.g., image data) is encoded using the lossless codec 170. While encoding the frame of data, the rate controller 110 may determine that the conditions are met for switching to the first lossy codec 181, such that a second portion 820 of the frame of data 800 is encoded using the first lossy codec 181.
  • FIG. 9B is a flowchart depicting a method for updating a state of a rate controller of a hybrid data compression codec using multiple lossy codecs with return to lossless in operation 970 according to one embodiment of the present disclosure. Additional operations depicted in FIG. 9B will be described in more detail below.
  • the rate controller 110 determines whether to switch back to a higher bitrate codec (e.g., the lossless codec 170). Accordingly, as shown in FIG. 9B , after first updating the state of the rate controller 110 in operation 971 based on the bits spent ( bitsSpent ) and the bitrate of the next lossy codec ( nextLossyBpp ) , in operation 972, the rate controller 110 determines whether it is currently at a position in the frame of data where return is permitted.
  • a higher bitrate codec e.g., the lossless codec 170
  • the rate controller 110 proceeds with updating the selection of the current codec (current) in operation 973 in a manner similar to that described above with respect to FIG. 7B .
  • the one or more particular positions may be predetermined positions, such as a beginning of each line, a beginning of each even line, a beginning of each odd line, a beginning of every fourth line, etc.
  • the rate controller 110 determines whether the current codec is the lossless codec 170 ( lossless ) . If so, then there is no need to consider whether to return to the lossless codec 170, and the rate controller 110 continues with operation 973. If the current codec is a lossy codec, then in operation 976 the rate controller 110 determines whether it should return to a higher bitrate codec (e.g., the lossless codec 170).
  • the determination of whether to return to the lossless codec is made based on the remaining bit budget (RemBits) and whether the remaining bit budget is expected to be sufficient to encode the remaining portion of the frame of data at the bitrate of the higher bitrate encoder (e.g., when RemBits ⁇ LosslessBitsExpected, where LosslessBitsExpected may be (totalPixels - codedPixels) ⁇ targetBpp).
  • the rate controller 110 sets the current codec to the lossless codec 170. If not, then the rate controller 110 continues with updating the codec in accordance with operation 973. In the embodiment shown in FIG. 8 , at position 825, the rate controller 110 determines that the current codec can return to the lossless codec 170.
  • the rate controller 110 may determine whether the current codec can return to a lossy codec with a higher bitrate, based on whether the remaining bit budget is sufficient to encode the data using the higher bitrate lossy codec, but not sufficient to encode the data using, for example, the lossless codec 170.
  • the rate controller 110 may determine whether the current codec can return to a higher bitrate lossy codec (e.g., from the second lossy codec 182 to the first lossy codec 181), such as by determining whether RemBits > (totalPixels - codedPixels) ⁇ LossyBpp 1 .
  • the rate controller 110 determines again that the first lossy codec 181 should be used, and controls the encoder to encode a fourth portion 840 of the frame of data 800 using the first lossy codec 181, and further on in the encoding, the rate controller 110 may determine that the second lossy codec 182 should be used to continue encoding, such that a fifth portion 850 of the frame of data 800 is encoded using the second lossy codec 182.
  • some aspects of embodiments of the present disclosure relate to allowing transitions to a higher bitrate codec at one or more positions within the frame of data.
  • extra margin bits are allocated to the bit budget to accommodate each transition from a higher bitrate codec to a lower bitrate codec.
  • extra margin bits are further allocated for each possible additional transition.
  • the rate controller 110 can transition the encoder 100b from the lossless codec 170 to the first lossy codec 181 and from the first lossy codec 181 to the second lossy codec 182.
  • DPCM differential pulse coded modulation
  • each pixel corresponds to, for example: in the red-green-blue (RGB) color space, these components may include a red component, a green component, and a blue component; in the YCbCr color space, these components may include a luma (Y) component, a chrominance blue (Cb) component, and a chrominance red (Cr) component; and in the YCoCg color space, these components may include a luma (Y) component, a chrominance green (Cg) component, and a chrominance orange (Co) component.
  • RGB red-green-blue
  • the same predictor is used for the lossless and lossy codecs.
  • a previous reconstructed sample X ⁇ ( i - 1,j) (in the same line or row) is considered as the predictor for the current sample.
  • the first sample in each line does not contain a predictor.
  • the residue is between [-7, 7] (e.g., the smallest value is 0 and the largest value is 7, and therefore the largest magnitude of the difference is 7, and either the residue or the current value could be the larger value). Accordingly, encoding this residue directly would require one additional bit for the sign value of the residue.
  • Table 1 shows the offset and qtlen values for the lossless codec and first and second lossy codecs according to this example embodiment: Table 1 Lossless codec Lossy 1.8:1 Lossy 3:1 offset 0 2 3 qtlen 8 12 14
  • modulo subtraction is used to undo the effect of the modulo addition.
  • Tables 2 and 3 provide examples of quantization for ranges of residues for the first lossy codec and the second lossy codec, respectively: Table 2 - Quantization - Lossy 1.8:1 Range of residue Quantized value (Encoder) Dequantized value [-2, 1] 0 0 [2, 5] 1 4 [6, 9] 2 7 Table 3 - Quantization - Lossy 3:1 Range of residue Quantized value (Encoder) Dequantized value [-3, 3] 0 0 [4, 10] 1 7
  • Tables 4, 5, and 6 illustrate examples of codebooks mapping quantized values to bit representations thereof for the lossless codec, first lossy codec (lossy 1.8:1), and second lossy codec (lossy 3:1), respectively: Table 4 - Codebook - Lossless Quantized value Bit representation 0 0 1 101 2 1100 3 1110 4 11110 5 11111 6 1101 7 100 Table 5 - Codebook - Lossy 1.8:1 Quantized value Bit representation 0 0 1 10 2 11 Table 6 - Codebook - Lossy 3:1 Quantized value Bit representation 0 0 1 1
  • embodiments of the present disclosure are not limited thereto and may be applied to image data having different bit depths (e.g., 8 bits per channel), different numbers of lossy codecs (e.g., one lossy codec or more than two lossy codecs), and may also be applied to digital data other than image data.
  • aspects of embodiments of the present disclosure relate to systems and methods for encoding frames of data using a hybrid encoder configured to provide nearly lossless fixed rate compression wherein one or more fallback lossy codecs are applied when a default lossless codec is unable to meet a bit budget.
  • the one or more modules are assumed to be separate functional units, a person of skill in the art will recognize that the functionality of the modules may be combined or integrated into a single module, or further subdivided into further sub-modules without departing from the scope of the claims.
  • the encoder 100 and the decoder 200 are implemented in one or more processing circuits.
  • processing circuit is used herein to mean any combination of hardware, firmware, and software, employed to process data or digital signals.
  • Processing circuit hardware may include, for example, application specific integrated circuits (ASICs), general purpose or special purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices such as field programmable gate arrays (FPGAs).
  • ASICs application specific integrated circuits
  • CPUs general purpose or special purpose central processing units
  • DSPs digital signal processors
  • GPUs graphics processing units
  • FPGAs programmable logic devices
  • each function is performed either by hardware configured, i.e., hard-wired, to perform that function, or by more general purpose hardware, such as a CPU, configured to execute instructions stored in a non-transitory storage medium.
  • a processing circuit may be fabricated on a single printed circuit board (PCB) or distributed over several interconnected PCBs.
  • a processing circuit may contain other processing circuits; for example a processing circuit may include two processing circuits, an FPGA and a CPU, interconnected on a PCB.
  • any numerical range recited herein is intended to include all sub-ranges of the same numerical precision subsumed within the recited range.
  • a range of "1.0 to 10.0" is intended to include all subranges between (and including) the recited minimum value of 1.0 and the recited maximum value of 10.0, that is, having a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0, such as, for example, 2.4 to 7.6.
  • Any maximum numerical limitation recited herein is intended to include all lower numerical limitations subsumed therein and any minimum numerical limitation recited in this specification is intended to include all higher numerical limitations subsumed therein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Crystallography & Structural Chemistry (AREA)
  • Computer Hardware Design (AREA)
  • Discrete Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
EP21164402.6A 2020-04-06 2021-03-23 Systèmes et procédés pour codecs de compression de données hybrides à faible complexité, presque sans perte et à débit fixe Active EP3893504B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063005880P 2020-04-06 2020-04-06
US16/900,833 US11244476B2 (en) 2020-04-06 2020-06-12 Systems and methods for low-complexity near lossless fixed-rate hybrid data compression codecs

Publications (2)

Publication Number Publication Date
EP3893504A1 true EP3893504A1 (fr) 2021-10-13
EP3893504B1 EP3893504B1 (fr) 2024-03-13

Family

ID=75203069

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21164402.6A Active EP3893504B1 (fr) 2020-04-06 2021-03-23 Systèmes et procédés pour codecs de compression de données hybrides à faible complexité, presque sans perte et à débit fixe

Country Status (4)

Country Link
US (1) US11244476B2 (fr)
EP (1) EP3893504B1 (fr)
KR (1) KR102763185B1 (fr)
CN (1) CN113497939B (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12425567B2 (en) 2022-10-31 2025-09-23 Samsung Display Co., Ltd. System and method for encoding image

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013006370A1 (fr) * 2011-07-05 2013-01-10 Qualcomm Incorporated Compression sans perte/avec perte sélective de données d'image sur la base d'une capacité binaire allouée
US20160301950A1 (en) * 2015-04-13 2016-10-13 Qualcomm Incorporated Rate-constrained fallback mode for display stream compression

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5933195A (en) * 1997-09-26 1999-08-03 Sarnoff Corporation Method and apparatus memory requirements for storing reference frames in a video decoder
US6980695B2 (en) 2002-06-28 2005-12-27 Microsoft Corporation Rate allocation for mixed content video
US7224731B2 (en) 2002-06-28 2007-05-29 Microsoft Corporation Motion estimation/compensation for screen capture video
US7415158B1 (en) 2004-06-30 2008-08-19 Sun Microsystems, Inc. Method for determining whether to use a lossy or lossless codec to compress a digital image using a table of non-allowed pixel values
KR100723505B1 (ko) * 2005-10-06 2007-05-30 삼성전자주식회사 하이브리드 방식의 영상 데이터 처리 시스템 및 영상데이터 처리 방법
US20090238259A1 (en) * 2008-03-19 2009-09-24 Sung-Hung Yeh Method of rate control for video frame compression and encoder thereof
US8194736B2 (en) 2008-04-15 2012-06-05 Sony Corporation Video data compression with integrated lossy and lossless compression
US9578336B2 (en) 2011-08-31 2017-02-21 Texas Instruments Incorporated Hybrid video and graphics system with automatic content detection process, and other circuits, processes, and systems
US8582876B2 (en) 2011-11-15 2013-11-12 Microsoft Corporation Hybrid codec for compound image compression
JP6857189B2 (ja) 2016-02-26 2021-04-14 ヴァーシテック・リミテッドＶｅｒｓｉｔｅｃｈ Ｌｉｍｉｔｅｄ 画像のロッシー圧縮およびロスレス圧縮のための形状適応モデルベースコーデック
US10212421B2 (en) * 2016-12-30 2019-02-19 Intopix S.A. Method and device for digital data compression with sign bit removal
CN108347607B (zh) * 2017-01-25 2020-08-18 联咏科技股份有限公司 固定码率且基于行的嵌入式视频压缩方法和图像处理设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013006370A1 (fr) * 2011-07-05 2013-01-10 Qualcomm Incorporated Compression sans perte/avec perte sélective de données d'image sur la base d'une capacité binaire allouée
US20160301950A1 (en) * 2015-04-13 2016-10-13 Qualcomm Incorporated Rate-constrained fallback mode for display stream compression

Also Published As

Publication number Publication date
KR20210124884A (ko) 2021-10-15
US20210312666A1 (en) 2021-10-07
KR102763185B1 (ko) 2025-02-05
CN113497939B (zh) 2024-04-26
US11244476B2 (en) 2022-02-08
EP3893504B1 (fr) 2024-03-13
CN113497939A (zh) 2021-10-12

Similar Documents

Publication Publication Date Title
US7526124B2 (en) Match MSB digital image compression
EP3610648B1 (fr) Diffusion d'erreur de prédiction de point milieu pour une compression de flux d'affichage
JP2008532083A (ja) 画像データをフレームメモリに一時的に記憶する新たな圧縮フォーマットを用いる新たな圧縮フォーマット及び装置
US10609382B2 (en) Method and apparatus for compressing video data
KR20140037309A (ko) 이미지 압축 회로와 이를 포함하는 디스플레이 시스템
US11936898B2 (en) DPCM codec with higher reconstruction quality on important gray levels
US8675732B2 (en) Method and apparatus of compressing image data
US12075054B2 (en) Compression with positive reconstruction error
KR20180136618A (ko) 영상 압축 방법 및 이를 수행하는 표시 장치
EP3849185A1 (fr) Procédé de codage et de décodage de contenus d'images et système de transfert de contenu d'images
EP3893504B1 (fr) Systèmes et procédés pour codecs de compression de données hybrides à faible complexité, presque sans perte et à débit fixe
KR20160116298A (ko) 이미지 프레임의 시퀀스를 코딩하는 방법 및 그 시스템
KR20070037248A (ko) 영상 부호화장치 및 방법, 영상 복호화장치 및 방법과 이를채용한 디스플레이 구동회로 및 방법
JP2025519522A (ja) 画像符号化、復号方法及び装置
EP1579389B1 (fr) Codage par longueur de ligne d'une image numerique quantifiee
JP2013126139A (ja) 画像符号化装置、画像復号化装置、画像符復号化システム、画像符号化方法及び画像復号化方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220413

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230516

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/176 20140101ALI20230907BHEP

Ipc: H04N 19/146 20140101ALI20230907BHEP

Ipc: H04N 19/12 20140101AFI20230907BHEP

INTG Intention to grant announced

Effective date: 20231004

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602021010252

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240613

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240613

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240613

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240614

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1666827

Country of ref document: AT

Kind code of ref document: T

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240715

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240715

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240713

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602021010252

Country of ref document: DE

Ref country code: BE

Ref legal event code: MM

Effective date: 20240331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240331

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240323

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240331

26N No opposition filed

Effective date: 20241216

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20250220

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250220

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240513

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250224

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20210323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20210323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240313