[go: up one dir, main page]

WO2013009131A2 - Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt - Google Patents

Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt Download PDF

Info

Publication number
WO2013009131A2
WO2013009131A2 PCT/KR2012/005584 KR2012005584W WO2013009131A2 WO 2013009131 A2 WO2013009131 A2 WO 2013009131A2 KR 2012005584 W KR2012005584 W KR 2012005584W WO 2013009131 A2 WO2013009131 A2 WO 2013009131A2
Authority
WO
WIPO (PCT)
Prior art keywords
mmt
packet
video
information
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2012/005584
Other languages
English (en)
Korean (ko)
Other versions
WO2013009131A3 (fr
Inventor
이진영
서광덕
윤국진
정원식
허남호
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electronics and Telecommunications Research Institute ETRI
Original Assignee
Electronics and Telecommunications Research Institute ETRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electronics and Telecommunications Research Institute ETRI filed Critical Electronics and Telecommunications Research Institute ETRI
Priority claimed from KR1020120076395A external-priority patent/KR20130009671A/ko
Publication of WO2013009131A2 publication Critical patent/WO2013009131A2/fr
Publication of WO2013009131A3 publication Critical patent/WO2013009131A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2381—Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H04N21/64322—IP

Definitions

  • the present invention relates to a packet transmission apparatus and method, and a packet receiving apparatus and method, and more particularly, to a packet transmission apparatus and method based on a timing model for an MPEG Media Tranport (MMT) system, and a packet receiving apparatus and method. It is about.
  • MMT MPEG Media Tranport
  • RTP has a transmitter-oriented flexible timing model (the description of the receiver is not intentionally specifically limited to allow for flexibility).
  • the MPEG-2 system provides a clear timing model for the receiver. Therefore, due to the difference between the timing model of the MPEG-2 system and the timing model for the receiver requiring the timing model of the RTP system, it is difficult to match the packet synchronization in the MMT system, which requires a complicated configuration. .
  • An object of the present invention is to provide an apparatus and method for transmitting a packet based on a timing model of an integrated MMT system in consideration of the advantages of the timing model and the RTP timing model in an MPEG-2 system. It is to provide an apparatus and method.
  • another object of the present invention is to transmit a packet stream transmitted through the MPEG-2 system or RTP into a new type of packet including time stamp information used for network jitter compensation, time synchronization, and the like. It is an object of the present invention to provide a packet transmission apparatus and method, and a packet receiving apparatus and method in an MMT system that effectively perform synchronization without countering this.
  • a packet transmission apparatus of an MMT system for achieving the above object is a device for transmitting a packet in a MPEG Media Transport (MMT) system, the packet is encoded video stream and the audio stream corresponding to the video stream based on the input picture And an MMT packetizer for generating an MMT packet, wherein the MMT packetizer includes synchronization information generated based on a system time clock (STC) and an MMT buffering delay, which are master program clocks of a video and audio encoder. To generate the MMT packet.
  • MMT MPEG Media Transport
  • the MMT packet transmission apparatus may include a video encoder that encodes the input picture to generate a video stream; A video buffer that stores the encoded video stream; And a transmitter configured to transmit the packetized MMT packet, wherein the video buffer may output the video stream at a constant rate.
  • the MMT packetizer may packetize the video stream output from the video buffer including MCR clock reference (MCR) time stamp information generated by time stamping the video stream output from the video buffer.
  • MCR MCR clock reference
  • the packetizer may control at least one of a data rate and a packet rate in a channel for transmitting the MMT packet using the MCR time stamp information.
  • the MMT packetizer may insert the MCR time stamping information into the D.1 layer of the MMT packet.
  • the MMT packetization unit generates a PTS (Presentation Time Stamp) generated by adding a sum of a buffering delay at the transmitting apparatus, a buffering delay at the receiving apparatus, and a network delay to a time stamp value of the input picture time stamped using the STC. ) May be generated to include the MMT packet.
  • PTS Presentation Time Stamp
  • the MMT packetizer may insert the PTS information into the E.3 layer of the MMT packet.
  • the STC value varies depending on a program, and the MMT packetizer may multiplex different programs having different STCs.
  • MMT packet in the method for transmitting a packet in the MMT system, MMT packet by packetizing the encoded video stream and the audio stream corresponding to the video stream based on the input picture And generating an MMT packetization step, wherein the MMT packetization step includes the MMT packet including synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, and an MMT buffering delay. It may include the step of generating.
  • STC System Time Clock
  • a packet receiving apparatus of an MMT system for achieving the above object is an apparatus for receiving a packet in an MPEG Media Transport (MMT) system, the MMT depacket to depacketize the received MMT packet to generate a video stream and an audio stream
  • MMT depacketization unit includes a video unit, and estimates network jitter using synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, included in the MMT packet, and an MMT buffering delay.
  • STC System Time Clock
  • the packet receiving apparatus includes a video buffer for storing the video stream; A video decoder for decoding the video stream; And a reorder buffer for reordering the decoded video picture for display.
  • the MMT packet digitizing unit calculates a difference value between an estimated arrival time of the MMT packet and an actual arrival time using MCR clock reference (MCR) time stamp information among the synchronization information and the STC value;
  • a network jitter estimator for estimating the size of the network jitter based on the difference value;
  • a digitizer configured to perform synchronization by digitizing the MMT packet using the estimated size of network jitter.
  • MCR MCR clock reference
  • the MMT packet digitizing unit may perform a digitizing operation before the encoded MCR time stamp information is received.
  • the MMT packet digitizing unit may extract the MCR time stamp information from the header of the D.1 layer of the received MMT packet.
  • the MMT depacketizer may generate information on whether the video picture is an I picture, a P picture, or a B picture by comparing PTS (Presentation Time Stamp) information included in the MMT packet with the STC value.
  • PTS Presentation Time Stamp
  • the MMT depacketizer may parse the PTS information to generate time information when the video stream is output from the video buffer and decoded.
  • the MMT depacketizer may parse the PTS information to generate time information that is output from the reordering buffer to the decoded video picture.
  • the MMT depacketizer may parse the PTS information from the E.3 layer of the received MMT packet.
  • a packet receiving method of an MMT system for achieving the above object is a method of receiving a packet in an MPEG Media Transport (MMT) system, the MMT depacket to depacketize the received MMT packet to generate a video stream and an audio stream
  • the MMT depacketizing step may include an MMT packet digitizing step of digitizing the MMT packet by estimating network jitter using MCR information included in the MMT packet.
  • the MMT depacketizing step includes generating at least one of time information at which the video stream is decoded and time information at which the decoded video picture is displayed using PTS (Presentation Time Stamp) information included in the MMT packet. It may include.
  • PTS Presentation Time Stamp
  • the MMT transmission apparatus / receiver STC clock locking is achieved. It is effective.
  • the packet transmission apparatus and method and the packet reception apparatus and method in the MMT system of the present invention, it is possible to efficiently perform the synchronization of packets transmitted through the MPEG-2 system or RTP without violating the conventional standard. It works.
  • 1A is a conceptual diagram illustrating an MMT hierarchical structure
  • 1B is a conceptual diagram illustrating a transmission delay of a packet transmitter and a packet receiver in an MMT system
  • FIG. 2 is a block diagram schematically showing the configuration of a packet transmission apparatus in an MMT system according to an embodiment of the present invention
  • FIG. 3 is a conceptual diagram illustrating encapsulation of an MPEG TS packet into a D layer payload of an MMT
  • FIG. 4 illustrates an MMT packet including an MPEG TS packet having MCR information in a TS packet header
  • FIG. 5 is a diagram illustrating MCR time stamp information existing in a D layer header
  • FIG. 6 is a block diagram schematically showing the configuration of a packet receiving apparatus in an MMT system according to an embodiment of the present invention
  • FIG. 7 is a detailed block diagram illustrating an MMT depacketization unit of a packet receiving apparatus in an MMT system according to an embodiment of the present invention
  • FIG. 8 is a detailed block diagram illustrating in detail an MMT packet digitizing unit of a packet receiving apparatus in an MMT system according to an embodiment of the present invention
  • FIG. 9 is a view for explaining an interlocking relationship between an MMT depacketizer and a receiver buffer model for MPEG-2 TS packets in a packet receiving apparatus in an MMT system according to another embodiment of the present invention.
  • FIG. 10 is a flowchart illustrating a method of processing synchronization according to a video picture by parsing MCR information in an MMT depacketizer of a packet receiving apparatus in an MMT system according to an embodiment of the present invention.
  • first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.
  • the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.
  • 1A is a conceptual diagram illustrating an MMT hierarchical structure.
  • the MMT layer includes an encapsulation layer, a delivery layer, and a functional area of the S layer.
  • the MMT layer operates on a transport layer.
  • the encapsulation layer may be responsible for, for example, packetization, fragmentation, synchronization, multiplexing, and the like of transmitted media.
  • Encapsulation layer (E-layer), as shown in Figure 1a, MMT E.1 Layer (MMT E.1 Layer), MMT E.2 Layer (MMT E.2 Layer) and MMT E.3 Layer (MMT) E.3 Layer).
  • MMT E.1 Layer MMT E.1 Layer
  • MMT E.2 Layer MMT E.2 Layer
  • MMT E.3 Layer MMT E.3 Layer
  • the E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to create an M-Unit.
  • MFU Media Fragment Unit
  • the MFU may have a format, independent of any particular codec, that can carry data units that can be consumed independently in the media decoder.
  • the MFU can be, for example, a picture or slice of the video.
  • the M-unit may consist of one or a plurality of MFUs and may have a format, independent of a particular codec, that may carry one or a plurality of access units.
  • the E.2 layer encapsulates the M-units created in the E.3 layer to create an MMT asset.
  • An MMT asset is a data entity composed of one or a plurality of M-units from a single data source and is a data unit in which composition information and transport characteristics are defined.
  • MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream), etc.
  • PES packetized elementary streams
  • the E.1 layer creates an MMT package by encapsulating the MMT asset generated in the E.2 layer.
  • the MMT package may be composed of one or more MMT assets together with additional information such as composition information and transport characteristics.
  • Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, it indicates a relationship between a plurality of MMT packages. It may further include information.
  • the transport characteristics may include transmission characteristic information necessary for determining a delivery condition of an MMT asset or an MMT packet, and may include, for example, a traffic description parameter and a QoS descriptor. ) May be included.
  • the MMT package may correspond to a program of MPEG-2 TS.
  • the delivery layer may perform, for example, network flow multiplexing, network packetization, and QoS control of media transmitted through a network.
  • the transport layer (D-layer) is, as shown in Figure 1a, MMT D.1 Layer (MMT D.1 Layer), MMT D.2 Layer (MMT D.2 Layer) and MMT D.3 Layer (MMT) D.3 Layer).
  • the D.1 layer receives the MMT package generated in the E.1 layer and generates an MMT payload format.
  • the MMT payload format is a payload format for carrying MMT assets and for transmitting information for consumption by the MMT application protocol or other existing application transport protocol such as RTP.
  • the MMT payload may include a fragment of the MFU along with information such as AL-FEC.
  • the D.2 layer receives the MMT payload format generated in the D.1 layer and generates an MMT transport packet or an MMT packet.
  • the MMT transport packet or MMT packet is a data format used in an application transport protocol for MMT.
  • D.3 layer supports QoS by providing the ability to exchange information between layers by cross-layer design.
  • the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer.
  • the S layer performs a signaling function. For example, signaling functions for session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization, etc. Can be done.
  • the S layer may be configured of an MMT S.1 layer and an MMT S.2 layer.
  • S.1 layer includes service discovery, media session initialization / termination of media, media session presentation / control of media, delivery (D) layer and encapsulation (E).
  • the interface function with the layer can be performed.
  • the S.1 layer may define the format of control messages between applications for media presentation session management.
  • the S.2 layer is responsible for flow control, delivery session management, delivery session monitoring, error control, and hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.
  • the S.2 layer supports delivery session establishment and release, delivery session monitoring, flow control, error control, resource scheduling for established delivery sessions, and synchronization in a complex delivery environment to support the behavior of the delivery layer.
  • Signaling for adaptive delivery, and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the S.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above.
  • the S.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.
  • Maintaining a timing relationship between packets of a single media stream or different media streams is an essential element in MPEG Media Transport (MMT) systems.
  • MMT MPEG Media Transport
  • the MMT system re-adjusts the timing relationship for ensuring synchronization time between MMT packets, and has synchronization functions and de-jittering algorithms.
  • the timing model for the MMT system is an extended concept from any of the MPEG system standards, which utilizes a network de-jitter estimation function similar to a real time transport protocol (RTP) based streaming system.
  • RTP real time transport protocol
  • FIG. 1B is a conceptual diagram illustrating a transmission delay of a packet transmitter and a packet receiver in an MMT system. As shown in FIG. 1B, the packet transmission apparatus and the packet reception apparatus according to an embodiment of the present invention transmit and receive packets with a constant data delay.
  • the MMT system may include a transmitter 10, an IP network 20, and a receiver 30.
  • the transmitter 10 generates an MMT packet by packetizing the input picture, and transmits the packet to the receiver 30.
  • IP network 20 may be a wireless network.
  • the transmission device 10 buffers the video stream encoded by the video encoder in order to synchronize with the audio stream. Therefore, in the transmission device 10, a delay D S, i occurs due to buffering. In addition, the delay D N, i may occur on the transmission path in the IP network 20.
  • the receiving device 30 After receiving the MMT packet, the receiving device 30 must also buffer until the received packet is decoded. If the decoded data includes the B pictures, the receiving device 30 needs to reorder the buffers, so the buffering is necessary. D R, i ). Due to these delays, the synchronization between the audio data and the video data may be a problem in a service broadcast in real time.
  • the synchronization problem can be solved by integrating the delay (D R, i ) to obtain the total delay (D T, i ) and determining the representation information time at which the received image is displayed in consideration of the obtained delay (D T, i ). have. Therefore, the transmitting device 10 of the present invention transmits the packet in a form including the above synchronization parameter, and when the receiving device 30 receives the packet, it extracts the synchronization parameter to synchronize the decoding to perform decoding. Can be.
  • the packet transmission apparatus 10 includes an audio encoder 110, an audio buffer 120, a video encoder 130, a video buffer 140, an MMT packetizer 150, and an STC 160. System Time Clock).
  • the audio encoder 110 encodes an audio block of content to be transmitted to the receiving device 30 to generate an audio stream.
  • the audio buffer 120 stores the encoded audio stream until it is packetized by the MMT packetizer 150.
  • the audio buffer 120 may receive audio data output from the audio encoder 110 invariably as an input and output the same at a constant rate. By doing so, the packetization with the video stream inputted at a constant rate by the MMT packetizer 150 can be induced to be easier.
  • the video encoder 130 receives a picture as an input and performs encoding to generate a video stream.
  • the video buffer 140 stores the encoded video stream and transmits the encoded video stream to the MMT packetizer 150.
  • the MMT packetizer 150 generates an MMT packet by packetizing a video stream and an audio stream corresponding to the video stream. At this time, synchronization information may be inserted into the MMT packet.
  • the STC 160 belongs to a specific program and is the master clock of the video encoder 130 and the audio encoder 110 for that program. That is, at the input of the encoders 110 and 130, the generation time of the input picture or the audio block is sampled and recorded by the STC 160. Thus, each picture or audio block obtains timing information t i .
  • the STC 160 may be different depending on the program.
  • the input picture is input to the video encoder 130 at a constant rate, and the video encoder 130 encodes the input picture. Since the time required may vary depending on the picture, the encoded video stream is output at various rates. Accordingly, the video buffer 140 stores video streams output at various rates and outputs them at a constant rate. The video stream output at a constant rate through the video buffer 140 is transmitted to the MMT packetizer 150.
  • the MMT packetizer 150 generates an MMT packet by packetizing the video stream output at a constant rate and the audio stream output through the audio buffer 120, and then receives the received device 30 through the IP network 20. To send.
  • the reception device 30 may depacketize the MMT packet and then decode the MMT packet to display the picture.
  • the MMT system model may assume that the delay occurs only in the buffers 120 and 140 when it is assumed that the delay time at the decoder is 0 and that the delay from the encoders 110 and 130 is also 0. .
  • the MMT system model accepts the fact that zero-delay elements, such as transmission path delays, can have a constant delay without disrupting operation.
  • the decoder may be synchronized with the encoders 110 and 130 by time stamps.
  • the transmission device 10 may include a master oscillator, a counter, and the STC 160.
  • the transmission device 10 may generate a presentation time stamp (PTS) by adding a predetermined amount of delay equal to the delay of the encoder and the decoder to the STC value for the picture or audio block.
  • the video encoder 130 transmits the MMT buffering delay, that is, the buffering delay in the transmitting apparatus 10 and the buffering delay information in the receiving apparatus 30 to the MMT packetizer 150.
  • the MMT packetizer 150 may generate a PTS by calculating the MMT buffering delay with a time stamp of an input picture based on the STC 160. In some cases, assuming an ideal model, the operation time of the MMT packetization process and the depacket process may be assumed to be zero.
  • the MMT packetizer 150 inserts the PTS information into the E.3 layer of the MMT packet representing the picture or audio block.
  • Decoding Time Stamp (DTS) and PTS are the same except for the case of changing the order of B-pictures. This is because the I picture or the P picture is not changed in order, and therefore the time to be decoded is the time to be displayed.
  • the PTS can be used to indicate the time at which the access device is output from the buffer of the receiving device 30, enters the decoder and decodes.
  • the PTS related information may be used to indicate the time output from the reordering buffer for display by the display device of the related picture.
  • the output from video buffer 140 i.e., the video stream, includes a time stamped time with an STC value, which is referred to as MMT Clock Reference (MCR).
  • MCR MMT Clock Reference
  • the MCR time stamp may be inserted in the D.1 layer of the packetization process of the D.1 MMT packet.
  • the MCR time stamp may be required to be generated at maximum 100 ms interval in MPEG-2 TS.
  • all video and audio streams included in the program have time stamped values from the common STC 160, to achieve synchronization of the video and audio decoders.
  • An important characteristic of the system of MCR time stamps is that the data rate and packet rate on the channel (at the output of the transmitting device 10) can be completely asynchronous with the STC 160, and still the STC 160 can still be synchronized at the decoder. will be. It may also be sent in the multiplexed MMT packet while different programs with different STCs 160 allow recovery of the STC 160 for each program.
  • MPEG-2 TS is a standard format for storing and transmitting audio, video and data, and can be used in broadcasting systems such as DVB and ATSC. According to the requirements of the MMT, the MMT may support streaming of pre-stored content. In order to transmit an MPEG-2 TS packet using an IP network-based MMT system, the MMT packet must provide timing information for calculating network jitter.
  • MPEG-2 TS was originally developed to be used for digital broadcasting services, and TS packets can be transmitted based on a constant circuit switching network with a relatively short transmission delay compared to an IP-based packet switched network.
  • a transmission delay may be processed by using a timing buffer model of the TS in a T-STD.
  • MMT using an IP-based packet-switched network may require a redesign of the buffer model because the T-STD cannot handle reliable transmissions due to variations in arrival time and jitter due to packet delays. have.
  • FIG. 3 is a conceptual diagram illustrating encapsulation of the MPEG TS packet 300 in the D layer payload of the MMT. As shown in FIG. 3, 16 MPEG TS packets 300 are shown, and 7 MPEG TS packets 300 may be packetized into one MMT packet 310 or 320, and one MMT. The header 312 and 322 data may be attached to the payloads 314 and 324 of the packet and transmitted.
  • the TS packet 300 mapped to the D layer payloads 314 and 324 may include an MCR in a TS packet header field included in the D layer payloads 314 and 324 as described above. May contain information.
  • the MMT packetizer 150 generates MCR information based on the MMT buffering delay, the network delay, and the STC value received from the video encoder 130, and the packet payloads 314 and 324 of the D layer are generated. It is included in the TS packet header field included in the packetized packet so that the decoder can easily extract synchronization information.
  • the packet headers 312 and 322 of the D layer may include MCR time stamp information.
  • FIG. 4 is a diagram illustrating an MMT packet 400, 420 including an MPEG TS packet having MCR information 410, 434, 440 in a TS packet header.
  • MMT packet 400, 420 includes an IP / UDP header 402, 422 and TS packet including D layer header 404, 424 and TS packet header 406, 426, 430, 436 and TS packet payload 408, 428, 432, 438.
  • the MCR information 410 is located in the packet header 406 of the first TS packet in the D layer payload. That is, the MCR information 410 is located in the packet header 406 of the first TS packet only in the first MMT packet 400, and in the case of the second MMT packet 420, the packet header 426 of the first TS packet 400 is located in the packet header 426 of the first TS packet. There is no MCR information, and the MCR information 434 and 440 are included in the second packet header 420 to the seventh TS packet header 436. Therefore, only the MCR value present in the TS packet cannot calculate the accurate jitter value in the MMT service scenario. Accordingly, there is a need for a new reference clock shared by both the MMT transmitting device 10 and the receiving device 30, such as MCR time stamp information.
  • MCR time stamp information 502 is included in the header 500 of the D layer of the MMT packet.
  • the MCR time stamp information is included in a sampling instance clock value of the first byte in the D layer payload.
  • this value may be used for the purpose of system clock locking between the transmitting device 10 and the receiving device 30 as well as jitter estimation and round trip time (RTT) calculation.
  • the MMT packet receiving apparatus 600 may include an MMT depacketizer 610, an audio buffer 620, an audio decoder 630, a video buffer 640, Video decoder 650 and reorder buffer 660.
  • the MMT packet receiving apparatus 600 receives an MMT packet transmitted from the transmitting apparatus 690 through the IP network 680, depackets the received MMT packet into an audio block and a video picture. And the synchronization information between the two is extracted and displayed in a synchronized state.
  • the MMT depacketizer 610 depackets an MMT packet to generate a video stream and an audio stream.
  • the MMT packet depacketizer 610 may estimate network jitter using the MCR information included in the D.1 layer of the MMT packet and the STC 607 (S602). Digitization may be performed based on the estimated network jitter to synchronize synchronization between MMT packets.
  • S604 by comparing the PTS included in the E.3 layer of the MMT packet with the STC 607 (S604), synchronization information necessary for decoding in the video decoder 650 may be provided. Through the comparison (S604), it may be determined whether the video picture decoded is an I picture, a P picture, or a B picture.
  • synchronization information is provided to the reordering buffer 660 to change the order so that pictures may be sequentially output.
  • the audio buffer 620 stores the audio stream and outputs it to the audio decoder 630 at a constant rate. This is to store the audio stream in front of the decoder 630 in order to keep the input to the decoder 630 in the service provided in real time.
  • the audio decoder 630 generates an audio block by receiving an audio stream from the audio buffer 620 and performing decoding.
  • the video buffer 640 like the audio buffer 620, stores a video stream.
  • the stored video stream is output to the audio decoder 630 at a constant rate.
  • the video decoder 650 receives the video stream from the video buffer 640 and performs decoding to generate a video picture.
  • the video decoder 650 may perform decoding in consideration of the decoding time of the picture by providing synchronization information calculated by comparing the PTS and the STC 607 from the MMT depacketizer 610.
  • decoding may be performed based on the PTS. That is, the PTS is used to indicate the time that should be output from the video buffer 640 and decoded.
  • the PTS related information may be used to indicate the time displayed by the reordering buffer when the display device of the image is displayed.
  • the reorder buffer 660 rearranges and displays the video picture generated by decoding from the video decoder 650. For video pictures consisting only of I pictures or B pictures, the reorder buffer 660 does not perform a special function. Because there is no room for change. However, when the B picture is included, since the B picture needs to be changed in order, the B picture needs to be changed in order with the previous picture, and such an operation is performed through the reorder buffer 660.
  • the reordering buffer 660 extracts the display time of the image using the PTS information on the video picture identified as the B picture by comparing the PTS and the STC 607 in the MMT depacketizer 610 (S604). The picture may be output at the display time.
  • the MMT system supports elasticity to packet arrival jitter and adjusts end-to-end delay in order to perform continuous decoding and presentation of content under various network conditions.
  • This goal can be achieved in RTP-based systems by supporting a cooperative manner of RTP time stamps and NTP time stamps. With such a time stamp, it is possible to estimate network jitter, packet loss rate, and packet round trip time, which are essential information to protect the system in operation against the network change.
  • the MMT depacketizer 610 may include an MMT packet digitizer 710 and a depacketizer 720.
  • the MMT digitizing unit 710 estimates network jitter occurring during MMT packet transmission using MCR time stamp information.
  • the MMT digitizing unit 710 may calculate the time difference between the expected arrival time and the actual arrival time of the MMT packet with the MCR time stamp information and the local STC clock. This time difference can be used to estimate the actual value of network jitter.
  • the depacketizer 720 depackets the digitized packet through the MMT packet digitizer 710 to estimate the network jitter, and divides the packet into an audio stream and a video stream.
  • the divided video stream is sent to the video buffer, and the divided audio stream is sent to the audio buffer.
  • the MMT packet digitizer 710 may include a digital buffer 810, an extractor 820, a difference value calculator 830, a network jitter estimator 840, and a digital digitizer ( 850).
  • the digit buffer 810 stores the received MMT packet until digitization of the MMT packet is performed to restore the temporal relationship between the MMT packets at the output.
  • the size of the digital buffer 810 may be calculated using MCR information.
  • the extractor 820 parses the D.1 layer of the MMT packet stored in the digital buffer 810 and extracts MCR time stamp information included therein.
  • the D.1 layer of the MMT packet packetized in the transmitting apparatus includes the MCR time stamp information of which the video stream is time stamped based on the STC. MCR time stamp information is extracted from the.
  • the difference calculator 830 calculates a difference value between the estimated arrival time of the MMT packet and the actual arrival time by using the STC value and the MCR time stamp information extracted by the extractor 820. Since the MCR time stamp information is a time stamped value based on the STC value, the MCR time stamp information includes information on the time of packetization, and thus, the estimated time of arrival of the MMT packet may be calculated. By calculating the difference between the calculated estimated arrival time and the actual arrival time of the MMT packet, the time difference value can be calculated.
  • the network jitter estimator 840 estimates the size of the network jitter based on the time difference calculated by the difference calculator 830. If the time difference is large, the size of network jitter also increases proportionally, and thus the size of jitter can be estimated using a proportional relationship.
  • the digitizer 850 digitizes the MMT packet using the size information of the network jitter estimated by the network jitter estimator 840. That is, the delay is offset by the amount of network jitter so that synchronization between MMT packets can be achieved. Accordingly, the MMT packet passing through the digitizing unit 850 is a digitized packet and is transmitted to the depacketizer 720 because synchronization between the MMT packets is correct.
  • the MMT digitizing unit 710 may be preferably performed before the MCR encoded encoding is transferred to the MMT receiving apparatus. .
  • the MMT digitizer 710 may remove jitter generated by the IP network, and may forward the digitized MMT packet to the depacketizer 720.
  • FIG. 9 is a diagram illustrating an interworking relationship between an MMT depacketizer and a receiver buffer model for MPEG-2 TS packets in a packet receiving apparatus in an MMT system according to another embodiment of the present invention.
  • the received MMT packet is digitized through the digitizer 910, depacketized into MPEG-2 TS packets through the depacketizer 920, and MPEG-2 T-STD.
  • the model 930 can complete the delay process.
  • an MMT packet may include a plurality of MPEG-2 TS packets in a payload, and as described above, an MPEG-2 TS packet is developed to be used for a digital broadcasting service and is generally based on IP. Compared to a packet switched network, the transmission delay is relatively short and can be transmitted based on a constant circuit switched network.
  • the MPEG-2 T-STD model 930 which is in charge of delay processing of the MPEG-2 TS packet can be utilized. This improves backward compatibility.
  • the digitizing unit 910 calculates network jitter based on MCR time stamp information in the same manner as the MMT digitizing unit 710 described above, calculates an appropriate size of the digital buffer and generates a digital MMT packet through the same. .
  • the depacketizer 920 simply converts an MMT packet into an MPEG-2 TS packet.
  • the packet passing through the depacketizer 920 becomes a digitized MPEG-2 TS packet.
  • the MPEG-2 T-STD model 930 basically has a structure for processing a transmission delay using a TS timing buffer model and may perform delay processing on a TS packet.
  • FIG. 10 is a flowchart illustrating a method of processing synchronization according to a video picture by parsing MCR information in an MMT depacketizer of a packet receiving apparatus in an MMT system according to an embodiment of the present invention.
  • the MMT depacketizer 610 depackets an MMT packet and extracts a PTS included in the E.3 layer of the MMT packet (S1010). Then, by comparing the extracted PTS information and the STC value (S1020), it is possible to provide synchronization information necessary for decoding in the video decoder. On the basis of the comparison result, it is determined whether the B picture exists in the decoded video picture (S1030). That is, it is possible to determine whether the picture is an I picture, a P picture, or a B picture.
  • the display time is determined by parsing the PTS information (S1040). Then, the display time is transmitted to the reordering buffer (S1050), and the pictures are sequentially output.
  • the reordering buffer enables the corresponding picture to be output at the display time based on the display time received for the video picture identified as the B picture.
  • the decoding time is determined based on the PTS information (S1060). Then, the time information is transmitted to the video decoder (S1070). Since the PTS information is the same as the DTS, the video decoder receiving the time information may perform decoding based on the PTS. That is, the PTS is used to indicate the time that should be output from the video buffer and decoded through the video decoder. The video decoder may perform decoding in consideration of the decoding time of the picture based on the synchronization information received from the MMT depacketizer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne un dispositif et un procédé pour émettre un paquet ainsi qu'un dispositif et un procédé pour recevoir un paquet sur la base d'un modèle temporel d'un système MMT (transport multimédia MPEG), que l'on intègre en tenant compte des avantages d'un modèle temporel dans un système MPEG-2 et d'un modèle temporel RTP. Le dispositif d'émission de paquets émet le paquet à partir du système MMT et comprend un flux vidéo qui est codé sur la base d'une image entrée, ainsi qu'un module de mise en paquets MMT qui génère le paquet MMT en mettant en paquets un flux audio qui correspond au flux vidéo. Le module de mise en paquets MMT génère le paquet MMT en incluant une information de synchronisation qui est générée sur la base d'une horloge système (STC) qui est une horloge de programme maître pour un codeur vidéo et audio, et d'un retard de mise en tampon MMT. L'invention permet ainsi la synchronisation de diverses sources multimédia, la mesure de la gigue d'arrivée des paquets, ainsi que le verrouillage de l'horloge STC du dispositif d'émission et du dispositif de réception du MMT.
PCT/KR2012/005584 2011-07-13 2012-07-13 Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt Ceased WO2013009131A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20110069325 2011-07-13
KR10-2011-0069325 2011-07-13
KR1020120076395A KR20130009671A (ko) 2011-07-13 2012-07-13 엠엠티 시스템에서의 패킷 전송 장치 및 방법, 및 패킷 수신 장치 및 방법
KR10-2012-0076395 2012-07-13

Publications (2)

Publication Number Publication Date
WO2013009131A2 true WO2013009131A2 (fr) 2013-01-17
WO2013009131A3 WO2013009131A3 (fr) 2013-03-07

Family

ID=47506739

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/005584 Ceased WO2013009131A2 (fr) 2011-07-13 2012-07-13 Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt

Country Status (1)

Country Link
WO (1) WO2013009131A2 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2750337C2 (ru) * 2013-04-19 2021-06-28 Самсунг Электроникс Ко., Лтд. Способ и устройство для передачи медиаданных в мультимедийной транспортной системе
EP4054199A1 (fr) * 2013-12-16 2022-09-07 Panasonic Intellectual Property Corporation of America Dispositif de réception et procédé de réception

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3644995B2 (ja) * 1995-01-25 2005-05-11 株式会社日立国際電気 符号化伝送方式におけるタイムスタンプ値演算方法
EP1613016A1 (fr) * 2004-07-01 2006-01-04 Thomson Licensing Méthode pour transmettre des paquets dans un système de transmission
KR20070008069A (ko) * 2005-07-12 2007-01-17 엘지전자 주식회사 음성/영상신호의 동기화 장치 및 방법
US8467444B2 (en) * 2008-06-16 2013-06-18 Canon Kabushiki Kaisha Information processing system, information processing apparatus, information processing method, and program

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2750337C2 (ru) * 2013-04-19 2021-06-28 Самсунг Электроникс Ко., Лтд. Способ и устройство для передачи медиаданных в мультимедийной транспортной системе
US11245940B2 (en) 2013-04-19 2022-02-08 Samsung Electronics Co., Ltd. Method and apparatus for transmitting media data in multimedia transport system
US11665384B2 (en) 2013-04-19 2023-05-30 Samsung Electronics Co., Ltd. Method and apparatus for transmitting media data in multimedia transport system
EP4054199A1 (fr) * 2013-12-16 2022-09-07 Panasonic Intellectual Property Corporation of America Dispositif de réception et procédé de réception
US11722714B2 (en) 2013-12-16 2023-08-08 Panasonic Intellectual Property Corporation Of America Transmitting method, receiving method, transmitting device and receiving device

Also Published As

Publication number Publication date
WO2013009131A3 (fr) 2013-03-07

Similar Documents

Publication Publication Date Title
US8379151B2 (en) Synchronization of audio and video streams
CA2559225C (fr) Suppresseur de gigue dans le flux de transport
KR102049907B1 (ko) Ｍｍｔ 시스템을 위한 미디어 데이터 전송 장치 및 방법, 그리고 미디어 데이터 수신 장치 및 방법
TWI455573B (zh) 恢復系統時鐘的裝置和方法
US9577682B2 (en) Adaptive forward error correction (FEC) system and method
US8774287B2 (en) Precise compensation of video propagation duration
KR20130078643A (ko) Ｍｍｔ 복합 전달 서비스에서 ｍｍｔ 패킷 스트림 동기화를 위한 ｍｍｔ 제어 계층 시그널링을 이용한 타이밍 정보 제공 방법 및 ｍｍｔ 패킷 스트림 동기화 방법
WO2019053853A1 (fr) Système de commutation vidéo
KR20130009670A (ko) 엠엠티 시스템에서의 패킷 전송 장치 및 방법, 및 패킷 수신 장치 및 방법
WO2012128563A2 (fr) Dispositif et procédé d'émission/réception de contenu radiodiffusé lié grâce à un réseau hétérogène
WO2013112027A1 (fr) Procédé d'estimation de gigue de réseau dans un appareil de transmission de données multimédia codées
JP2012513139A (ja) 外部のコプロセッサを有するマルチプレクサにおいてトランスポートストリームを同期させる方法
US20100172374A1 (en) System and method for transport of a constant bit rate stream
WO2013009131A2 (fr) Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt
EP1836786B1 (fr) Procede de transmission de flux mpeg sur ip et dispositif, procede de reception et recepteur correspondants
KR20130009671A (ko) 엠엠티 시스템에서의 패킷 전송 장치 및 방법, 및 패킷 수신 장치 및 방법
WO2014010830A1 (fr) Procédé et appareil de transmission et de réception de paquets dans un service de transmission hybride de mmt
Seo et al. A new timing model design for MPEG Media Transport (MMT)
WO2013055168A1 (fr) Procédé de transmission de données multimédias reliées à des informations de composition et des informations de caractéristiques de transmission par un réseau ip hétérogène
WO2013009132A2 (fr) Dispositif et procédé d'émission de paquets et dispositif et procédé de réception de paquets dans un système mmt
WO2013042998A1 (fr) Appareil et procédé pour transmettre des données multimédia pour système mmt, et appareil et procédé pour recevoir des données multimédia
WO2013042999A1 (fr) Procédé et appareil pour transmettre des données multimédia pour système mmt, et appareil et procédé pour recevoir des données multimédia
WO2015105311A1 (fr) Procédé et dispositif de réémission de paquet de service mmt, et procédé et dispositif de demande de réémission
WO2012173391A2 (fr) Procédé de transmission de données multimédia par paquets comprenant un en-tête dans lequel le surdébit est réduit au minimum
WO2014061925A1 (fr) Procédé pour transmettre de manière adaptative des données de parité fec au moyen de l'optimisation inter-couches

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12810592

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12810592

Country of ref document: EP

Kind code of ref document: A2