[go: up one dir, main page]

WO2013091010A1 - Procédés et dispositifs d'émission de contenu multimédia - Google Patents

Procédés et dispositifs d'émission de contenu multimédia Download PDF

Info

Publication number
WO2013091010A1
WO2013091010A1 PCT/AU2012/001578 AU2012001578W WO2013091010A1 WO 2013091010 A1 WO2013091010 A1 WO 2013091010A1 AU 2012001578 W AU2012001578 W AU 2012001578W WO 2013091010 A1 WO2013091010 A1 WO 2013091010A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
data sample
timestamp
received
media data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/AU2012/001578
Other languages
English (en)
Inventor
Davin MCCALL
Matthew Wesley KELLER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SAMSARA TECHNOLOGY Pty Ltd
Original Assignee
SAMSARA TECHNOLOGY Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2011905362A external-priority patent/AU2011905362A0/en
Application filed by SAMSARA TECHNOLOGY Pty Ltd filed Critical SAMSARA TECHNOLOGY Pty Ltd
Publication of WO2013091010A1 publication Critical patent/WO2013091010A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/28—Timers or timing mechanisms used in protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]

Definitions

  • the invention relates to methods and device for outputting a media data sample.
  • a media capture device that is, a video and/or audio capture device such as a video camera, microphone etc
  • the transmission can be an analogue transmission or a digital transmission.
  • media output devices that can receive digital transmissions include digital televisions, computers, smart phones etc.
  • video and/or audio of the live event captured by the media capture device is encoded into a digital stream comprising a plurality of data samples before the digital stream is transmitted to the media output device.
  • the received data samples are decoded to output or play the video and/or audio of the live event captured by the media capture device.
  • the H.264 video encoding and compression standard describes a video encoding and compression technique where each video data sample of a digital video stream is associated with a timestamp.
  • the timestamp indicates the time at which the video data sample should be output or played at the media output device relative to other video data samples of the digital video stream.
  • the output or playback rate of the digital video stream is determined by timestamps associated with the video data samples of the digital video stream.
  • video data samples and timestamps of a received digital video stream are stored in a buffer before the video data samples are output or played at the media output device.
  • the video data samples are output or played only when the buffer reaches a certain size.
  • the invention provides a method of outputting a media data sample at a media output device, comprising:
  • the invention provides a method of outputting a media data sample at a media output device, comprising:
  • the invention provides a media output device for outputting a media data sample, the media output device comprising:
  • a media receiver arranged to receive from a media capture device a media data sample and a timestamp associated with the received media data sample; a timestamp adjuster arranged to associate the received media data sample with an adjusted timestamp;
  • a media player arranged to output the received media data sample according to the adjusted timestamp associated with the received media data sample.
  • the invention provides a media output device for outputting a media data sample, the media output device comprising:
  • a media receiver arranged to receive a media data sample from a media capture device
  • a padded media generator arranged to, upon the size of the received media data sample being less than a predetermined media data sample size, generate a padded media data sample comprising padding data and the received media data sample;
  • a media player arranged to output the padded media data sample.
  • Figure 1 is a block diagram of a live event streaming system comprising a plurality of media capture devices and a plurality of media output devices;
  • Figure 2 is a functional block diagram showing the components of a media output device;
  • Figure 3 is a functional block diagram showing the components of a media modifier of the media output device
  • Figure 4 is a diagram of a media stream and a modified media stream
  • Figure 5 is a functional block diagram showing the components of a media player of the media output device
  • Figure 6 is a flowchart of a method of an embodiment
  • Figures 7 A and 7B are flowcharts showing how a timestamp adjustment is determined by the timestamp adjustment generator
  • Figure 8 is a flowchart showing how a padded media data sample is generated by a padded media data generator
  • Figure 9 is a flowchart showing how a received or padded media data sample is output by the media player.
  • Figure 10 is a diagram of a padded video data sample.
  • a media output device 30 that can be used in a live event streaming system 300 for streaming a live event such as a live music concert or a live sports game.
  • FIG. 1 is a diagram of the live event streaming system 300.
  • the system 300 is arranged to provide users with a plurality of video streams of a live event.
  • the live event system 300 may be arranged to provide other forms of media streams including an audio stream of a live event, or a video and audio stream of a live event.
  • the system 300 comprises a plurality of media capture devices 10. Each media capture device 10 is arranged to generate a video stream of the live event.
  • the media capture devices 10 are spread around the live event venue to capture the live event from different geographic positions.
  • Each of the media capture devices 10 is a video camera.
  • media capture devices other than IP video cameras may be used in other examples, and that such other media capture devices include devices that are capable of generating audio streams of the live event and devices that are capable of generating combined video and audio streams of the live event. Examples of such media capture devices include microphones, Internet Protocol (IP) web cameras etc.
  • IP Internet Protocol
  • the video streams generated by the media capture devices 10 are encoded and compressed according to the H.264 video encoding and compression standard (see “Information technology— Coding of audio-visual objects— Part 10: Advanced Video Coding" ISO/IEC 14496-10:2005 published by International Organization for Standardization (ISO)).
  • ISO International Organization for Standardization
  • the media capture devices 10 of other examples may generate media streams that are encoded and/or compressed using other encoding and/or compression techniques, or that are not encoded and/or compressed.
  • the media capture devices 10 are connected to a media server 50 which in turn is connected to a network 20.
  • the media server 50 is arranged to receive and aggregate the video streams captured by the media capture devices 10.
  • the video streams are transmitted from the media capture devices 10 to the media server 50 using the Real-time Transport Protocol (RTP) and the Real-time Streaming Protocol (RTSP) protocols.
  • RTP Real-time Transport Protocol
  • RTSP Real-time Streaming Protocol
  • the media server 50 is arranged to multicast the video streams via the network 20.
  • the video streams may be distributed in other ways, for example, by broadcasting or uni-casting the video streams from the media server 50 or directly from the media capture devices 10.
  • the system 300 also comprises a plurality of media output devices 30.
  • Each of the media output devices 30 is arranged to output one or more of the video streams of the live event distributed from the media server 50 via the network 20.
  • Examples of media output devices 30 include a smart phone, a tablet computer, a laptop computer, a MP3 player etc.
  • a media output device may be any device that is able to output or play a video stream of the live event.
  • each one of the media output devices 30 can receive one or more of the video streams captured by the media capture devices 10 via a multicast of the video stream or video streams from the media server 50 via the network 20.
  • the network 20 may comprise a number of devices (for example, wireless access points, modems, routers etc) to receive the video streams of the live event captured by the media capture devices 10 and to forward the video streams of the live event to the media output devices 30.
  • data is transmitted from the media server 50 to the media output devices 30 via one of the IEEE 802.1 1 standards.
  • other communication protocols such as Bluetooth, UMTS etc
  • Bluetooth, UMTS etc may be used in other examples to transfer data from the media server 50 to the media output devices 30.
  • the media capture devices 10 and the media output devices 30 may be connected to the network via different transmission mediums such as optical fibre, radio etc.
  • FIG. 2 is a block diagram showing the functional components of a media output device 30 in the form of a smart phone arranged to implement a software application arranged to output a video stream derived from the video stream received from the network 20.
  • the software application is implemented via Application Programming Interfaces (APIs) provided by the operating system (such as Apple iOS or Google Android) running on a processor in the smart phone.
  • APIs Application Programming Interfaces
  • the operating system such as Apple iOS or Google Android
  • the software application could be implemented in an alternative manner, for example, as a dedicated circuit.
  • the media output device 30 is arranged to receive a multicast of a video stream captured by a media capture device 10 that is formatted according to the H.264 video encoding and compression standard.
  • the video stream comprises a plurality of video data samples and a plurality of timestamps corresponding to respective ones of the video samples.
  • the video data samples is denoted by VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ... where VS ⁇ VS 2 and VS 3 represent the first, second and third video data samples respectively, and TS ⁇ TS 2 and TS 3 represent timestamps that are respectively associated with the first, second and third video data samples.
  • the software application comprises a number of modules.
  • One of the modules is a media modifier 80.
  • the media modifier 80 is arranged to generate a modified video stream based on the received video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ...
  • the media modifier 80 receives the video stream VS ⁇ TSi ; VS 2 , TS 2 ; VS 3 , TS 3 ... and outputs a modified video stream derived from the video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ... in response to receiving the video stream.
  • the modified video stream is output as a MPEG Transport Stream (MPEG-TS) comprising video data samples formatted according to the H.264 video encoding and compression standard.
  • MPEG-TS MPEG Transport Stream
  • the media player 90 is arranged to receive the modified video stream from the media modifier 80, and to output the modified video stream. In addition, the media player 90 is arranged to output a notification when the media player 90 is outputting the received video stream. In use, the media player 90 outputs the modified video stream in response to receiving the modified video stream from the media modifier 80.
  • the media player 90 outputs the modified video stream by displaying the H.264 video data samples of the modified video stream on a display.
  • the display is part of the smart phone 30. However, it is envisaged that the display may be a separate device from the smart phone 30.
  • the media output device 30 also comprises a user interface 70.
  • the user interface 70 is arranged to control the media player 90 to receive the modified video stream derived from the received video stream VS ⁇ TSi ; VS 2 , TS 2 ; VS 3 , TS 3 ... instead of the received video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ...
  • the user interface 70 is arranged to receive a notification from the media player 90, and to forward or relay the notification to the media modifier 80.
  • FIG. 3 is a block diagram showing the functional components of the media modifier 80 of the media output device 30.
  • the media modifier 80 is arranged to generate a modified video stream based on the received video stream VSi , TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ... received by the media output device 30.
  • a number of sub-modules are used to generate the modified video stream.
  • the sub-modules include a media receiver 830, a padded media generator 860, and a timestamp adjuster 890.
  • the media receiver 830 is arranged to receive the video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ...
  • the media receiver 830 is arranged to output the received video data samples VS ⁇ VS 2 , VS 3 ... to the padded media generator 860, and to output the received timestamps TS ⁇ TS 2 , TS 3 ... to the timestamp adjuster 890.
  • the padded media generator 860 is arranged to receive the video data samples of the video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ... from the media receiver 830, and to generate a padded video data sample for any video data sample that is less than a predetermined video data sample size. In use, the padded media generator 860 generates the padded video sample in response to receiving a video data sample from the media receiver 830.
  • a media size determiner 868 of the padded media generator 860 is arranged to determine the size of a video data sample.
  • the media size determiner 868 determines the size of the received video data sample by counting the number of bytes in the received video data sample.
  • a media size comparator 864 of the padded media generator 860 is arranged to determine whether or not the size of the received video data sample is less than a predetermined video data sample size.
  • the predetermined video data sample size is the largest possible video data sample size according to the H.264 standard.
  • the predetermined video data sample size may be less than or greater than the largest possible video data sample size according to the H.264 standard.
  • the media size comparator 864 determines whether or not the size of the received video data sample is less than the predetermined video data sample size by comparing the size of the received video data sample and a predetermined video data sample size.
  • the padded media generator 860 generates a padded media data sample based on the determination by the media size comparator 864. If the size of a received video data sample is less than the predetermined video data sample size, the padded media generator 860 generates a padded video data sample by adding padding data to the video data sample so that the padded video data sample is of the predetermined video data sample size.
  • the padded video data samples are denoted by VS ⁇ and VS 2 *.
  • the padding data comprises an NAL padding packet as defined in the H.264 video encoding and compression standard.
  • padding data such as an MPEG-TS "null packet" (as defined by the MPEG standard) may be used.
  • the media modifier 80 receives a notification from the user interface 70 in response to the media player 90 outputting a video stream.
  • the timestamp adjuster 890 is arranged to receive the timestamps TS ⁇ TS 2 , TS 3 ... , and to output an adjusted timestamp for each one of the received timestamps TS ⁇ TS 2 , TS 3 ... based on whether or not the media modifier 80 has received a notification.
  • the timestamp adjuster 890 outputs an adjusted timestamp in response to receiving a timestamp.
  • the timestamp adjuster 890 comprises an adjusted timestamp generator 898 and a timestamp adjustment generator 894.
  • the timestamp adjustment generator 894 is arranged to determine a timestamp adjustment. The determination of the timestamp adjustment is based on whether or not the media modifier 80 has received a notification. If the media modifier 80 has not received a notification, the timestamp adjustment generator 894 generates a timestamp adjustment based on a received timestamp (5 baS ed on received timestamp)- Otherwise the timestamp adjustment generator 894 retrieves or calculates a target timestamp adjustment (5 targ et).
  • a timestamp adjustment based on a received timestamp is generated by the timestamp adjustment generator 894 when the timestamp adjuster 890 receives a timestamp from the media receiver 830.
  • the timestamp adjustment generator 894 determines whether or not the received timestamp is associated with the first video data sample of the video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ...
  • the timestamp adjustment generator 894 determines that the received timestamp is not associated with the first video data sample, the timestamp adjustment generator 894 retrieves from memory the previous timestamp adjustment ( ⁇ ⁇ ⁇ 0 ⁇ 5 ) output by the timestamp adjustment generator 894, and generates a timestamp adjustment that is equal to the result of adding the predetermined timestamp adjustment increment (6 increment ) to the previous timestamp adjustment (6 pre vious) generated by the timestamp adjustment generator 894.
  • the predetermined timestamp adjustment increment is equal to the initial timestamp adjustment.
  • the predetermined timestamp adjustment increment can be different to the initial timestamp adjustment.
  • the timestamp adjustment generator 894 generates or retrieves a target timestamp adjustment (Starget)- First, the timestamp adjustment generator 894 attempts to retrieve a target timestamp adjustment (5 targ et)- That is, the timestamp adjustment generator 894 checks whether or not a target timestamp adjustment (5 tar get) has previously been generated for the current video stream. If so, the timestamp adjustment is equal to the target timestamp adjustment (5 tar get) in memory. Otherwise, the timestamp adjustment generator 894 generates a target timestamp adjustment (5 targ et) based on the amount of time the media player 90 takes to output a video data sample.
  • Starget target timestamp adjustment
  • the timestamp adjustment generator 894 calculates the target timestamp adjustment (5t arget ) by subtracting the time at which the media player 90 receives the video data sample from the time at which the media player 90 outputs the video data sample, and normalizing the result based on the clock rate of the received timestamps.
  • the adjusted timestamp generator 898 is arranged to generate an adjusted timestamp for each one of the timestamps of the video stream VS ⁇ TS ⁇ VS 2 , TS 2 ; VS 3 , TS 3 ...
  • the generation of the adjusted timestamp is based on the received timestamp and the timestamp adjustment generated by the timestamp adjustment generator 894.
  • the adjusted timestamp generator 898 generates the adjusted timestamp by subtracting the received timestamp by the timestamp adjustment.
  • the adjusted timestamps are denoted by TS ⁇ , TS 2 * and TS 3 *.
  • the media modifier 80 is arranged to generate a modified video stream that includes the timestamps generated by the time adjuster 890, and the padded video data samples generated by the padded media data sample generator 860.
  • the media modifier 80 generates the modified video stream by (i) associating an adjusted timestamp with a padded video data sample, or (ii) associating an adjusted timestamp with a received video data sample, (i) is carried out by the media modifier 80 when the timestamp adjuster 890 generates an adjusted timestamp generated and the padded media generator 860 generates a padded video data sample, and (ii) is carried out by the media modifier 80 when the media modifier 80 receives an adjusted timestamp generated by the timestamp adjuster 890 but the padded media generator 860 does not produce a padded video data sample.
  • the media modifier 80 associates an adjusted timestamp with a received video data sample by removing the timestamp associated with the video data sample, and replacing the removed timestamp with the adjusted timestamp.
  • the media modifier 80 associates an adjusted timestamp with a padded video data sample by removing the received video data sample and the timestamp associated with the received video data sample, and replacing the removed video data sample and timestamp with the padded video data sample and adjusted timestamp.
  • Figure 4 is a diagram of a video stream received by the media modifier 80 and a modified video stream that is generated by the media modifier 80 based on the received video stream.
  • VS ⁇ VS 2 and VS 3 represents respectively the first, second and third video data sample of the received video stream.
  • TS 1 represents the timestamp that is associated with VS 1 .
  • TS 2 represents the timestamp that is associated with VS 2 .
  • TS 3 represents the timestamp that is associated with VS 3 .
  • the modified video stream comprises two padded video data samples VS ⁇ and VS 2 *, and a received video data sample VS 3 .
  • the size of each one of the video data samples VS ⁇ , VS 2 * and VS 3 is equal to the predetermined video data sample size (that is, the largest possible video data sample size according to the H.264 standard).
  • Figure 5 is a block diagram showing the functional components of the media player 90.
  • the media player 90 is arranged to receive the modified video stream from the media modifier 80, and to output the modified video stream by playing the video data samples of the modified video stream according to the adjusted timestamps of the modified video stream.
  • the media player 90 comprises a buffer 940 arranged to store the video data samples and timestamps of the modified video stream.
  • the media player 90 plays the video data samples stored in the buffer 940 when the buffer reaches a predetermined buffer size
  • the media player 90 comprises a buffer monitor 980 which is arranged to generate a notification that the media player 90 is playing the video data samples in the buffer 940 when the buffer 940 reaches the predetermined buffer size.
  • whether or not the video data samples stored in the buffer 940 is played by the media player 90 may depend on a condition other than the amount of video data samples stored in the buffer (that is, the buffer size).
  • whether or not the video data samples stored in the buffer 940 is played by the media player 90 may depend on the difference in the timestamp values of the first and last (most recently added) samples stored in the buffer 940, in that it can be recognized the buffer contains samples spanning a duration which is greater than a specific threshold.
  • the buffer monitor 980 would be arranged to generate a notification that the media player 90 is playing the video data samples in the buffer 940 only upon a determination that the condition has occurred.
  • the user interface 70 is arranged to receive the notification from the buffer monitor 980, and to forward the notification to the media modifier 80 in response to receiving the notification from the buffer monitor 980.
  • Figure 6 is a flowchart of a method of an embodiment.
  • the media output device 30 receives a video stream comprising a video data sample and a timestamp associated with the video data sample.
  • the timestamp adjustment generator 890 of the timestamp adjuster 890 generates a timestamp adjustment. Details of step 120 are provided in Figures 7A and 7B.
  • the adjusted timestamp generator 898 of the timestamp adjuster 890 generates an adjusted timestamp based on the received timestamp and the timestamp adjustment generated by the timestamp adjustment generator 894.
  • a padded video data sample is generated by the padded media data generator 860 if required. Details of step 150 are provided in Figure 8.
  • the media modifier 80 associates the adjusted timestamp with the padded video data sample. Otherwise, the media modifier 80 associates the adjusted timestamp with the video data sample received by the media output device 30. The media modifier 80 then outputs the received or padded video data sample and the adjusted timestamp associated with the received or padded video data sample to the media player 90.
  • the media player 90 is controlled by the user interface 70 to receive the modified video stream from the media modifier 80.
  • the media player 90 stores the received or padded video data sample and the adjusted timestamp associated with the received or padded video data sample in the buffer 940.
  • the steps 1 10, 120, 130, 150, 160 and 180 are repeated for subsequent video data samples and timestamps of the video stream received by the media output device 30.
  • the media player 90 outputs the video data samples of the buffer 940 according to the timestamps associated with the respective ones of the video data samples of the buffer 940 at step 190.
  • the received or padded video data sample is output by the media player 90 according to the adjusted timestamp associated with the received or padded video data sample. Details of step 190 are provided in Figure 9.
  • Figure 7A is a flowchart showing how a timestamp adjustment is generated by the timestamp adjustment generator 894 at step 120.
  • the timestamp adjustment generator 894 determines whether or not a notification has been received from the user interface 70. If the timestamp adjustment generator 894 determines that a received or padded video data samples has not been received from the user interface 70, the timestamp adjustment generator 894 generates a timestamp adjustment based on the received timestamp (5 base d on received timestamp) at step 122, and the timestamp adjustment generator 894 outputs a timestamp adjustment that is equal to the timestamp adjustment based on the received timestamp (5 base d on received timestamp) at step 128.
  • step 122 Details of step 122 are provided in Figure 7B. If the timestamp adjustment generator 894 determines that a received or padded video data samples has been received from the user interface 70, the timestamp adjustment generator 894 retrieves or generates a target timestamp adjustment (5 tar get) at step 123. Specifically, the timestamp adjustment generator 894 retrieves a target timestamp adjustment (5 targ et) from memory if the target timestamp adjustment (5 tar get) was previously generated by the timestamp adjustment generator 894 with respect to a previous video data sample of the received video stream.
  • a target timestamp adjustment (5 targ et
  • the timestamp adjustment generator 894 generates a target timestamp adjustment ( ⁇ ⁇ 3 ⁇ 9 ⁇ by subtracting the time at which the media player 90 received the video data sample from the time at which the media player 90 output the video data sample, and normalizing the result based on the clock rate of the received timestamps.
  • the timestamp adjustment generator 894 subsequently outputs a timestamp adjustment that is equal to the target timestamp adjustment (5 ta rget)-
  • Figure 7B is a flowchart showing how a timestamp adjustment based on a received timestamp is generated by the timestamp adjustment generator 894 at step 1 22.
  • the timestamp adjustment generator 894 determines whether or not the received timestamp is a timestamp that is associated with the first video data sample of the received video stream. If the timestamp adjustment generator 894 determines that the received timestamp is associated with the first video data sample of the received video stream, the timestamp adjustment generator 894 retrieves an initial timestamp adjustment (6 0 ) from memory at step 204, and outputs a timestamp adjustment that is equal to the initial timestamp adjustment ( ⁇ 0 ) at step 208.
  • the timestamp adjustment generator 894 determines that the received timestamp is not associated with the first video data sample of the received video stream, the timestamp adjustment generator 894 retrieves a timestamp adjustment increment (5 in crement) from memory at step 202, and a previous timestamp adjustment (5 preV i 0 us) generated by the timestamp adjustment generator 894 from memory at step 202. Then, the timestamp adjustment generator 894 adds the timestamp adjustment increment (5 incr em8nt) and the previous timestamp adjustment (5 pre vious), and outputs a timestamp adjustment that is equal to the sum of timestamp adjustment increment increment) and the previous timestamp adjustment (6 preV i 0 us) at step 209.
  • Figure 8 is a flowchart showing how a padded media data sample is generated by a padded media data generator 860 at step 1 50.
  • the media size determiner 864 of the padded media data generator 860 determines the length of the video data sample (L) received by the media output device 30.
  • the media size comparator 868 of the padded media generator 860 determines whether or not the length of the received video data sample (L) is less than a predetermined video data sample size (L*). If the media size comparator 868 determines that the length of the received video data sample (L) is not less than a predetermined video data sample size (L*), the padded media data generator 860 does not generate a padded video data sample.
  • the padded media data generator 860 determines that the length of the received video data sample (L) is less than a predetermined video data sample size (L*)
  • the padded media data generator 860 generates a padded video data sample having a length L* by adding padding data.
  • the padding data comprises a padding NAL packet.
  • the padded media data generator 860 generates a padded video data sample, the padded video data sample is associated with an adjusted timestamp. Otherwise, the adjusted timestamp is associated with the video data sample received by the media output device 30. The received or padded video data sample and the adjusted timestamp are then transmitted from the media modifier 80 to the media player 90 and stored in the buffer 940 of the media player 90.
  • Figure 9 is a flowchart showing how the received or padded video data sample stored in the buffer 940 is output by the media player 90 of this example.
  • the buffer monitor 980 determines the number of video data samples in the buffer 940 (N bUff er)- If the buffer monitor 980 determines that the number of video data samples in the buffer 940 (N b u ff er) is equal to a predetermined number of video data samples, the media player 90 outputs the video data samples stored in the buffer 940 according to the timestamps associated with the respective video data samples. In addition, the media player 90 outputs a notification to the user interface 70 which in turn relays the notification to the media modifier 80. If the buffer monitor 193 determines that the number of video data samples in the buffer 193 (N buffer ) is not equal to a predetermined number of video data samples, the media player 90 waits for the next video data sample and timestamp.
  • the media modifier 80 comprises both a timestamp adjuster 890 and a padded media generator 860.
  • the media modifier 80 may comprise only the timestamp adjuster 890 or only the padded media generator 860. That is, an alternative example of the live event streaming system 300 may be arranged to adjust the timestamps of the video data samples of a received video stream but not to pad the video data samples of the received video stream, or to pad the video data samples of a received video stream but not to adjust the timestamps of the video data samples of the received video stream.
  • the live event streaming system 300 comprises a media server 50 connected to plurality of IP cameras 10. Each one of the IP cameras is arranged to generate a video stream encoded and compressed according to the H.264 standard, and to transmit the video stream to the media server 50.
  • the media server 50 is arranged to distribute the video streams using IP multicasting.
  • the live event streaming system 300 also comprises a number of media output devices including one in the form of a smart phone 30.
  • the smart phone 30 is configured to receive the video stream from the media server 50 via a wireless local area network 20.
  • the smart phone 30 implements a number of software modules to modify the received video stream, and to play the modified video stream.
  • the modules include a media modifier in the form of an internal HTTP server 80, a user interface in the form of a GUI 70, and a media decoder/player 90 provided by the operating system running on the smart phone 30.
  • the GUI 70 is operated by a user to initiate the media decoder/player 90, and requests the media decoder/player 90 to play the modified video stream via a URL referring to a virtual MPEG-TS file to be served by the internal HTTP server 80.
  • the modified video stream is generated by the internal HTTP server 80 by generating an adjusted timestamp for each one of the video data samples of the received video stream, generating a padded video data sample for any one of the video data samples of the received video stream that is less than a predetermined video data sample size, and associating each adjusted timestamp with a received or padded video data sample.
  • Each padded video data sample is generated by padding or appending the received video data sample with a H.264 NAL packet such that the size of the padded video data sample is equal to the predetermined video data sample size.
  • H.264 NAL packet increases the size (in bytes) of the video sample, but that the addition does not affect the semantics of the video data sample.
  • the structure of a padded video data sample is illustrated in Figure 10. In this example, the size of the video data sample received by the smart phone 30 is 15,000 bytes, and the predetermined video data sample size is 20,000 bytes.
  • the video data packet received by the smart phone 30 consists of two NAL packets separated by a marker sequence.
  • the marker sequence is a 4-byte packet defined in the H.264 video compression standard that is used to separate the two NAL packets.
  • the padded video data sample comprises the received video data packet and padding data in the form of a marker sequence and a padding NAL packet comprising 4996 bytes.
  • an adjusted timestamp is generated by the internal HTTP server 80 for each one of the video data samples of the received video stream.
  • Each adjusted timestamp is generated by generating a timestamp adjustment, and generating the adjusted timestamp based on the timestamp adjustment and the received timestamp.
  • the timestamp adjustment is generated by retrieving an initial timestamp adjustment from memory ( ⁇ 0 ), and the adjusted timestamp is generated by subtracting the received timestamp with the timestamp adjustment.
  • the timestamp adjustment is generated by summing the previous timestamp adjustment with a timestamp adjustment increment until the internal HTTP server 80 receives a notification from the GUI 70, and the adjusted timestamp is generated by subtracting the timestamp adjustment from the received timestamp.
  • the timestamp increment 5increment 500.
  • the modified video stream comprises a plurality of received or padded video data samples, and a plurality of adjusted timestamps associated with respective ones of the received or padded video data samples.
  • the modified video stream is transmitted from the media server 50 via the wireless local area network to the smart phone 30.
  • the received or padded video data samples and adjusted timestamps are received and stored by the media player 90 in a buffer 940.
  • the media player 90 plays the video data samples in the buffer 940 according to the adjusted timestamps associated with the video data samples, and a notification is output by the media player 90 to the GUI 70.
  • the notification is then forwarded by the GUI 70 to the internal HTTP server 80.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne un procédé d'émission d'un échantillon de données multimédias au niveau d'un dispositif d'émission de contenu multimédia, lequel procédé consiste à : recevoir, à partir d'un dispositif de capture de contenu multimédia, un échantillon de données multimédias et un horodatage associé à l'échantillon de données multimédias reçu ; associer l'échantillon de données multimédias reçu à un horodatage ajusté ; et émettre l'échantillon de données multimédias reçu selon l'horodatage ajusté associé à l'échantillon de données multimédias reçu. Un procédé d'émission d'un échantillon de données multimédias au niveau d'un dispositif d'émission de contenu multimédia, consiste à : recevoir un échantillon de données multimédias à partir d'un dispositif de capture de contenu multimédia ; lorsque la dimension de l'échantillon de données multimédias reçu est inférieure à une dimension d'échantillon de données multimédias prédéterminée, générer un échantillon de données multimédias rempli comprenant des données de remplissage et l'échantillon de données multimédias reçu ; et émettre l'échantillon de données multimédias rempli.
PCT/AU2012/001578 2011-12-21 2012-12-21 Procédés et dispositifs d'émission de contenu multimédia Ceased WO2013091010A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2011905362 2011-12-21
AU2011905362A AU2011905362A0 (en) 2011-12-21 Media output methods and devices

Publications (1)

Publication Number Publication Date
WO2013091010A1 true WO2013091010A1 (fr) 2013-06-27

Family

ID=48667513

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2012/001578 Ceased WO2013091010A1 (fr) 2011-12-21 2012-12-21 Procédés et dispositifs d'émission de contenu multimédia

Country Status (1)

Country Link
WO (1) WO2013091010A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016074615A1 (fr) * 2014-11-12 2016-05-19 Mediatek Inc. Réglage dynamique de la fréquence d'échantillonnage de trames vidéo
US20240015043A1 (en) * 2019-12-31 2024-01-11 Dish Network Technologies India Private Limited Dynamic low latency mode for a digital video production system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7545794B2 (en) * 2003-08-14 2009-06-09 Intel Corporation Timestamping network controller for streaming media applications
US20100177791A1 (en) * 2007-06-08 2010-07-15 Adrian Charles Turner Timestamp conversion
US7890985B2 (en) * 2006-05-22 2011-02-15 Microsoft Corporation Server-side media stream manipulation for emulation of media playback functions

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7545794B2 (en) * 2003-08-14 2009-06-09 Intel Corporation Timestamping network controller for streaming media applications
US7890985B2 (en) * 2006-05-22 2011-02-15 Microsoft Corporation Server-side media stream manipulation for emulation of media playback functions
US20100177791A1 (en) * 2007-06-08 2010-07-15 Adrian Charles Turner Timestamp conversion

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016074615A1 (fr) * 2014-11-12 2016-05-19 Mediatek Inc. Réglage dynamique de la fréquence d'échantillonnage de trames vidéo
US20240015043A1 (en) * 2019-12-31 2024-01-11 Dish Network Technologies India Private Limited Dynamic low latency mode for a digital video production system
US12407889B2 (en) * 2019-12-31 2025-09-02 Dish Network Technologies India Private Limited Dynamic low latency mode for a digital video production system

Similar Documents

Publication Publication Date Title
US10104408B1 (en) Synchronous and multi-sourced audio and video broadcast
TWI729997B (zh) 傳輸經寫碼音訊資料
JP6694022B2 (ja) 送信方法および送信装置
US9769236B2 (en) Combined broadcast and unicast delivery
US20210084389A1 (en) System and method for synchronizing metadata with audiovisual content
CN104023250B (zh) 基于流媒体的实时互动方法和系统
US20130294747A1 (en) Content playing device, content playing method, distribution system, content playing program, recording medium, and data structure
US10503460B2 (en) Method for synchronizing an alternative audio stream
US20090055549A1 (en) Content Reproduction Apparatus, Content Reproduction Method, and Program
CN105325005B (zh) 内容供应装置、内容供应方法、存储介质、终端装置以及内容供应系统
CN107018466A (zh) 增强音频记录
KR20120114016A (ko) 사용자 컨텐츠를 외부 단말기에서 네트워크 적응적으로 스트리밍하는 방법 및 장치
US20230188808A1 (en) Inserting advertisements in atsc content
CN102550038A (zh) 播放媒体文件的方法、终端、服务器和系统
WO2015064383A1 (fr) Dispositif et procédé de transmission, dispositif et procédé de réception
TW202114432A (zh) 使用帶內元資料作為存取參考指紋的基礎，以促進與內容相關的動作
KR102085192B1 (ko) 렌더링 시간 제어
WO2008028367A1 (fr) Procédé permettant de réaliser des pistes audio multimédia pour un système de diffusion multimédia mobile
WO2013091010A1 (fr) Procédés et dispositifs d'émission de contenu multimédia
WO2008031293A1 (fr) Procédé de lecture rapide de canaux de diffusion multimédia
RU2527732C2 (ru) Способ озвучивания видеотрансляции
CN100544448C (zh) 一种移动多媒体网络的时钟同步系统
KR101999235B1 (ko) Mmtp기반 하이브리드 브로드캐스트 브로드밴드 서비스 제공 방법 및 시스템
JP2004140552A (ja) マルチキャストシステム，ストリーミングサーバ，クライアント装置，プログラム，およびストリーミングサーバの送信方法
KR20160074310A (ko) 하이브리드망에서의 스트리밍 방법 및 그 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12860655

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12860655

Country of ref document: EP

Kind code of ref document: A1