WO2025145169A1

WO2025145169A1 - Method and apparatus to support synchronized viewing and ancillary digital interaction for live video streaming events

Info

Publication number: WO2025145169A1
Application number: PCT/US2024/062292
Authority: WO
Inventors: Catherine MULLEN; Mark Peters
Original assignee: Wearexalt LLC
Current assignee: Wearexalt LLC
Priority date: 2023-12-29
Filing date: 2024-12-30
Publication date: 2025-07-03
Anticipated expiration: 2026-06-29

Abstract

A system and method of cost effectively delivering content for synchronized consumption of live video, audio, chat and transactions in real time without manipulation of the content of the stream.

Description

METHOD AND APPARATUS TO SUPPORT SYNCHRONIZED VIEWING AND

ANCILLARY DIGITAL INTERACTION FOR LIVE VIDEO STREAMING EVENTS

BACKGROUND

[0001] Content streaming is used to transmit content to one or more recipients. When transmitting content to a plurality of recipients, the data transmitted takes different paths in a network for delivery to each of the plurality of recipients. This can cause the content to be delivered to each recipient at different times due to differences in path length and path delays and can result in each recipient viewing the content at different times. The difference in playback among the plurality of recipients can cause issues with real-time discussion of the content among the plurality of recipients. The difference in playback can prevent meaningful real-time discussion among the plurality of recipients regarding the content since the recipients are viewing the content at different times.

FIELD OF THE INVENTION

[0002]The present disclosure relates generally to streaming and, more particularly, synchronizing stream viewing and ancillary digital interaction for live video streaming events.

SUMMARY

[0003]According to aspects of the present disclosure, the techniques described herein relate to a method for supporting synchronized viewing of streaming video, including: receiving an indication of a delay between a send time and an arrival time of an encrypted video feed transmitted from a content delivery network to an HTTP live streaming (HLS) player; adjusting a playhead position on the HLS player based on the delay; and transmitting a synchronized feed to clients.

[0004] In further aspects, the techniques described herein relate to a method, wherein the send time and the arrival time are determined based on a central clock. In some aspects, the techniques described herein relate to a method, wherein the receiving, adjusting, and transmitting are performed repeatedly. In some aspects, the techniques described herein relate to a method, wherein the adjusting includes advancing the playhead position on the HLS player. In some aspects, the techniques described herein relate to a method, wherein the adjusting includes delaying the playhead position on the HLS player. In some aspects, the techniques described herein relate to a method, including receiving an unencrypted video feed transmitted from the content delivery network at a commentator interface. In some aspects, the techniques described herein relate to a method, including transmitting a commentator feed from the commentator interface to the HLS player. In some aspects, the techniques described herein relate to a method, wherein transmitting the synchronized feed to the clients includes synchronizing the encrypted video feed and the commentator feed. In some aspects, the techniques described herein relate to a method, wherein transmitting the commentator feed from the commentator interface to the HLS player includes determining whether a streaming video event requires an ultra low-latency (ULL) commentator feed. In some aspects, the techniques described herein relate to a method, including transmitting a ULL commentator feed to the HLS player based on a positive determination. In some aspects, the techniques described herein relate to a method, including receiving an unencrypted video feed transmitted from the content delivery network at a production interface. In some aspects, the techniques described herein relate to a method, including transmitting production data from the production interface to the HLS player, wherein the production data includes one or more real-time interactive features. In some aspects, the techniques described herein relate to a method, wherein transmitting the synchronized feed to the clients includes synchronizing the encrypted video feed, the commentator feed, and the production data.

[0005]According to additional aspects of the present disclosure, the techniques described herein relate to a method for optimizing latency of streaming video, including: transmitting a latent streaming video feed from a content delivery network to a plurality of clients; identifying a group of clients requiring an ultra-low latency (ULL) video streaming feed; and transmitting a ULL streaming video feed from the content delivery network to the identified group of clients; wherein identifying the group of clients is based on a triggering event relating to the group of clients. [0006] In further aspects, the techniques described herein relate to a method, wherein the identifying is performed using an ad trigger. In some aspects, the techniques described herein relate to a method, including determining whether the triggering event is active. In some aspects, the techniques described herein relate to a method, including switching the identified group of clients from the latent video streaming feed to the ULL video streaming feed based on a determination that the triggering event is active. In some aspects, the techniques described herein relate to a method, including switching the identified group of clients from the ULL video streaming feed to the latent video streaming feed based on determination that the triggering event is not active.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 shows a hybrid schematic of a system of the present disclosure including components and signals transmitted therebetween, according to the present disclosure;

[0008] FIG. 2 shows a schematic of a network which is used to stream content according to one embodiment;

[0009] FIG. 3A shows a high-level overview of process steps performed by the system of the present disclosure that essentially enables an integrated external server/bot into the player fully synchronized with the stream according to one embodiment;

[00010] FIG. 3B shows a simplified workflow of the high-level overview shown in FIG3A;

[00011] FIG. 4 shows a schematic of the Control Plane according to one embodiment;

[00012] FIG. 5A is a network diagram illustrating generation of multiple streams from a source feed, according to the present disclosure;

[00013] FIG. 5B is a network diagram illustrating return paths for first-party data to rightsholders and media licensees, according to the present disclosure;

[00014] FIG. 5C is an exemplary graphical user interface generated by the system of the present disclosure; and

[00015] FIG. 6 shows a high-level diagram of a computer according to an embodiment. DETAILED DESCRIPTION

[00017] Streaming applications are subject to significant network latencies and other delays. As such, users of a given streaming application (e.g. streaming video) would not typically see the same thing at the same time. If these users are also engaged in a chat session or other real-time communication while using the streaming application, it is difficult to integrate a chat or have a meaningful conversation and maintain the quality of the video across a plurality of streams at the same time. That is, one participant may be commenting on something that another participant has not yet seen or if the stream is synchronized then those with a high quality I low latency device I distribution method will have to suffer from a downgraded video quality experience to be able to synchronize with those with a lower quality I higher latency device I distribution method. This is important in use cases such as, for example, live sports and live sports betting, where “spoilers” or lower video quality are consistent problems due to this issue.

[00018] There are several synchronization technologies and methodologies that are available in the market but there are a number of deficiencies as outlined herein.

[00019] Certain currently available live stream synchronization technologies and methodologies manipulate the stream in order to align streams originating from different sources, which inhibits high quality viewing experiences reliably at scale. [00020] Some synchronization technologies work as a software development kit (“SDK”) in a pre-existing streaming app, which requires significant investment and revenue sharing with application stores (e.g., Google Play Store, Apple App Store). According to another example, a discord server is integrated into a streaming player, which allows the player to be distributed via a link, rather than requiring an SDK within an app. As those of ordinary skill in the art will understand, Discord is a platform which provides voice over internet protocol (“VoIP”), instant messaging, and other social functionalities. Users have the ability to communicate with voice calls, video calls, and text messaging, as well as to share media and other files in private chats or as part of communities called servers.

[00021] Lastly, there are some use cases where synchronizing a plurality of streams at an ultra-low latency (“ULL”, e.g., less than 2 milli-seconds) is crucial (e.g., sports betting, fantasy sports, auctions) and a number of currently available solutions do cater to this scenario; however, there are many use cases where there is no need to invest the high cost required for ULL (e.g. integrating chat or community features where users simply want to experience the event together, at the same time).

Currently, there are no solutions which drive a decision tree, in real-time, to shift a user to the most cost-efficient latency path, as needed.

[00022] Video streaming relies on the internet infrastructure, which is designed for transferring files between locations. While opening a channel at scale may be feasible for audio, it remains impractical for video due to substantially larger data requirements.

[00023] Current streaming solutions involve dividing a video stream into smaller segments, such as packets, and transmitting them across the network. These packets are duplicated and distributed among various entities, with each entity passing the packets to another until they reach their final destination. Due to the inherent nature of data transmission, these packets will inevitably arrive at their respective final destinations at different times, resulting in the video stream being out of sync among the various entities receiving the video stream.

[00024] Out of sync video streams is a problem faced in live sports streaming cases today, as the delay can be 30 seconds or more. Factors contributing to the magnitude of the delay include, but are not limited to, distance between a source of the video feed and an endpoint, device types, network types, and the like. That is why live streaming use cases for sports or music concerts, for example, are especially challenging.

[00025] Examples exist in the market that have utilized methodologies such as an external clocked system to synchronize concurrent streams of a live video feed.

[00026] For example, U.S. Patent No. 10,601 ,914 presents a synchronization methodology that ensures a plurality of consumers synchronously consume an ordered data stream by adjusting a quality of a data chunk or a latency of the ordered data stream upon determining that a consumer has received a substantial percentage of data chunks late.

[00027] U.S. Patent No. 9,380,327 describes a method that allows video content to be transmitted over links with unknown or variable bandwidth characteristics. To accommodate different bandwidths that may be available, the video content may be offered in several formats, each with different bandwidth requirements. If the quality of a communication link degrades during delivery, such that the selected format is no longer supported, a server may stop transmitting a currently selected format and select a format with lower bandwidth requirements for transmission. Varying the format of a video stream overtime is known as adaptive streaming. A need exists for systems and methods that enable adaptive streaming that are modular, scalable, and efficient.

[00028] U.S. Patent No. 9,953,347 describes a method of dynamically binding supplemental content to live video content and includes receiving the live video content at a device and identifying a description of the live video content. The method also includes obtaining the supplemental content based on the description, where the supplemental content provides additional information about one or more products or services related to the live video content. The method further includes dynamically binding the supplemental content to the live video content and positioning the supplemental content in association with the live video content using a supplemental interactive display. For example, in the case of a video description being, “live soccer game,” it will dynamically generate supplemental content relating to soccer .

[00029] U.S. Patent No. 9,344,754 describes a method for presenting advertisements for commercial products in video productions, whereby the commercial product is placed in the video production as an element of the video production. A viewer is enabled to interact with the video production to select the product. Information is then displayed about the selected product; and the viewer is enabled to purchase the selected product.

[00030] The synchronization systems and methods disclosed herein address the issues described above by adjusting a playhead position of a client player instead of manipulating the video data stream to achieve the synchronized delivery of video streams. Those of ordinary skill in the art should understand that, as used herein, the playhead is not a physical playhead, such as the needle of a record player, but is a representation indicating what part of a video is being played. According to some aspects of the present disclosure, the playhead can be represented on a graphical user interface as an arrow, line, or other element that can be used to indicate a position along an axis representing a timeline related to a video stream. According to other aspects of the present disclosure, the playhead may not be graphically represented on a user interface and may only be used by the system as a (e.g., calculated or determined) reference point. For example, the systems and methods described herein can use the playhead as reference point to indicate a current playback position of video stream, without displaying same playhead to a user. The synchronization can be performed continuously via a series of time stamp pings to an external clock. In so doing, the systems and methods of the present disclosure are not adapting or reducing the quality of a stream to delay and align feeds to the same latency. Instead, client player playhead positions can be continuously adjusted to constantly adapt and align the streams to a common latency. According to some aspects of the present disclosure, the client leverages a central clock signal and timing data embedded into the video stream via datetime stamp to calculate a deviation from a standard. For example, a particular amount of delay may be acceptable and considered standard. The client then constantly shifts the video playhead position to align to that standard position thereby achieving synchronization (see FIG. 1).

[00031] This is important in use cases where stream quality and synchronization are essential, like live sports, for example, where “spoilers” caused by users receiving the stream at different times means they cannot watch remotely with theirfriends. It also enables synchronization of the stream, interactions, and transactions, as well as other features such as alternative live commentary streams or chat functionality (e.g., “audio rooms” and 3rd party chat integrations being active at the same time at a high quality, reliably, efficiently and at scale as shown and described in connection with FIGS. 2, 3A and 3B).

[00032] When a user enters an experience, such as, for example, by selecting a pair of commentators out of many (e.g. the manning cast), the user has the same experience such that the video, the audio streams, the data and the chat are synced to what the user is seeing on the screen at high quality. Accordingly, the video experience for everyone in a collective chat room and commentary room is unified. Everyone interacting is watching the same feed, not a manipulated version of that feed.

[00033] The stream, the interactive experiences, including integrated chat, and transactions are synced across all platforms in a live stream environment. [00034] Also, unlike other solutions in the market, there is no dedicated (e.g., mobile or desktop) application required. A such, a link can be sent to another user and they will be completely synced up at high quality, reliability and at scale.

[00035] According to some aspects of the present disclosure, rightsholders can direct users to the appropriate path based on whether ultra-low latency is required (e.g., betting) or whether just synchronization is required to enable chat, alternative audio, interactive graphics and transactions to be aligned. For example, users in specific “commentator rooms,” where ultra-low latency is required (e.g., betting or fantasy use cases), can be separated out from users interested in community use cases where lower cost I higher latency can be tolerated (e.g., as long as a user is in a “community room,” the users are fully synchronized, so it does not matter if a user has a different stream outside of the community). According to some aspects of the present disclosure, a "community room" can be a virtual space on a platform where members of a given community can gather, interact, and participate in discussions or activities, similar to how a physical community room would function, but accessible through the internet, whereas in betting use cases, users could arbitrage by having a separate, faster feed (e.g. traditional broadcast television vs. over-the-top delivery, so providing ultra-low latency is critical).

[00036] FIG. 1 shows a hybrid schematic that includes components and signals transmitted between components according to embodiments of the present disclosure (hereinafter, “system 1”). The hybrid schematic of system 1 represents SyncPerfect according to one embodiment. The backend function of SyncPerfect provides a single, central, clock pulse generator and a protocol for providing the central clock signal to a client. The client function leverages the central clock signal and timing data embedded into the video stream via datetime stamp to calculate the deviation from the standard (i.e., how much of a delay there is from sending to receiving and whether that delay exceeds a threshold/standard), and then constantly shifts the video playhead position to align to that standard position thereby achieving synchronization. Those of ordinary skill in the art will understand that the client, as referred to herein, can comprise instructions stored on a computer readable media that, when executed by a processor, cause the processor to perform the functions described herein (e.g., software). Accordingly, the client, as well as other components and modules disclosed herein can be applications executed by a processor, such as, but not limited to, those of a desktop computer, a mobile device, a cloud-based server (e.g., for a web application), or the like. Furthermore, components and modules disclosed herein can also be programmed for, and executed within, one or more operating system environments, such as, but not limited to, Windows, Mac OS, Android, iOS, and Chrome OS.

[00037] A central external clock is used as a common reference for all clients and operations involving synchronization (e.g., a source of truth).

[00038] It is not uncommon to delay feeds from different acquisition sources to synchronize multiple concurrent streams to the highest latency as that is more cost effective for use cases that do not require streams to be synchronized to the lowest latency (e.g. chat integration, transactions, multiple video and audio streams).

[00039] What is different about the present disclosure is that instead of manipulating the video data stream to achieve the synchronized delivery of chunks of video, the systems and methods of the present disclosure instead adjust the playhead position in the client player, as shown in the embodiment of FIG. 1 .

[00040] With continued reference to FIG. 1 , a signal is acquired and encoded at satellite truck 10 and transmitted to content delivery network (“CDN”) 20. External clock 30 provides a central, external clock signal that is used by all clients. A signal (e.g., a ping) is sent to cloud-based compute service 40 (e.g., Amazon Web Services (“AWS”) Lambda) including the program datetime stamp or a metadata container (e.g., ID3) along with the time the signal was sent as shown by arrow 80. Cloudbased compute service 40 determines the delta between the send time and the arrival time of streamed content and transmits this information to HTTP live streaming (“HLS”) player 50 as shown by arrow 84. Media segments are transmitted from CDN 20 to HLS player 50 as shown by arrow 82 and displayed by HLS player 50 on display 60. The delta information is used by display 60 to shift playhead positions on HLS player 50 to align positions to common external clock 30 as described in caption 88. As shown by arrow 86, these steps are repeated to constantly adapt playback according to external clock 30. Synchronized feed 90 is then transmitted to clients 70 (e.g., users). It should be noted that one or more of an authorization service (e.g., Open Authorization, or“OAuth”), Chat integration (e.g., Discord server) application programing interface (“API”), commentator panel and producer panel can all be synchronized and integrated using, for example, but not limited to, a separate websocket protocol with a player used by clients 70.

[00041] FIG. 2 shows a schematic of network 100 which is used to stream content according to embodiments of the present disclosure . Program feed 102 provides content that is generated during production. For example, content could be a live event, pre-recorded event, or other media. During production, content can be edited as desired. Program feed 102 transmits content to distribution/origination site 104. In one embodiment, distribution 104 is a device for receiving and delivering content. The functions of distribution 104 can also be performed by a service provider such as Amazon Web Services Encoder, Origin, or another content delivery network.

[00042] Distribution 104 transmits content to producer panel 106 which can modify the content and transmit it to socket management device 108. For example, according to one embodiment of the present disclosure, socket management is provided by Ably global websocket platform using an Ably message protocol. The Ably websocket platform is highly scalable and provides high throughput socket management. According to another exemplary embodiment of the present disclosure, socket management can be provided Socket. io. Socket management device 108 also transmits content to commentator panel 110 which is also in communication with service 112. According to aspects of the present disclosure, service 112, is a talent recruitment service providing for talent (e.g., TV personalities, journalists, social media influencers, etc.) identification and training for partners to use as commentators on the platform.

[00043] Service 112 is also in communication with Cloudflare 114, or a similar service, via an application programming interface according to some embodiments of the present disclosure.

[00044] Central clock 116 generates a timing signal and transmits the timing signal to data track processor 118. Distribution 104 transmits program date and time stamp or ID3 to data track processor 118.

[00045] Media source extensions/Multimedia extensions (“MSE/MME”) enabled player page 120 receives program segments from distribution 104 and receives data from Cloudflare 114. According to some embodiments, Cloudflare 114 transmits Web real-time communications (“WebRTC”) WebRTC-HTTP egress protocol (“WHEP”) to MSE/MME enabled player page 120. According to alternative embodiments of the present disclosure, AWS Kinesis can be used to transmit the WebRTC-HTTP egress protocol to the MSE/MME enabled player page 120. MSE/MME enabled player page 120 is also in communication with an authentication login API 122 and/or application 124. The authentication login API 122 and/or application 124 can be components of the system 100, or the authentication login API 122 and/or application 124 can be services provided by a third party (e.g., single sign on (“SSO” services provided by way of social logins). For example, according to one embodiment, the login API 122 can be a Discord SSO login API and the authentication application 124 can be a Discord SSO App, allowing a Discord user to use a single sign on to access content according to one embodiment. Overlay and authentication service 126 is in communication with the authentication login API 122 and/or application 124 and MSE/MME enabled player page 120.

[00046] Network 100 can be used to present synchronized, ultra-low latency community viewing within large-scale live-event streaming experiences-enabling interactive workflows such as in-stream sports betting, eCommerce and non-fungible token (“NFT”)-based transactions, as well as real-time chatting and other gamified use cases. Network 100 supports an interactive platform for live broadcast enabling fans and commentators to engage in a community, which drives interaction - further driving loyalty and monetization opportunities.

[00047] In one embodiment, a backend function (referred to as SyncPerfect, discussed in connection with FIG. 1 ) provides a single, central, clock pulse generator for synchronization of viewing and a protocol function to provide a central clock signal to each client. A client application leverages the central clock signal and timing data embedded into an ultra-low latency HLS video stream via datetime stamp (program date and time (“PDT”) or ID3-based) to calculate the deviation from relative real-time of the video feed. The client then issues speed control playback commands to the video player to speed the stream up, or slow the stream down, until the playhead position is aligned to relative real-time, thereby achieving synchronization across all client viewing experiences.

[00048] In one embodiment, if clients are receiving data too slow, adjustments can be made to reduce the latency. In one embodiment, the central clock is used to time playback and determine if latency a client is experiencing is too high. The central clock, in one embodiment, is used as a basis to adjust the playback of content so that each client displays the content at the same time.

[00049] According to some embodiments of the present disclosure, online communities (such as those on Discord) can be integrated into a stream to augment the content. This technology, referred to as Discord Integration, comprises a backend function in the form of a Discord bot that, when invited to the Discord server, provides single sign on access to validate channel members. According to such embodiments, users can also use Discord login credentials to access streamed content provided by system 100 instead of creating a new separate account. In one embodiment, the protocol functions used are Standard OAuth 2.0 and Standard Discord bot function. In one embodiment, the client function is a standard login with a Discord function. Accordingly, users can avail themselves of the chat functionality provided by system 100 by joining the Discord Chat via Discord or via authentication provided by system 100; however, according to some embodiments of the present disclosure, users may be required to authenticate via system 100 in order to join the live stream or simulation.

[00050] According to some embodiments, alternative (e g., alternative to the commentators associated with the streamed content) and/or community-based commentators can provide commentary regarding the content and this commentary is integrated into the stream to augment the content. This technology is referred to herein as “Cast” and comprises a backend function of real-time broadcasting one to many broadcasts using a real-time protocol associated with session control. In one embodiment, Internet Engineering Task Force (“IETF”) WebRTC-HTTP ingestion protocol/ (“WHIP”) is used for low latency ingress of source feeds (e.g., commentator feed) and WebRTC-HTTP egress protocol (“WHEP”) is used for egress of the streamed content to a large audience size (e.g., audiences in excess of 1 M users/viewers). In one embodiment, the client application can generate a graphical user interface having an overlay of one or more (e.g., dual) commentator windows on top of a synchronized video feed, for visual display of the commentator feed(s) to a user.

[00051] According to some embodiments of the present disclosure, a producer can manage the production of additional information added to the original content received for transmission (e.g., a live video feed and/or one or more commentator feeds) by way of a Producer Panel graphical user interface generated by the system 100.

[00052] According to embodiments of the present disclosure, the Producer Panel can also enable producers to speak directly with commentators via an interruptible foldback ( “I FB”).. Those of ordinary skill in the art will understand that an IFB is a one-way communication system used in film, TV, radio, and video production, which allows a producer or director to communicate with on-air talent. Using the IFB, the producer or director can interrupt a main monitor mix to provide instructions to the talent, in other words, producers can choose to speak to commentators directly (e.g., via audio), or they can use internal chat functionality provided by the producer panel (e.g., chat functionality separate from and not viewable by, normal participants viewing a feed).

[00053] According to some embodiments, system 100 can generate a graphical user interface with an interactive layer for on screen interactivity, referred to herein as “Live Layer,” which can include a back-end function that allows set up and management of producer triggered on-screen elements and provides producer control of said elements. The client function of Live Layer allows for the presentation of on-screen elements and fully interactive HTML5 graphics, enabling users to interact with the stream via these graphics, and enabling stream/content providers additional tools for driving engagement of the users during the live stream. Examples of the on-screen elements of the Live Layer include, but are not limited to, interactive contextual advertising, live shopping, participation in polls (see, e.g., survey system 128 discussed herein), quizzes and sweepstakes, and signup/registration for new products or services related to the subject matter of the live stream (e.g., graphics with links to official World Cup merchandise presented during a live stream of the World Cup) or the stream/content provider (e.g, a graphic/logo showing a live feed of the World Cup is being provided by ESPN, or the like).

[00054] According to embodiments of the present disclosure, survey system 128 can be used to administer polls and count votes. The survey system 128 can be a subsystem of the system 100, can be a third-party system/service integrated into the system 100 (e.g., via an API), or can be a combination thereof. The survey system 128, can comprise a graphical user interface generated by the system 100 that allows producers and commentators to trigger interactive polls and/or quizzes relating to, for example, the subject matter of a live stream, present the interactive polls and/or quizzes to viewers of the live stream, receive input (e.g., votes) from the viewers of the live stream, calculate the results of the polls and/or quizzes, and present (e.g., displays on-screen) the results to one or more of the producers, commentators, and/or viewers. For example, a producer can use the survey system 128 to present sports trivia (e.g., one or more prompts with answers in a multiplechoice format) related to a player featured on live stream of a sporting event, allow the viewers of the stream to submit answers (e.g., a multiple-choice selection) for a predetermined period of time (e.g., 5 seconds, 10 seconds, 15 seconds, etc.) set by the producer, and present the results of the trivia to the users (e.g., displaying the names of user(s) who answered correctly, or displaying a list of users with the greatest number of correct answers for a series of trivia questions. Accordingly, the survey system 128 provides another tool the stream/content providers can employ to drive engagement and encourage users to sign up and/or share their information with the stream/content providers.

[00055] According to additional embodiments of the present disclosure, ad detector 130 is used to detect advertisements.

[00056] As discussed hereinabove, Discord is a platform which provides voice over internet protocol (“VoIP”), instant messaging, and other social functionalities. Users have the ability to communicate with voice calls, video calls, and text messaging, as well as to share media and other files in private chats or as part of communities called servers..

[00057] Additionally, Discord bots are Al-driven tools that can assist users to automate certain tasks on a Discord server. Bots make it easier to build a community that is truly engaged and can be used to moderate a server, welcome new members, or even ban people who are creating a bad environment for other users.

[00058] FIG. 3A shows a high-level overview of process steps carried out by the system of the present disclosure, whereby the system includes one or more bots and an application (e.g., a desktop or web-based application) generates a graphical user interface that integrates one or more functions of the one or more bots into the player (e.g., player 120, as shown and described in connection with FIG. 2), which can be fully synchronized with the stream, according to some embodiments of the present disclosure. Each bot can be an external server/bot (e.g., a Discord bot or other 3^rd party server/bot) in communication with (e.g., integrated into) the system, the bots can be part of the system of the present disclosure (e.g., the bots being hosted or run on one or more servers of the system), or a combination thereof. [00059] FIG. 3B shows process steps carried out by the system of the present disclosure for authenticating a user and providing an authenticated user access to the application and one or more functions of the bots shown and described in connection with FIG. 3A.

[00060] FIG. 4 shows a schematic of a Control Plane 400 of the system of the present disclosure, according to some embodiments thereof. As discussed herein, many currently available content streaming solutions are focused on ultra-low latency (e.g., sports betting use cases), but these solutions may be cost prohibitive. For example, Phenix Real Time Solutions charges 20 cents per hour per user for content distribution through their services. So, if there are 1 M users/clients watching a 2-hour game, that is a $400K distribution bill per game. Furthermore, if only a small percentage of clients are betting and require low latency, and the other clients do not require low latency, this results in an expensive cost for such a low requirement for actual low latency usage. This makes monetization of content streaming, using solutions like those provided by Phenix Real Time Solutions, less profitable, or even unprofitable in certain scenarios.

[00061] According to embodiments of the present disclosure, Control Plane 400 (used with networks that support the functionality described here, e.g., 5G), enables a shift from a latent (e.g., high-latency) feed to a real-time (e.g., low-latency or ultra low-latency) feed, thereby opening new use cases, such as, but not limited to, sports betting and other scenarios where a low-latency connection may be desirable for a portion of a live stream or streaming event. The control plane 400 shifts from a latent feed to a real-time feed by adjusting Ad cue data (e.g., data that identifies when an advertisement should be displayed and how much time is available to display advertisements) to shorten a first commercial break when the system determines that enough betting players/users are taking part in a game to justify the higher cost of real-time delivery - all of which is seamless to the end user. When the system determines that the low-latency feed delivery is no longer necessary (e.g., betting has concluded or there are no longer enough betting players/users to satisfy a minimum threshold requirement to justify low-latency streaming), control plane 400 shifts the stream from the low-latency feed back to the high-latency feed. In addition to reducing costs, as discussed above, switching between high and low-latency feeds optimizes network performance and utilization of resources (e.g., streaming bandwidth, processing power, etc.) by avoiding excess traffic volume on the network and hindrance of network performance (e.g., caused by unnecessary over-utilization of low-latency feeds).

[00062] This approach can be considered somewhat analogous to how video quality (e.g., bitrate and/or resolution) may be reduced for content being streamed to a mobile device (e.g., a cellular phone) because full HD/4K may not be necessary, or provide any benefit, on a mobile device screen having less than full HD/4K resolution, or being sufficiently small such that higher resolution (e.g., full HD/4K) would be imperceptible to the human eye at normal viewing distances. However, unlike the way in which video quality may be reduced for content being streamed to a mobile device, the control plane 400 of the system of the present disclosure is different in that the latency of the feed is reduced, increasing costs and/or network resources for delivery of the feed, in response to certain triggering events (e.g., active betting users joining the stream), which enables real-time use cases (e.g., betting), but only when reduced latency is needed. All other use cases (e.g., where reduced latency is not necessary) go through a lower cost and/or resource intensive path, for example, as described above in connection with the SyncPerfect system of FIG. 1.

[00063] According to some embodiments of the present disclosure, server-side ad insertion (“SSAI”) is the technology that inserts personalized ads and stitches them seamlessly into a single stream. Rather than on the client side (e.g., client-side ad insertion (“CSAI”)), ad insertion happens on the server side. Leveraging SSAI enables the system of the present disclosure to insert ads and product drops, but also enables the additional benefit of driving the control plane 400 decision tree, as discussed above.

[00064] FIGS. 5A-5C illustrate a system 500 for managing video feeds and associated data for media licensees, according to embodiments of the present disclosure. Specifically, FIG. 5A is a network diagram illustrating how system 500 generates multiple streams 502 from a source feed 504, FIG. 5B is a network diagram illustrating how system 500 provides a return path for first-party data to rightsholders and media licensees, and FIG. 50 is a graphical user interface 550 generated by the system 500 of the present disclosure, including exemplary statistical analyses and engagement metrics.

[00065] As discussed in greater detail hereinbelow, FIGS. 5A-C illustrate how rightsholders can adapt a single video source broadcast feed from a single language one path to its media licensees, offer its media licensees the ability to create localized interactive broadcasts throughout the world (e.g., a “world feed”), and collect unified first party data that can be shared across the network and with the rightsholder. Media licensees can also benefit from shared economics, data, and a localized and personalized stream versus a single language only feed.

[00066] With reference to FIG. 5A, a rightsholder 506 (e.g., the National Football League, Major League Baseball, Federation Internationale de Football Association, etc.) first sends a clean video feed 504 with natural sounds (e.g., using SRT or a point-to-point connection), along with a statistics API feed or integrated real-time data API (e.g., a leaderboard, player statistics, game statistics, etc.), to system 500. System 500 then aligns these feeds via deterministic latency to synchronize the data feed with the video feed. The video feed and ancillary data move in BANK (closed captioning).

[00067] After the feeds are aligned, the system 500 encodes an enhanced clean feed once via secure reliable transcript (“SRT”) and transmits the encoded feed to the cloud, or other location, for further processing and/or distribution.

Accordingly, a rightsholder production team, for example, will not have to live encode on a 1 :1 basis. Notably, encoding the aligned feeds with all of the video feed data and ancillary data utilized by each of a plurality of media licensees 508 into an enhanced data feed 510 can significantly reduce the number of egress points by media licensees 508 to the system 500 (e.g., from 100 egress points to a single egress point) because all of the licensees 508 use the same enhanced feed 510. [00068] The system 500 distributes the video and data to the media licensees 508 across multiple countries (e.g., AWS code for SRT stamps) by spinning up instances of the enhanced feed 510 in each of a plurality of geographically diverse locations (e.g., approximately 100 instances throughout the world). The enhanced feed 510 is then ingested by clients 512, where it is parsed and displayed with data- driven graphics in various languages corresponding to each of the media licensees 508. The foregoing method of distribution reduces the total number of failure points and enhances workflow efficiencies, thereby simplifying troubleshooting.

[00069] Each media licensee 508 can receive base package with a client 512 that generates a graphical user interface including one commentator panel (e.g., providing for two commentators 514) and one producer panel for localizing in-vision commentators. Additional commentator and/or producer panels can also be provided. According to some embodiments of the present disclosure, media licensees 508 are provided with a separate subdomain, allowing them to customize their authentication features, such as free features, registration walls, or paywalls, and each media licensee 508 can have a rightsholder-approved branded login page. Media licensees 508 can also leverage licensed commentator and producer panels to create localized alternate broadcasts, HTML5 graphics, quizzes, and polls, as described herein.

[00070] The system 500 also standardizes fan-friendly data capture for the rightsholder 506 and its media licensees 508, and can ensure uniform terms and conditions, moderation, profanity filtering, geo-restriction, and digital rights management (“DRM”) across all devices.

[00071] As shown in FIG. 5B, through an authentication layer 516 of system 500 (e.g., via client 512), users 518 can submit first-party data (e.g., via client 512), including name, email, phone number, date of birth, gender, etc. Additionally, when users 518 engage in a live experience via the client 512, the system 500 captures engagement and/or survey data, which the system 500 can provide to the licensees 508 and/or the rightsholder 506, thereby enabling rightsholders 506 and media licensees 508 to improve the user experience over time across the network.

[00072] According to some embodiments of the present disclosure, the system 500 can process the first party user engagement and/or survey data captured during a live experience via the client 512, and then generate deep insight and audience development data for the rightsholders 506 and media licensees 508. For example, FIG. 5C is a graphical user interface 550 generated by the system 500, including exemplary statistical analyses and engagement metrics determined by the system 500, which are based on the user and/or survey data captured by the system 500 during authentication and/or a live experience via the client 512. [00073] Those of ordinary skill in the art will understand that the devices, components, and/or functions described herein can be implemented using a computer or plurality of computers. A high-level block diagram of such a computer is illustrated in FIG. 6. Computer 602 contains a processor 604 which controls the overall operation of the computer 602 by executing computer program instructions which define such operation. The computer program instructions may be stored in a storage device 612, or other computer readable medium (e.g., magnetic disk, CD ROM, etc.), and loaded into memory 610 when execution of the computer program instructions is desired. Thus, the devices, components, and/or functions of FIGS. 1- 5C can be defined by the computer program instructions stored in the memory 610 and/or storage 612 and controlled by the processor 604 executing the computer program instructions. For example, the computer program instructions can be implemented as computer executable code programmed by one skilled in the art. Accordingly, by executing the computer program instructions, the processor 604 executes an algorithm defined to implement the devices, components, and/or functions of FIGS. 1-5C. The computer 602 also includes one or more network interfaces 606 for communicating with other devices via a network. The computer 602 also includes input/output devices 608 that enable user interaction with the computer 602 (e.g., display, keyboard, mouse, speakers, buttons, etc.) One skilled in the art will recognize that an implementation of an actual computer could contain other components as well, and that FIG. 6 is a high-level representation of some of the components of such a computer for illustrative purposes.

[00074] The foregoing Detailed Description is to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the inventive concept disclosed herein is not to be determined from the Detailed Description, but rather from the claims as interpreted according to the full breadth permitted by the patent laws. It is to be understood that the embodiments shown and described herein are only illustrative of the principles of the inventive concept and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the inventive concept. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the present disclosure.

Claims

CLAIMS What is claimed is:

1 . A method for supporting synchronized viewing of streaming video, comprising: receiving an indication of a delay between a send time and an arrival time of an encrypted video feed transmitted from a content delivery network to an HTTP live streaming (HLS) player; adjusting a playhead position on the HLS player based on the delay; and transmitting a synchronized feed to clients.

2. The method of claim 1 , wherein the send time and the arrival time are determined based on a central clock.

3. The method of claim 1 , wherein the receiving, adjusting, and transmitting are performed repeatedly.

4. The method of claim 1 , wherein the adjusting comprises advancing the playhead position on the HLS player.

5. The method of claim 1 , wherein the adjusting comprises delaying the playhead position on the HLS player.

6. The method of claim 5, comprising receiving an unencrypted video feed transmitted from the content delivery network at a commentator interface.

7. The method of claim 6, comprising transmitting a commentator feed from the commentator interface to the HLS player.

8. The method of claim 7, wherein transmitting the synchronized feed to the clients comprises synchronizing the encrypted video feed and the commentator feed.

9. The method of claim 7, wherein transmitting the commentator feed from the commentator interface to the HLS player comprises determining whether a streaming video event requires an ultra low-latency (ULL) commentator feed.

10. The method of claim 9, comprising transmitting a ULL commentator feed to the HLS player based on a positive determination.

11 . The method of claim 8, comprising receiving an unencrypted video feed transmitted from the content delivery network at a production interface.

12. The method of claim 11 , comprising transmitting production data from the production interface to the HLS player, wherein the production data comprises one or more real-time interactive features.

13. The method of claim 12, wherein transmitting the synchronized feed to the clients comprises synchronizing the encrypted video feed, the commentator feed, and the production data.

14. A method for optimizing latency of streaming video, comprising: transmitting a latent streaming video feed from a content delivery network to a plurality of clients; identifying a group of clients requiring an ultra-low latency (ULL) video streaming feed; and transmitting a ULL streaming video feed from the content delivery network to the identified group of clients; wherein identifying the group of clients is based on a triggering event relating to the group of clients.

15. The method of claim 14, wherein the identifying is performed using an ad trigger.

16. The method of claim 14, comprising determining whether the triggering event is active.

17. The method of claim 16, comprising switching the identified group of clients from the latent video streaming feed to the ULL video streaming feed based on a determination that the triggering event is active.

18. The method of claim 17, comprising switching the identified group of clients from the ULL video streaming feed to the latent video streaming feed based on determination that the triggering event is not active.