WO2025180119A1 - Data transmission method and apparatus, and device and storage medium - Google Patents

Abstract

Provided in the embodiments of the present disclosure are a data transmission method and apparatus, and a device and a storage medium. The data transmission method comprises: on the basis of the current first network state information of a first network path and the current second network state information of a second network path, determining whether a redundant-data cross-path transmission condition is currently met; if the redundant-data cross-path transmission condition is currently met, determining target redundant data corresponding to the current target media data to be transmitted; and transmitting the target media data by means of the first network path, and then transmitting the target redundant data by means of the second network path. By means of the technical solution in the embodiments of the present disclosure, the cross-path transmission of redundant data can be realized, such that the data transmission quality is improved with a low traffic consumption, thereby improving the user experience.

Description

Data transmission method, device, equipment and storage medium

This application claims priority to Chinese Patent Application No. 202410224497.9 filed on February 28, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application.

Technical Field

The embodiments of the present disclosure relate to a data transmission method, apparatus, device, and storage medium.

Background Art

With the rapid development of computer technology, real-time transmission of media data is often required to achieve real-time communication. Currently, real-time communication typically uses a single network path established between the sender and receiver to transmit media data. When data packet transmission is delayed or lost, it is necessary to use this network path to simultaneously transmit the media data and its redundant data to improve the success rate of data reception at the receiving end. However, when problems arise with this single network path, media data and redundant data are still subject to transmission delays or loss, resulting in the data being unable to be decoded in a timely manner at the receiving end, affecting the user experience.

Summary of the Invention

The present disclosure provides a data transmission method, apparatus, device, and storage medium to achieve cross-path transmission of redundant data, thereby improving data transmission quality with lower traffic consumption and enhancing user experience.

In a first aspect, an embodiment of the present disclosure provides a data transmission method, including:

determining whether a redundant data cross-path transmission condition is currently met based on current first network state information of the first network path and current second network state information of the second network path;

If the redundant data cross-path transmission condition is currently met, determining the target redundant data corresponding to the target media data to be transmitted;

The target media data is transmitted via the first network path, and the target redundant data is transmitted via the second network path.

In a second aspect, an embodiment of the present disclosure further provides a data transmission device, including:

a cross-path transmission detection module, configured to determine whether a redundant data cross-path transmission condition is currently satisfied based on current first network status information of the first network path and current second network status information of the second network path;

a redundant data determination module, configured to determine target redundant data corresponding to target media data to be transmitted if a redundant data cross-path transmission condition is currently met;

A data transmission module is configured to transmit the target media data through the first network path and transmit the target redundant data through the second network path.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

one or more processors;

a storage device for storing one or more programs,

When one or more programs are executed by one or more processors, the one or more processors implement the data transmission method as described in any one of the embodiments of the present disclosure.

In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute a data transmission method as described in any one of the embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

FIG1 is a flow chart of a data transmission method provided by an embodiment of the present disclosure;

FIG2 is an exemplary diagram of a data transmission process involved in an embodiment of the present disclosure;

FIG3 is a flow chart of another data transmission method provided by an embodiment of the present disclosure;

FIG4 is a schematic structural diagram of a data transmission device provided by an embodiment of the present disclosure; and

FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

DETAILED DESCRIPTION

The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

Figure 1 is a flow chart of a data transmission method provided by an embodiment of the present disclosure. This embodiment of the present disclosure is applicable to the transmission of media data in real-time communication scenarios, where real-time communication scenarios may include business scenarios such as live broadcasts, remote conferences, or cloud gaming. The method can be executed by a data transmission device, which can be implemented in software and/or hardware, and optionally by an electronic device, such as a mobile terminal, a PC, or a server.

As shown in FIG1 , the data transmission method specifically includes the following steps:

S110 : Determine whether a redundant data cross-path transmission condition is currently met based on current first network status information of the first network path and current second network status information of the second network path.

Among them, the first network path may refer to a network transmission channel for transmitting media data. The second network path may be a network transmission channel for assisting the first network path in transmitting redundant data when a problem occurs in the first network path. Media data may refer to audio and video stream data that needs to be transmitted in real-time communication. Redundant data may be data obtained by redundantly encoding the media data. For example, based on the forward error correction coding method (Forward Error Correction, FEC), the media data to be transmitted is redundantly encoded to obtain redundant data corresponding to the media data. Redundant data can be used as protection data for the media data, so that the receiving end can quickly recover the media data based on the redundant data.

In this embodiment, the first network path can be used as the main path, and the second network path can be used as the auxiliary path. The first network path and the second network path are two completely different network paths. For example, the first network path is a wireless local area network path, and the second network path is a cellular network path. A wireless local area network path refers to a network path for wireless transmission via WiFi. A cellular network path refers to a network path for wireless transmission via a mobile network. Generally, the network speed of a wireless local area network path is high, the network access is relatively stable, and user access is generally free of charge, and the bandwidth provider pays for it. Therefore, the wireless local area network path can be used as the main path for the transmission of media data. The cellular network path is provided by a communication operator and uses the user's own data network. Therefore, the user needs to pay a certain fee to use the cellular network, so the cellular network path can be used as an auxiliary path for auxiliary data transmission. In view of this, it is necessary to reduce the traffic consumption of the cellular network path while improving the quality of data transmission, thereby saving user usage costs and improving user experience.

The current first network status information of a first network path may refer to the network status information of the first network path during current data transmission. The current first network status information can be used to characterize the network quality of the first network path during current data transmission. For example, the current first network status information may include, but is not limited to, the current first round-trip time (RTT) of the first network path. A higher current first round-trip time indicates that the current transmission delay of the first network path is greater and the network quality is poorer. Similarly, the current second network status information can be used to characterize the network quality of the second network path during current data transmission. For example, the current second network status information may include, but is not limited to, the current second round-trip time of the second network path. A higher current second round-trip time indicates that the current transmission delay of the second network path is greater and the network quality is poorer. The redundant data cross-path transmission condition may be a pre-set condition that must be met to allow the second network path to transmit redundant data. For example, the redundant data cross-path transmission condition may mean that the transmission quality of the second network path is higher than that of the first network path, so that the auxiliary transmission of the second network path can enhance the overall data transmission quality.

Specifically, the current first network status information of the first network path and the current second network status information of the second network path can be obtained in real time or periodically, and the current first network status information and the current second network status information can be compared to determine whether the redundant data cross-path transmission condition is currently met.

Exemplarily, step S110 may include: comparing a current first round-trip delay of the first network path with a current second round-trip delay of the second network path, and determining whether a redundant data cross-path transmission condition is currently met based on the comparison result.

Specifically, if the current second round-trip delay is less than or equal to the current first round-trip delay, it indicates that the transmission quality of the second network path is higher than the transmission quality of the first network path, and the second network path can play an enhancing role. At this time, it can be determined that the redundant data cross-path transmission condition is currently met, otherwise it is determined that the redundant data cross-path transmission condition is not currently met. Alternatively, the current first round-trip delay of the first network path is added to the preset delay (for example, 50ms), and the addition result is the target round-trip delay corresponding to the first network path. The target round-trip delay is compared with the current second round-trip delay of the second network path, and based on the comparison result, it is determined whether the redundant data cross-path transmission condition is currently met. For example, if the current second round-trip delay is less than or equal to the target round-trip delay, it indicates that the transmission quality of the second network path is higher than the transmission quality of the first network path or is close to the transmission quality of the first network path, and the second network path can play an enhancing role. At this time, it can be determined that the redundant data cross-path transmission condition is currently met. If the current second round-trip delay is greater than the target round-trip delay, it indicates that the transmission quality of the second network path is significantly lower than the transmission quality of the first network path, that is, the transmission quality of the first network path is significantly higher than the transmission quality of the second network path. At this time, there is no need to perform cross-path transmission of redundant data, that is, it is determined that the conditions for cross-path transmission of redundant data are not currently met.

S120: If the redundant data cross-path transmission condition is currently met, determine the target redundant data corresponding to the target media data to be transmitted.

The target media data may refer to the audio or video stream data to be transmitted, and the target redundant data may be used as protection data for the target media data so as to quickly restore the target media data at the receiving end.

Specifically, when it is determined that the conditions for redundant data cross-path transmission are currently met, the target media data to be transmitted can be redundantly encoded based on the current packet loss rate of the first network path to obtain target redundant data to be transmitted on the second network path. For example, the data volume of the target media data to be transmitted can be multiplied by the current packet loss rate of the first network path to obtain a target redundant data volume. Based on the target redundant data volume, the target media data can be redundantly encoded to obtain target redundant data having the target redundant data volume. The target redundant data volume is smaller than the target media data volume. By dynamically generating an appropriate amount of target redundant data based on the current packet loss rate of the first network path, an appropriate amount of target redundant data can be generated, thereby further reducing traffic consumption on the second network path while ensuring data transmission quality.

S130 : Transmit target media data through the first network path, and transmit target redundant data through the second network path.

Specifically, after the conditions for redundant data cross-path transmission are currently met, the target media data is transmitted through the first network path, and the target redundant data is transmitted through the second network path at the same time, thereby utilizing the two network paths to transmit media data and redundant data respectively, and dynamically unloading the redundant data on the first network path to the second network path for transmission, thereby improving the success rate of redundant data transmission, and thereby improving the success rate of data recovery at the receiving end.

It should be noted that when it is determined that the conditions for cross-path transmission of redundant data are not currently met, it indicates that the network quality of the first network path is better. At this time, the target redundant data corresponding to the target media data to be transmitted can be generated according to the original logic, and the target media data and the target redundant data can be transmitted simultaneously through the first network path without the participation of the second network path, thereby reducing the traffic consumption of the second network path.

For example, referring to FIG2 , two network paths are established between the sending end and the receiving end, namely the first network path and the second network path. When the network status of the first network path is good, no redundant data will be generated in the controller of the sending end, and the target media data to be transmitted will be sent to the first network path for transmission. When packet loss, freezing, etc. occur during transmission on the first network path, the controller in the sending end will generate a certain amount of target redundant data, and when it is determined that the redundant data cross-path transmission conditions are currently met, that is, when it is confirmed that the second network path can play a good data recovery role, the target redundant data will be sent to the second network path for transmission, thereby realizing cross-path transmission of redundant data. If it is determined that the redundant data cross-path transmission conditions are not currently met, the target redundant data can continue to be sent to the first network path for transmission, avoiding wasting the transmission traffic of the second network path. Compared with the single path transmission method, the embodiment of the present disclosure can make full use of the robustness of the two network paths to cope with network jitter, and can also take into account traffic overhead, effectively balancing traffic consumption and data transmission quality, while saving traffic as much as possible and optimizing user experience.

The technical solution of the embodiment of the present disclosure determines whether a condition for cross-path transmission of redundant data is currently met based on the current first network status information of the first network path and the current second network status information of the second network path. If the condition for cross-path transmission of redundant data is currently met, target redundant data corresponding to the target media data to be transmitted is determined, the target media data is transmitted through the first network path, and the target redundant data is transmitted through the second network path, thereby dynamically offloading part of the traffic of the first network path to the second network path for transmission, realizing cross-path transmission of redundant data and improving the success rate of data reception. At the same time, a large amount of traffic on the second network path will not be consumed, thereby improving data transmission quality with lower traffic consumption and enhancing user experience.

Based on the above technical solution, as an optional method, the "transmitting the target redundant data through the second network path" in step S130 may include: if a target transmission code rate corresponding to the target redundant data is greater than the current network bandwidth of the second network path, determining a first transmission code rate for transmitting the redundant data through the first network path and a second transmission code rate for transmitting the redundant data through the second network path based on the target transmission code rate and the current network bandwidth; transmitting part of the redundant data in the target redundant data through the first network path at the first transmission code rate, and transmitting the remaining redundant data in the target redundant data through the second network path at the second transmission code rate.

The target transmission bit rate may refer to the amount of target redundant data transmitted per unit time. The current network bandwidth may refer to the currently available bandwidth of the second network path. The first transmission bit rate may refer to the bit rate at which the first network path transmits redundant data. The second transmission bit rate may refer to the bit rate at which the second network path transmits redundant data. The sum of the first transmission bit rate and the second transmission bit rate equals the target transmission bit rate.

Specifically, whether the second network path is capable of transmitting all redundant data can be determined by detecting whether the target transmission code rate corresponding to the target redundant data is greater than the current network bandwidth of the second network path. If the target transmission code rate corresponding to the target redundant data is greater than the current network bandwidth of the second network path, it indicates that the second network path is unable to transmit the corresponding target redundant data at the target transmission code rate. In this case, the target transmission code rates can be allocated based on the current network bandwidth to determine a first transmission code rate for transmitting redundant data on the first network path and a second transmission code rate for transmitting redundant data on the second network path. In addition to transmitting the target media data, the first network path also transmits some redundant data at the first transmission code rate, and the second network path transmits the remaining redundant data at the second transmission code rate. Therefore, by using two network paths to simultaneously transmit redundant data when the transmission bandwidth of the second network path is insufficient, the success rate of redundant data transmission can be further improved, thereby ensuring the success rate of data recovery at the receiving end.

Exemplarily, determining a first sending code rate for transmitting redundant data on a first network path and a second sending code rate for transmitting redundant data on a second network path based on a target sending code rate and a current network bandwidth may include: determining a difference between the target sending code rate and the current network bandwidth; determining the difference as the first sending code rate for transmitting redundant data on the first network path, and determining the current network bandwidth as the second sending code rate for transmitting redundant data on the second network path.

Specifically, the current network bandwidth of the second network path is directly used as the second transmission bit rate for redundant data transmission on the second network path, allowing the second network path to transmit more redundant data at its maximum bandwidth capacity. The difference between the target transmission bit rate and the current network bandwidth is used as the first transmission bit rate for redundant data transmission on the first network path, allowing the first network path to transmit the remaining redundant data at the first transmission bit rate, minimizing the data transmission pressure on the first network path and thereby optimizing data transmission efficiency and user experience.

FIG3 is a flow chart illustrating another data transmission method provided by an embodiment of the present disclosure. This embodiment of the present disclosure optimizes the step of "determining target redundant data corresponding to the target media data currently to be transmitted" based on the aforementioned embodiments. Explanations of terms that are identical or corresponding to those in the aforementioned embodiments are omitted here.

As shown in FIG3 , the data transmission method specifically includes the following steps:

S310: Determine whether a redundant data cross-path transmission condition is currently met based on current first network status information of the first network path and current second network status information of the second network path.

S320: If the redundant data cross-path transmission condition is currently met, determine whether the redundant data enhancement condition is currently met based on at least one of the current first network status information of the first network path and the current playback freeze information of the media data.

The current playback jam information of the media data may refer to jam information that occurs when the receiving end is currently playing the media data. As shown in Figure 2, the receiving end combines the media data and redundant data belonging to the same stream received from the first network path and the second network path for data recovery, and decodes and renders the recovered media data to enable playback of the media data. The current playback jam information can be used to characterize the user's quality of experience. The greater the jamming of the currently playing media data, that is, the less smooth the media data playback, the worse the user experience quality. This embodiment can use any metric that can characterize the user experience quality to characterize the current playback jam information. For example, the current playback jam information of the media data may include: the current decoding interval between the decoding of two adjacent frames in the media data. The current decoding interval may refer to the time difference between the most recently decoded frame and the decoding of the previous frame. The larger the current decoding interval, the greater the likelihood of playback jams. The redundant data enhancement condition may be a pre-set condition that must be met to generate a larger amount of redundant data.

Specifically, after determining that the conditions for cross-path transmission of redundant data are currently met, it is possible to determine whether the network status of the first network path is in an extremely weak network situation based on at least one of the current first network status information of the first network path and the current playback freeze information of the media data, thereby determining whether the redundant data needs to be enhanced in order to generate more redundant data.

Exemplarily, step S320 may include: if the redundant data cross-path transmission condition is currently met, detecting whether the current first round-trip delay of the first network path is greater than or equal to the preset delay, and whether the current decoding interval between the two adjacent frames currently decoded in the media data is greater than or equal to the preset time interval, and determining whether the redundancy enhancement condition is currently met based on the detection result.

The preset delay may be a preset minimum round-trip delay at which the first network path cannot transmit media data on time. The preset time interval may be a preset minimum decoding interval corresponding to playback freezes that are unacceptable to the user.

Specifically, when the current first network status information is the current first round-trip delay and the current playback freeze information is the current decoding interval between the current two adjacent frames being decoded, it can be determined whether the redundancy enhancement condition is currently met by detecting whether the current first round-trip delay of the first network path is greater than or equal to the preset delay, and whether the current decoding interval between the current two adjacent frames being decoded in the media data is greater than or equal to the preset time interval.

Exemplarily, determining whether the redundancy enhancement condition is currently met based on the detection result may include: if the current first round-trip delay of the first network path is greater than or equal to the preset delay, or the current decoding interval between the current two adjacent frames decoded in the media data is greater than or equal to the preset time interval, then determining that the redundancy enhancement condition is currently met; if the current first round-trip delay of the first network path is less than the preset delay, and the current decoding interval between the current two adjacent frames decoded in the media data is less than the preset time interval, then determining that the redundancy enhancement condition is not currently met.

Specifically, when the current first round-trip delay of the first network path is greater than or equal to the preset delay (for example, 500ms), it indicates that the first network path has been unable to complete the transmission of the media data on time. At this time, it can be determined that the redundancy enhancement condition is currently met, otherwise it is determined that the redundancy enhancement condition is not currently met. Alternatively, when the current decoding interval between the two adjacent frames in the media data is greater than or equal to the preset time interval (for example, 1000ms), it indicates that the playback freeze at the receiving end is unacceptable to the user. At this time, it can be determined that the redundancy enhancement condition is currently met, otherwise it is determined that the redundancy enhancement condition is not currently met. By utilizing the current first round-trip delay and the current decoding interval, it can be more accurately determined whether redundant data enhancement processing is required, further improving the success rate of data recovery.

Exemplarily, the current decoding interval between two adjacent frames in the media data is obtained by parsing a feedback message sent by the receiving end.

Specifically, during the decoding and recovery process of the received data, the receiving end can calculate the time difference between the currently decoded frame and the previous frame decoded. This time difference is the current decoding interval between the two adjacent frames being decoded, thereby obtaining the current decoding interval in real time. Based on the current decoding interval, the receiving end can generate a feedback message containing the current decoding interval. The message type of the feedback message can be, but is not limited to, the RTCP (RTP Control Protocol) network transmission protocol type. The receiving end can actively send the generated feedback message to the sending end at regular intervals (for example, every 200ms), thereby achieving active transmission of the feedback message. The receiving end can also generate and send a corresponding feedback message upon receiving a request message sent by the sending end, thereby achieving passive transmission of the feedback message. After receiving the feedback message, the sending end parses the feedback message to obtain the current decoding interval in the feedback message, allowing the sending end to accurately understand the media data playback jamming situation at the receiving end.

It should be noted that the receiving end can send the feedback message to the sending end through the first network path, or send the feedback message to the sending end through the second network path. It can be fixedly selected based on business needs or dynamically selected based on network status.

S330: Determine target redundant data corresponding to the target media data to be transmitted based on whether a redundant data enhancement condition is currently satisfied.

Specifically, if the redundant data enhancement conditions are not currently met, target redundant data corresponding to the target media data can be generated using the original redundant method. If the redundant data enhancement conditions are currently met, the redundant data generated using the original redundant method can be enhanced to generate enhanced target redundant data. The amount of enhanced target redundant data is greater than that generated using the original redundant method. By generating more target redundant data, the image can be fully restored even in extremely weak network conditions.

Exemplarily, step S330 may include: if the redundant data enhancement condition is currently met, determining target redundant data equal to the target media data currently to be transmitted; if the redundant data enhancement condition is currently not met, determining target redundant data corresponding to the target media data currently to be transmitted based on the current packet loss rate of the first network path.

Specifically, if the redundant data enhancement condition is currently satisfied, target redundant data equal to the target media data currently to be transmitted is determined, so that the amount of the generated target redundant data is equal to the amount of the target media data. If the redundant data enhancement condition is currently not satisfied, the amount of the target media data currently to be transmitted may be multiplied by the current packet loss rate of the first network path to obtain a target redundant data amount. Based on the target redundant data amount, the target media data is redundantly encoded to obtain target redundant data having the target redundant data amount. In this case, the amount of the generated target redundant data is less than the amount of the target media data.

It should be noted that when the redundant data enhancement conditions are currently met, target redundant data equal to the target media data is determined and sent. Even if all the media data transmitted by the first network path is lost, as long as the target redundant data is successfully transmitted, the receiving end can still recover, decode, render and play the media data, thereby ensuring that the data can be recovered at the receiving end, and improving the data recovery efficiency and success rate when the first network path is in poor condition.

As an optional method, if the redundant data enhancement condition is currently met, target redundant data equal to the target media data currently to be transmitted is determined, including: if the number of times the redundant data enhancement condition is met within the current time window is greater than or equal to a preset number, target redundant data equal to the target media data currently to be transmitted is determined within a preset time length, so as to continuously send the target redundant amount within the preset time length.

Among them, the current time window can be a sliding time window with a preset window length. For example, the preset window length can be 3 seconds. Specifically, it is possible to detect whether the redundant data enhancement condition is currently met at intervals within the current time window, so that multiple detections of whether the redundant data enhancement condition is met can be performed within the current time window. If the number of times the redundant data enhancement condition is detected to be met within the current time window is greater than or equal to the preset number (for example, 3 times), it indicates that the network status of the first network path continues to be poor. At this time, redundant enhancement can be continuously turned on within a preset time length (for example, 2 seconds), that is, target redundant data equal to the target media data is always sent within the preset time length, thereby ensuring that the second network path is continuously used for redundant enhancement when the network status of the first network path is poor.

It should be noted that if the redundant data enhancement conditions are met at the current moment and the target redundant data equal to the target media data is transmitted, the decoding interval at the receiving end becomes shorter and the jamming rate decreases. This may cause redundant enhancement to be disabled at the next moment due to the redundant data enhancement conditions not being met, resulting in only the original redundant data being transmitted. However, disabling redundant enhancement will cause jamming again, and this cycle will repeat, thus failing to effectively improve the user experience. To address this, the embodiments of the present disclosure enable redundant enhancement for a preset duration when the redundant data enhancement conditions are met a number of times greater than or equal to a preset number within the current time window. This avoids repeated playback jams and effectively improves the user experience.

For example, if the number of times the redundant data enhancement condition is met within the current time window is less than a preset number, the redundant enhancement does not need to continue for a preset period of time. At this time, target redundant data equal to the target media data to be transmitted can be directly determined and sent.

S340: Transmit target media data through the first network path, and transmit target redundant data through the second network path.

The technical solution of the embodiment of the present disclosure determines whether the redundant data enhancement condition is currently met based on at least one of the current first network status information of the first network path and the current playback jamming information of the media data. Based on whether the redundant data enhancement condition is currently met, more suitable target redundant data can be determined, thereby ensuring that the data can be recovered at the receiving end, and improving the data recovery efficiency and success rate when the first network path is in poor condition.

FIG4 is a schematic structural diagram of a data transmission device provided by an embodiment of the present disclosure. As shown in FIG4 , the device specifically includes: a cross-path transmission detection module 410 , a redundant data determination module 420 and a data transmission module 430 .

Among them, the cross-path transmission detection module 410 is used to determine whether the redundant data cross-path transmission condition is currently met based on the current first network status information of the first network path and the current second network status information of the second network path; the redundant data determination module 420 is used to determine the target redundant data corresponding to the target media data currently to be transmitted if the redundant data cross-path transmission condition is currently met; and the data transmission module 430 is used to transmit the target media data through the first network path and transmit the target redundant data through the second network path.

The technical solution provided by the embodiments of the present disclosure determines whether a condition for cross-path transmission of redundant data is currently met based on current first network status information of the first network path and current second network status information of the second network path. If the condition for cross-path transmission of redundant data is currently met, target redundant data corresponding to the target media data to be transmitted is determined, the target media data is transmitted through the first network path, and the target redundant data is transmitted through the second network path, thereby dynamically offloading part of the traffic of the first network path to the second network path for transmission, achieving cross-path transmission of redundant data and improving the success rate of data reception without consuming a large amount of traffic on the second network path. This improves data transmission quality with lower traffic consumption and enhances user experience.

Based on the above technical solution, the cross-path transmission detection module 410 is specifically used to compare the current first round-trip delay of the first network path with the current second round-trip delay of the second network path, and determine whether the redundant data cross-path transmission condition is currently met based on the comparison result.

Based on the above technical solutions, the redundant data determination module 420 includes:

a redundant data enhancement detection unit, configured to determine whether a redundant data enhancement condition is currently met based on at least one of current first network state information of the first network path and current playback freeze information of the media data;

The redundant data determining unit is configured to determine target redundant data corresponding to the target media data to be transmitted based on whether a redundant data enhancement condition is currently satisfied.

Based on the above technical solutions, the redundant data enhancement detection unit is specifically used to:

detecting whether a current first round-trip delay of the first network path is greater than or equal to a preset delay, and whether a current decoding interval between two adjacent frames of the media data is greater than or equal to a preset time interval, and determining whether a redundancy enhancement condition is currently satisfied based on the detection results;

The current decoding interval between two adjacent frames in the media data is obtained by parsing the feedback message sent by the receiving end.

Based on the above technical solutions, the redundant data determination unit includes:

A first determining subunit is configured to determine target redundant data equal to the target media data to be currently transmitted if a redundant data enhancement condition is currently satisfied;

The second determining subunit is configured to determine target redundant data corresponding to the target media data to be transmitted based on the current packet loss rate of the first network path if the redundant data enhancement condition is not currently met.

On the basis of the above technical solutions, the first determining subunit is specifically configured to:

If the number of times the redundant data enhancement condition is met in the current time window is greater than or equal to the preset number, target redundant data equal to the target media data to be transmitted is determined within the preset duration, so as to continuously send the target redundant amount within the preset duration.

Based on the above technical solutions, the data transmission module 430 includes:

a transmission code rate determining unit, configured to determine, if a target transmission code rate corresponding to the target redundant data is greater than a current network bandwidth of the second network path, a first transmission code rate for transmitting the redundant data through the first network path and a second transmission code rate for transmitting the redundant data through the second network path based on the target transmission code rate and the current network bandwidth;

The redundant data transmission unit is configured to transmit part of the target redundant data at a first transmission code rate through the first network path, and transmit the remaining redundant data in the target redundant data at the second transmission code rate through the second network path.

Based on the above technical solutions, the sending code rate determination unit is specifically configured to:

Determine a difference between the target sending code rate and the current network bandwidth; determine the difference as a first sending code rate for transmitting redundant data along the first network path, and determine the current network bandwidth as a second sending code rate for transmitting redundant data along the second network path.

Based on the above technical solutions, the first network path is a wireless local area network path, and the second network path is a cellular network path.

The data transmission device provided in the embodiments of the present disclosure can execute the data transmission method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG5 , a schematic diagram of the structure of an electronic device (such as a terminal device or server in FIG5 ) 500 suitable for implementing an embodiment of the present disclosure is shown below. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG5 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG5 , electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of electronic device 500 are also stored in RAM 503. Processing device 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

Typically, the following devices may be connected to the I/O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

The electronic device provided by the embodiment of the present disclosure and the data transmission method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the data transmission method provided by the above embodiment is implemented.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device is caused to: determine whether a condition for cross-path transmission of redundant data is currently met based on current first network status information of the first network path and current second network status information of the second network path; if the condition for cross-path transmission of redundant data is currently met, determine target redundant data corresponding to target media data currently to be transmitted; transmit the target media data via the first network path, and transmit the target redundant data via the second network path.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, Example 1 provides a data transmission method, including:

determining whether a redundant data cross-path transmission condition is currently met based on current first network state information of the first network path and current second network state information of the second network path;

If the redundant data cross-path transmission condition is currently met, determining the target redundant data corresponding to the target media data to be transmitted;

The target media data is transmitted via the first network path, and the target redundant data is transmitted via the second network path.

According to one or more embodiments of the present disclosure, Example 2 provides a data transmission method, further comprising:

Optionally, the determining whether a redundant data cross-path transmission condition is currently met based on the current first network state information of the first network path and the current second network state information of the second network path includes:

The current first round-trip delay of the first network path is compared with the current second round-trip delay of the second network path, and based on the comparison result, it is determined whether a redundant data cross-path transmission condition is currently met.

According to one or more embodiments of the present disclosure, Example 3 provides a data transmission method, further comprising:

Optionally, determining target redundant data corresponding to target media data currently to be transmitted includes:

Determining whether a redundant data enhancement condition is currently met based on at least one of current first network state information of the first network path and current playback freeze information of the media data;

Based on whether the redundant data enhancement condition is currently satisfied, target redundant data corresponding to the target media data to be transmitted is determined.

According to one or more embodiments of the present disclosure, Example 4 provides a data transmission method, further comprising:

Optionally, the determining whether the redundancy enhancement condition is currently met based on at least one of the current first network state information of the first network path and the current playback freeze information of the media data includes:

detecting whether a current first round-trip delay of the first network path is greater than or equal to a preset delay, and whether a current decoding interval between two adjacent frames of the media data is greater than or equal to a preset time interval, and determining whether a redundancy enhancement condition is currently satisfied based on the detection results;

The current decoding interval between two adjacent frames in the media data is obtained by parsing the feedback message sent by the receiving end.

According to one or more embodiments of the present disclosure, Example 5 provides a data transmission method, further comprising:

Optionally, determining target redundant data corresponding to the target media data to be transmitted based on whether a redundant data enhancement condition is currently satisfied includes:

If the redundant data enhancement condition is currently met, determining target redundant data equal to the target media data to be transmitted;

If the redundant data enhancement condition is not currently met, target redundant data corresponding to the target media data to be transmitted is determined based on the current packet loss rate of the first network path.

According to one or more embodiments of the present disclosure, Example 6 provides a data transmission method, further comprising:

Optionally, if the redundant data enhancement condition is currently met, determining target redundant data equal to the target media data currently to be transmitted includes:

If the number of times the redundant data enhancement condition is met in the current time window is greater than or equal to the preset number, target redundant data equal to the target media data to be transmitted is determined within the preset duration, so as to continuously send the target redundant amount within the preset duration.

According to one or more embodiments of the present disclosure, Example 7 provides a data transmission method, further comprising:

Optionally, transmitting the target redundant data through the second network path includes:

If a target sending code rate corresponding to the target redundant data is greater than a current network bandwidth of the second network path, determining a first sending code rate for transmitting the redundant data through the first network path and a second sending code rate for transmitting the redundant data through the second network path based on the target sending code rate and the current network bandwidth;

Part of the target redundant data is sent at a first sending code rate through the first network path, and the remaining redundant data in the target redundant data is sent at the second sending code rate through the second network path.

According to one or more embodiments of the present disclosure, Example 8 provides a data transmission method, further comprising:

Optionally, determining, based on the target sending code rate and the current network bandwidth, a first sending code rate for transmitting redundant data over the first network path and a second sending code rate for transmitting redundant data over the second network path includes:

Determining a difference between the target sending bit rate and the current network bandwidth;

The difference is determined as a first sending code rate for transmitting redundant data over the first network path, and the current network bandwidth is determined as a second sending code rate for transmitting redundant data over the second network path.

According to one or more embodiments of the present disclosure, Example 9 provides a data transmission method, further comprising:

Optionally, the first network path is a wireless local area network path, and the second network path is a cellular network path.

According to one or more embodiments of the present disclosure, Example 10 provides a data transmission device, including:

a cross-path transmission detection module, configured to determine whether a redundant data cross-path transmission condition is currently satisfied based on current first network status information of the first network path and current second network status information of the second network path;

a redundant data determination module, configured to determine target redundant data corresponding to target media data to be transmitted if a redundant data cross-path transmission condition is currently met;

A data transmission module is configured to transmit the target media data through the first network path and transmit the target redundant data through the second network path.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (12)

A data transmission method, comprising: determining whether a redundant data cross-path transmission condition is currently met based on current first network state information of the first network path and current second network state information of the second network path; If the redundant data cross-path transmission condition is currently met, determining target redundant data corresponding to the target media data to be transmitted; The target media data is transmitted via the first network path, and the target redundant data is transmitted via the second network path. The data transmission method according to claim 1, wherein the determining whether a redundant data cross-path transmission condition is currently met based on the current first network state information of the first network path and the current second network state information of the second network path comprises: The current first round-trip delay of the first network path is compared with the current second round-trip delay of the second network path, and based on the comparison result, it is determined whether the redundant data cross-path transmission condition is currently met. The data transmission method according to claim 1 or 2, wherein determining the target redundant data corresponding to the target media data currently to be transmitted comprises: determining whether a redundant data enhancement condition is currently met based on at least one of current first network state information of the first network path and current playback freeze information of the media data; Based on whether the redundant data enhancement condition is currently satisfied, target redundant data corresponding to the target media data to be transmitted is determined. The data transmission method according to claim 3, wherein the determining whether the redundancy enhancement condition is currently met based on at least one of the current first network status information of the first network path and the current playback freeze information of the media data comprises: detecting whether a current first round-trip delay of the first network path is greater than or equal to a preset delay, and whether a current decoding interval between two adjacent frames of the media data is greater than or equal to a preset time interval, and determining whether the redundancy enhancement condition is currently satisfied based on the detection results; The current decoding interval between two adjacent frames in the media data is obtained by parsing a feedback message sent by the receiving end. The data transmission method according to claim 3 or 4, wherein determining the target redundant data corresponding to the target media data to be transmitted based on whether the redundant data enhancement condition is currently satisfied comprises: If the redundant data enhancement condition is currently met, determining target redundant data equal to the target media data to be transmitted; If the redundant data enhancement condition is not currently met, target redundant data corresponding to the target media data to be transmitted is determined based on the current packet loss rate of the first network path. The data transmission method according to claim 5, wherein if the redundant data enhancement condition is currently satisfied, determining target redundant data equal to the target media data currently to be transmitted comprises: If the number of times the redundant data enhancement condition is met within the current time window is greater than or equal to the preset number, the target redundant data equal to the target media data to be transmitted is determined within the preset duration, so as to continuously send the target redundant data within the preset duration. The data transmission method according to any one of claims 1 to 6, wherein transmitting the target redundant data through the second network path comprises: If a target sending code rate corresponding to the target redundant data is greater than a current network bandwidth of the second network path, determining a first sending code rate for transmitting the redundant data through the first network path and a second sending code rate for transmitting the redundant data through the second network path based on the target sending code rate and the current network bandwidth; Part of the target redundant data is sent at the first sending code rate through the first network path, and the remaining redundant data is sent at the second sending code rate through the second network path. The data transmission method according to claim 7, wherein determining, based on the target sending code rate and the current network bandwidth, a first sending code rate for transmitting redundant data on the first network path and a second sending code rate for transmitting redundant data on the second network path comprises: Determining a difference between the target sending bit rate and the current network bandwidth; The difference is determined as a first sending code rate for transmitting redundant data over the first network path, and the current network bandwidth is determined as a second sending code rate for transmitting redundant data over the second network path. The data transmission method according to any one of claims 1 to 8, wherein the first network path is a wireless local area network path, and the second network path is a cellular network path. A data transmission device, comprising: a cross-path transmission detection module configured to determine whether a redundant data cross-path transmission condition is currently met based on current first network state information of the first network path and current second network state information of the second network path; a redundant data determination module configured to determine target redundant data corresponding to target media data to be transmitted if the redundant data cross-path transmission condition is currently met; The data transmission module is configured to transmit the target media data through the first network path and transmit the target redundant data through the second network path. An electronic device, comprising: one or more processors; A storage device configured to store one or more programs, wherein When the one or more programs are executed by the one or more processors, the one or more processors implement the data transmission method according to any one of claims 1 to 9. A storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the data transmission method according to any one of claims 1 to 9.