CN115002087A - Audio and video data transmission method, system, device and storage medium - Google Patents

Abstract

The present application discloses a method, system, device and storage medium for audio and video data transmission, and relates to the field of information media. The audio and video data transmission method provided by the present application replaces the original ordinary network card by using a smart network card, and preprocesses the decapsulated data, and performs hardware decoding on the processed data through a hardware decoding device. The data is repackaged and transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Since all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding together, the problem of data transmission delay will not be caused.

Description

A kind of audio and video data transmission method, system, device and storage medium

technical field

The present application relates to the field of information media, and in particular, to a method, system, device and storage medium for audio and video data transmission.

Background technique

In recent years, with the development of self-media technology and the increasing amount of information, the original volume of high-quality audio and video is also increasing. Taking 720P 30fps video as an example, one pixel is about 3 bytes, which is generated every second. 1280*720*33*3/1024/1024=87MB data, the data volume in one minute is 5.22GB, such a huge video cannot be directly transmitted in the network, the emergence of video coding technology is to solve the problem of video data transmission in the network The problem.

Existing audio and video data are compressed by encoding technology and then transmitted to the cloud data center. Dedicated hardware is usually used for decoding. The advantages of dedicated hardware are high bandwidth, high performance, and low latency. Dedicated hardware logic processing for audio and video data is relatively Central processing unit (Central Processing Unit, CPU) processing can greatly reduce latency and resource consumption. However, under the cloud computing architecture, multiple virtual machines on a host cannot use dedicated hardware to decode at the same time, and can only decode by one, which results in a delay in data transmission.

In view of the above technologies, it is an urgent problem for those skilled in the art to find an audio and video data transmission method that can solve the problem of data transmission delay.

SUMMARY OF THE INVENTION

The purpose of this application is to provide an audio and video data transmission method, system, device and storage medium, so as to solve the problem that under the architecture of cloud computing, multiple virtual machines on a host cannot use dedicated hardware for decoding at the same time, and can only use dedicated hardware for decoding. Therefore, only a single decoding can be performed, which leads to the problem of data transmission delay.

In order to solve the above-mentioned technical problems, the present application provides an audio and video data transmission method, which is applied to a host machine using an intelligent network card, including:

Decapsulating the data packet, the data packet is a data packet containing the audio and video data that needs to be transmitted;

Preprocessing the decapsulated data, and performing hardware decoding on the processed data through a hardware decoding device;

The data is repackaged through the smart network card, and transmitted to the virtual machine through a driver, so as to generate an executable program running on the smart network card.

Preferably, before performing the preprocessing on the decapsulated data and performing hardware decoding on the processed data through a hardware decoding device, the method further includes:

The data to be processed is arranged in a priority queue, and the priority order of the priority queue is determined according to the size of the data.

Preferably, before decapsulating the data packet, the method further includes:

The TCP/UDP protocol of the network traffic of the host is offloaded.

Preferably, the preprocessing of the decapsulated data transmission includes:

Delete the label of the corresponding network ID in the data, and set the corresponding relationship between the ID of the virtual machine and the data.

Preferably, after the data is repackaged through the smart network card and transmitted to the virtual machine through the driver, the method further includes:

The original virtual machine code is reconstructed, and the GCC compiler is used to recompile the traffic preprocessing program of the smart network card to generate an executable program that can run on the smart network card, and the executable program corresponds to the audio and video data. audio and video.

Preferably, the memory device of the smart network card is used as the data cache and queue, and the PCIE interface is used as the smart network card to communicate with the hardware decoding device.

Preferably, before decapsulating the data packet, the method further includes:

A communication buffer between the hardware decoding device and the interface end of the smart network card is created, and the data packets preliminarily processed by the smart network card are buffered in the DDR memory of the smart network card.

In order to solve the above problems, the present application provides an audio and video data transmission device, including:

A decapsulation module for decapsulating a data packet, the data packet being a data packet containing audio and video data that needs to be transmitted;

a decoding module for preprocessing the decapsulated data transmission, and performing hardware decoding on the processed data;

The packaging module is used to repackage the data through the smart network card, and transmit the data to the virtual machine through the driver, so as to generate an executable program that can be run on the smart network card.

Preferably, the device further includes:

The arranging module is used for arranging the data to be processed in a priority queue, and the priority order of the priority queue is determined according to the size of the data.

Preferably, the device further includes:

An offloading module, configured to offload the TCP/UDP protocol of the network traffic of the host.

Preferably, the device further includes:

In order to solve the above problems, the present application also provides an audio and video data transmission device, which is characterized in that it includes a memory for storing a computer program;

The processor is configured to implement the steps of the above-mentioned audio and video data transmission method when executing the computer program.

Preferably, the device further includes:

The reconstruction module is used for reconstructing the original virtual machine code, using the GCC compiler to recompile the traffic preprocessing program of the smart network card, and generating an executable program that can be run on the smart network card, and the executable program is Audio and video corresponding to the audio and video data.

Preferably, the device further includes:

The cache module is configured to create a communication cache between the hardware decoding device and the interface end of the smart network card, and cache the data packets preliminarily processed by the smart network card to the DDR memory of the smart network card.

In order to solve the above-mentioned problems, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned audio and video data transmission method are realized. .

The audio and video data transmission method provided by this application replaces the original ordinary network card by using a smart network card, and preprocesses the decapsulated data, and performs hardware decoding on the processed data through a hardware decoding device. Repackage the data and transmit it to the virtual machine through the driver, so as to generate an executable program that can run on the smart network card. Because the smart network card decapsulates and preprocesses the data, the hardware decoding device can decode the data in the data packet. The data is decoded by hardware, and finally, after the decoding is completed, it is packaged by the smart network card and transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Because all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding, and after decoding, it is sent back to the smart network card for repackaging, and the data corresponding to the audio and video files It is sent to the corresponding virtual machine, so it will not cause the problem of data transmission delay.

The audio and video data transmission device and the computer-readable storage medium provided by the present application correspond to the above-mentioned audio and video data transmission method, so the beneficial effects are the same as above, and are not repeated here.

Description of drawings

In order to describe the embodiments of the present application more clearly, the following will briefly introduce the drawings that are used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, which are not relevant to ordinary skills in the art. As far as personnel are concerned, other drawings can also be obtained from these drawings on the premise of no creative work.

1 is a flowchart of a method for transmitting audio and video data according to an embodiment of the present application;

2 is a schematic diagram of an audio and video data transmission apparatus provided by an embodiment of the present application;

FIG. 3 is a structural diagram of an audio and video data transmission apparatus provided by another embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative work fall within the protection scope of the present application.

The core of the present application is to provide an audio and video data transmission method, system, device and storage medium, so as to solve the problem that under the architecture of cloud computing, multiple virtual machines on one host cannot use dedicated hardware to decode at the same time, and can only decode through dedicated hardware. Therefore, only a single decoding can be performed, which leads to the problem of data transmission delay.

In a packet-switched network, a single message is divided into multiple data blocks, called packets, which contain the sender and receiver address information. These packets are then transported across one or more networks along different paths and reassembled at the destination. The Open System Interconnection (OSI) model is a standard defined by the International Organization for Standardization (ISO), which defines a layered architecture in which each layer defines protocols for different communication levels. The OSI model has 7 layers. Layers 1 to 7 are: physical layer, data link layer, network layer, transport layer, session layer, presentation layer, and application layer. The OSI model can be logically divided into two parts: the lower layers 1 to 3 are concerned with the transmission of raw data; the upper layers 4 to 7 are concerned with the applications under the network. The traditional hardware network card only implements the L1-L2 layer logic in the network protocol, and the intelligent network card has an intelligent processing chip, which can process the logic of the L3-L4 layer.

In order to make those skilled in the art better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

Fig. 1 is a kind of audio and video data transmission method that the embodiment of this application provides, is applied to the host machine that adopts intelligent network card, comprises:

S10: Decapsulate the data packet, and the data packet is a data packet containing audio and video data to be transmitted;

It should be noted that data encapsulation, generally speaking, is to map the service data into the payload of a certain encapsulation protocol, and then fill in the header of the corresponding protocol to form the data packet of the encapsulation protocol, and complete the rate adaptation. This may not be easy to understand. Simply put, when data packets are transmitted between different devices using the network, in order to send them to the destination reliably and accurately, and to use transmission resources (transmission devices and transmission lines) efficiently, the Split and package the data packets, add the destination address, local address, and some bytes for error correction to the sent data packets, when the security and reliability are high, encryption processing is required, etc. . These operations are called data encapsulation. Decapsulation is the reverse process of encapsulation. It disassembles the protocol packet, processes the information in the packet header, and extracts the service information in the payload.

In this embodiment, the data information in the data packet is not specifically limited. It can be understood that the audio and video data to be transmitted in the data packet needs to be transmitted to multiple virtual machines. , different data streams need to be received, so different virtual machines use different ports for communication before, so as to manage the virtual machines.

S11: Preprocess the decapsulated data, and perform hardware decoding on the processed data through a hardware decoding device;

It should be noted that, in this embodiment, the decapsulated data is preprocessed. In the field of program design, preprocessing generally refers to the process before binary code is generated in the process of translating program source code into target code. Typically, the program source code text is processed by a preprocessor, and the obtained result is further compiled by the compiler core. It can be understood that preprocessing is generally performed by software. In this embodiment, for The specific steps of preprocessing are not limited.

In this embodiment, decoding is the process of using a specific method to restore the digital code to the content it represents, or to convert electrical pulse signals, optical signals, radio waves, etc., into the information, data, etc. it represents. In this embodiment, the specific process of the hardware and the specific selection of the decoding device are not limited. It can be understood that the hardware decoding device may include but not limited to the following devices.

Digital Signal Processing (DSP) is a computer or special processing device that collects, transforms, filters, evaluates, enhances, compresses, and recognizes signals in digital form to obtain a signal form that meets user needs. Digital signal processing and analog signal processing are subsets of signal processing. The purpose of digital signal processing is to measure or filter real-world continuous analog signals. Therefore, the signal needs to be converted from the analog domain to the digital domain before digital signal processing, which is usually achieved by an analog-to-digital converter. The output of digital signal processing is often also transformed into the analog domain, which is achieved through a digital-to-analog converter.

Field-Programmable Gate Array (FPGA) is the product of its further development on the basis of programmable devices. It appears as a semi-custom circuit in the field of application-specific integrated circuits, which not only solves the shortcomings of custom circuits, but also overcomes the shortcomings of the limited number of original programmable device gate circuits.

fold

S12: Repackage the data through the smart network card, and transmit it to the virtual machine through the driver, so as to generate an executable program running on the smart network card.

Virtual machine, in the architecture of computer science, refers to a special kind of software that can create an environment between the computer platform and the end user, and the end user operates the software based on the environment created by the software . In computer science, a virtual machine refers to a software implementation of a computer that can run programs like a real machine. Virtual machine technology is a kind of virtualization technology. The so-called virtualization technology is to change things from one form to another. The most commonly used virtualization technology is the virtualization of memory in the operating system. The memory space may be much larger than the memory size of the physical machine. Using the memory virtualization technology, the user can virtualize a part of the hard disk into memory, which is transparent to the user.

The driver is a small piece of code added to the operating system, which contains relevant hardware devices. The driver is a configuration file written by the hardware manufacturer according to the operating system. It can be said that without the driver, the hardware in the computer cannot work. Different operating systems have different hardware drivers. In order to ensure the compatibility of the hardware and enhance the functions of the hardware, each hardware manufacturer will continuously upgrade the drivers. In this embodiment, the specific type of the driver is not limited. , the executable program mentioned in this embodiment is the original audio and video file transmitted, and the specific process and the like for generation are not limited in this embodiment.

The audio and video data transmission method provided by this embodiment replaces the original ordinary network card with a smart network card, preprocesses the decapsulated data, and performs hardware decoding on the processed data through a hardware decoding device. The network card repackages the data and transmits it to the virtual machine through the driver, so as to generate executable programs that can run on the smart network card. Because the smart network card decapsulates and preprocesses the data, the hardware decoding device can The data is decoded by hardware, and finally, after the decoding is completed, it is packaged by the smart network card and transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Because all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding, and after decoding, it is sent back to the smart network card for repackaging, and the data corresponding to the audio and video files It is sent to the corresponding virtual machine, so it will not cause the problem of data transmission delay.

In the above-mentioned embodiment, it is mentioned that before the decapsulated data is preprocessed, and the processed data is decoded by hardware decoding equipment, it also includes:

Arrange the data to be processed according to the priority queue, and the priority order of the priority queue is determined according to the size of the data.

The priority is a parameter that determines the priority level of each operating program to receive system resources when the computer time-sharing operating system processes multiple operating programs. Among the logical operators, the logical NOT operator (!) is at the same level as all unary operators, and is higher than the binocular arithmetic operator, relational operator, logical operator and (&) and logical operator or (‖) ; AND operator is higher than OR operator, they are lower than arithmetic and relational operators. In this embodiment, only the priority order is limited to be related to the data size, and the specific calculation method of the priority and other related calculation parameters and the like are not limited.

It should be noted that, in this embodiment, data is prioritized first, so as to ensure the smoothness of data processing and prevent data redundancy.

Considering the influence of the previous communication protocol of the host on this data transmission, before decapsulating the data packet, it also includes:

Offloads the TCP/UDP protocol of the host's network traffic.

Transmission Control Protocol (TCP) is a connection-oriented (connection-oriented), reliable, byte stream-based transport layer (Transport layer) communication protocol. In the simplified computer network Open System Interconnection (OSI) model, it completes the functions specified by the fourth transport layer, and the User Datagram Protocol (UDP) is another important function in the same layer. The transport protocol is a connectionless transport layer protocol in the OSI reference model that provides transaction-oriented simple unreliable information transfer services. IETF RFC 768 is the formal specification of UDP.

The application layer sends a data stream represented by 8-bit bytes for Internet transmission to the TCP layer, and then TCP divides the data stream into segments of appropriate length (usually controlled by the data link layer of the network to which the computer is connected). Maximum Transmission Unit (MTU) limit). Then TCP passes the result packet to the IP layer, which transmits the packet through the network to the TCP layer of the receiving entity.

UDP is a connectionless transport layer protocol in the OSI reference model. It is mainly used in transmissions that do not require the order of packets to arrive. The check and sorting of the packet transmission order is completed by the application layer, providing transaction-oriented simple and unreliable information transmission services. . The UDP protocol is basically the interface between the IP protocol and the upper layer protocol. The UDP protocol is applicable to multiple applications running on the same device.

By uninstalling the above protocol, the communication mode for the host is cleared, thereby ensuring the normal transmission of audio and video data communication.

The specific method of preprocessing is not limited in the above-mentioned embodiments, and a preferred solution is provided here. The preprocessing of the decapsulated data transmission includes:

Delete the label of the corresponding network ID in the data, and set the corresponding relationship between the ID of the virtual machine and the data.

It should be noted that the identity identification code (Identity, ID), also known as serial number or account number, is a relatively unique code in a certain system, which is equivalent to a kind of "identity card". In a specific thing, the ID number It is generally unchanged. As for what is used to identify the thing, it is determined by the rules made by the designer himself. Therefore, during preprocessing, the corresponding network address in the data is deleted in advance, and the label in this embodiment refers to the The embodiment of the network ID in the character field of the data, the ID of the virtual machine can be simply understood as the address symbol corresponding to the virtual machine in the program, so that the above preprocessing method is used to prevent data errors in subsequent transmissions, and the virtual machine's Addresses correspond to data, ensuring that each data is transferred to the correct virtual machine.

Considering the practicability of the final solution, that is, for the generation of related audio and video files after the transmission is completed, a preferred solution is provided here. After the data is repackaged through the smart network card and transmitted to the virtual machine through the driver, it also includes:

The original virtual machine code is reconstructed, and the GCC compiler is used to recompile the traffic preprocessing program of the smart network card to generate an executable program that can run on the smart network card. The executable program is the audio and video corresponding to the audio and video data.

It should be noted that an Open VSwitch (OVS) is a high-quality, multi-layer virtual switch (layers of network layers) designed to allow large-scale network automation to scale programmatically, while still supporting standard management Interfaces and protocols: NetFlow.SFlow...etc, and it also supports a distributed environment of multiple physical machines. The software is used to form switching components, so it is also called a software switch. Compared with traditional physical switches, virtual switches have many advantages: flexible configuration, because it is implemented by software, a physical server can be configured with dozens of ethers or Hundreds of virtual switches, and the number of ports can be flexibly selected. The cost is low, and the switching speed of 10Gbps can be easily achieved through software.

The GNU Compiler Collection (GUN) contains front-ends for many programming languages, as well as libraries for these languages. Free software released under the GPL license is a key part of the GNU project. Originally used as the official compiler of the GNU operating system, GCC has now been adopted as the standard compiler by most Unix-like operating systems.

In this embodiment, a subsequent executable file generation method is added, thereby ensuring the integrity of the solution and optimizing the user experience.

In the above embodiments, neither the communication mode nor the data buffering mode is limited, and a preferred solution is provided here. The memory device of the smart network card is used as the data cache and queue, and the PCIE interface is used as the smart network card to communicate with the hardware decoding device.

High-speed serial computer expansion bus standard (peripheral component interconnect express, PCIE), including higher maximum system bus throughput, lower I/O pin count and smaller physical size, better bus device performance scaling, More detailed error detection and reporting mechanism (Advanced Error Reporting, AER) and native hotplug capability. An updated version of the PCIE standard provides hardware support for I/O virtualization.

Considering the communication between the hardware decoding device and the smart network card, a preferred solution is provided here. Before decapsulating the data packet, it also includes:

Create a communication buffer between the hardware decoding device and the interface of the smart network card, and cache the data packets preliminarily processed by the smart network card in the DDR memory of the smart network card.

Double Data Rate (DDR) memory, compared with the traditional single data rate, DDR technology realizes two read/write operations in one clock cycle, that is, one read is performed on the rising edge and the falling edge of the clock. /write operations, so the cache rate is faster and the business is more efficient.

In this embodiment, a better caching method is provided to ensure that

In order to solve the above problems, the present application provides an audio and video data transmission device, including:

The decapsulation module is used to decapsulate the data packet, and the data packet is a data packet containing the audio and video data to be transmitted;

The decoding module is used to preprocess the decapsulated data transmission and perform hardware decoding on the processed data;

The packaging module is used to repackage data through the smart network card and transmit it to the virtual machine through the driver, so as to generate an executable program that can be run on the smart network card.

In the above embodiments, the audio and video data transmission method is described in detail, and the present application also provides the corresponding embodiment of the audio and video data transmission apparatus. It should be noted that this application describes the embodiments of the device part from two perspectives, one is based on the perspective of functional modules, and the other is based on the perspective of hardware.

FIG. 2 is a schematic diagram of an audio and video data transmission device provided by an embodiment of the application, and the device includes:

The decapsulation module 10 is used to decapsulate the data packet, and the data packet is a data packet containing the audio and video data to be transmitted;

The decoding module 11 is used to preprocess the decapsulated data transmission, and perform hardware decoding on the processed data;

The packaging module 12 is used to repackage the data through the smart network card, and transmit the data to the virtual machine through the driver, so as to generate an executable program that can be run on the smart network card.

Preferably, the device further includes:

The arrangement module is used to arrange the data to be processed according to the priority queue, and the priority order of the priority queue is determined according to the size of the data.

Preferably, the device further includes:

The offloading module is used to offload the TCP/UDP protocol of the network traffic of the host.

Preferably, the device further includes:

In order to solve the above problems, the present application also provides an audio and video data transmission device, which is characterized in that it includes a memory for storing a computer program;

The processor is configured to implement the steps of the above-mentioned audio and video data transmission method when executing the computer program.

Preferably, the device further includes:

The reconstruction module is used to reconstruct the original virtual machine code, use the GCC compiler to recompile the traffic preprocessing program of the smart network card, and generate an executable program that can run on the smart network card. The executable program corresponds to the audio and video data. audio and video.

Preferably, the device further includes:

The cache module is used to create a communication cache between the hardware decoding device and the interface end of the smart network card, and cache the data packets preliminarily processed by the smart network card to the DDR memory of the smart network card.

Since the embodiments of the apparatus part and the embodiments of the method part correspond to each other, the embodiments of the apparatus part and the corresponding beneficial effects may refer to the description of the embodiments of the method part, which will not be repeated here.

The audio and video data transmission device provided in this embodiment includes a decapsulation module, a decoding module and a packaging module, which can replace the original ordinary network card by using a smart network card, preprocess the decapsulated data, and process the decapsulated data. The resulting data is decoded by hardware decoding equipment, repackaged by the smart network card, and transmitted to the virtual machine through the driver, so as to generate executable programs that can run on the smart network card. processing, so that the hardware decoding device can perform hardware decoding on the data in the data packet, and finally, after the decoding is completed, it is packaged by the intelligent network card, and then transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Because all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding, and after decoding, it is sent back to the smart network card for repackaging, and the data corresponding to the audio and video files It is sent to the corresponding virtual machine, so it will not cause the problem of data transmission delay.

FIG. 3 is a structural diagram of an audio and video data transmission device provided by another embodiment of the application. As shown in FIG. 3 , the audio and video data transmission device includes: a memory 20 for storing a computer program;

The processor 21 is configured to implement the steps of the audio and video data transmission method mentioned in the above embodiments when executing the computer program.

The audio and video data transmission apparatus provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer.

The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be implemented by at least one hardware form among digital signal processing (Digital Signal Processor, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). . The processor 21 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake-up state, also called a central processing unit (CPU); A low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a graphics processor (Graphics Processing Unit, GPU), and the GPU is used for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor 21 may further include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.

Memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, the relevant steps of the audio and video data transmission method disclosed in any of the foregoing embodiments can be implemented. In addition, the resources stored in the memory 20 may also include an operating system 202, data 203, etc., and the storage mode may be short-term storage or permanent storage. The operating system 202 may include Windows, Unix, Linux, and the like. The data 203 may include, but is not limited to, the data involved in the audio and video data transmission method, and the like.

In some embodiments, the audio and video data transmission device may further include a display screen 22 , an input/output interface 23 , a communication interface 24 , a power supply 25 and a communication bus 26 .

Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the audio and video data transmission apparatus, and may include more or less components than those shown in the figure.

The audio and video data transmission apparatus provided by the embodiment of the present application includes a memory and a processor, and the processor can implement the following method when executing a program stored in the memory: the audio and video data transmission method involved in the above embodiments.

Since the embodiment of the apparatus part corresponds to the embodiment of the method part, for the embodiment of the apparatus part, please refer to the description of the embodiment of the method part, which will not be repeated here.

The audio and video data transmission device provided in this embodiment includes a memory and a processor. When the processor executes a program stored in the memory, the original ordinary network card can be replaced by a smart network card, and the decapsulated data can be pre-processed. Process, and decode the processed data through hardware decoding equipment, repackage the data through the smart network card, and transmit it to the virtual machine through the driver, so as to generate executable programs that can run on the smart network card. Decapsulation and preprocessing are performed, so that the hardware decoding device can perform hardware decoding on the data in the data packet. Finally, after the decoding is completed, it is packaged by the smart network card and transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Because all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding, and after decoding, it is sent back to the smart network card for repackaging, and the data corresponding to the audio and video files It is sent to the corresponding virtual machine, so it will not cause the problem of data transmission delay.

Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps described in the foregoing method embodiments are implemented.

It can be understood that, if the methods in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

Since the embodiments of the readable storage medium part correspond to the embodiments of the method part, the embodiments of the apparatus part refer to the description of the embodiments of the method part, which will not be repeated here.

The computer-readable storage medium provided by this embodiment stores a computer program, and when the computer program is executed by the processor, the original ordinary network card is replaced by using a smart network card, and the decapsulated data is preprocessed and processed. The resulting data is decoded by hardware decoding equipment, repackaged by the smart network card, and transmitted to the virtual machine through the driver, so as to generate executable programs that can run on the smart network card. processing, so that the hardware decoding device can perform hardware decoding on the data in the data packet, and finally, after the decoding is completed, it is packaged by the intelligent network card, and then transmitted to the virtual machine through the driver. Compared with the previous direct communication between the decoding hardware and the virtual machine, in this embodiment, the multi-layer virtual switch function of the intelligent network card is used to realize the communication between the virtual machine in the host machine and the hardware decoding device by using the intelligent network card as an intermediate medium. Because all the data of the received audio and video files can be transmitted to the hardware decoding device through the multi-layer virtual switch function of the smart network card for decoding, and after decoding, it is sent back to the smart network card for repackaging, and the data corresponding to the audio and video files It is sent to the corresponding virtual machine, so it will not cause the problem of data transmission delay.

The audio and video data transmission method, device, and computer-readable storage medium provided by the present application have been described in detail above. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present application, several improvements and modifications can also be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations. There is no such actual relationship or sequence between operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Claims (10)

1. An audio and video data transmission method is characterized by being applied to a host machine adopting an intelligent network card and comprising the following steps:

decapsulating a data packet, wherein the data packet is a data packet containing audio and video data to be transmitted;

preprocessing the decapsulated data, and performing hardware decoding on the processed data through hardware decoding equipment;

and repackaging data through the intelligent network card, and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program running on the intelligent network card.

2. The method for transmitting audio-video data according to claim 1, wherein before the preprocessing the decapsulated data and performing hardware decoding on the processed data by a hardware decoding device, the method further comprises:

and arranging the data to be processed according to a priority queue, wherein the priority order of the priority queue is determined according to the size of the data.

3. The method for transmitting audio-video data according to claim 2, further comprising, before the decapsulating the data packet:

and unloading the TCP/UDP protocol of the network flow of the host machine.

4. The method for transmitting audio-video data according to claim 2, wherein the preprocessing the decapsulated data transmission comprises:

and deleting the label of the corresponding network ID in the data, and setting the corresponding relation between the ID of the virtual machine and the data.

5. The audio-video data transmission method according to any one of claims 1 to 4, wherein after the data is repackaged by the smart network card and is transmitted to the virtual machine by driving, the method further comprises:

reconstructing codes of an original virtual machine, and recompiling a flow preprocessing program of the intelligent network card by adopting a GCC compiler to generate an executable program which can run on the intelligent network card, wherein the executable program is audio and video corresponding to the audio and video data.

6. The audio/video data transmission method according to claim 2, wherein the memory device of the intelligent network card is used as a data buffer and a queue, and the PCIE interface is used as the intelligent network card to communicate with the hardware decoding device.

7. The method for transmitting audio-video data according to claim 1, further comprising, before the decapsulating the data packet:

and creating a communication cache of the hardware decoding device and the interface end of the intelligent network card, and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.

8. An audio-video data transmission apparatus, comprising:

the de-encapsulation module is used for de-encapsulating a data packet, wherein the data packet contains audio and video data to be transmitted;

the decoding module is used for preprocessing the unpackaged data transmission and performing hardware decoding on the processed data;

and the packaging module is used for repackaging the data through the intelligent network card and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program capable of running on the intelligent network card.

9. An audio-video data transmission apparatus, comprising a memory for storing a computer program;

a processor for implementing the steps of the audio-visual data transmission method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the audio-visual data transmission method according to one of claims 1 to 7.

Images