WO2024169436A1 - Data processing method and apparatus, and storage medium and electronic apparatus - Google Patents

Abstract

A data processing method and apparatus, and a storage medium and an electronic apparatus. The method comprises: initializing a virtio frontend device by means of a control plane at a virtio backend interacting with the virtio frontend device (S202); and processing input data and/or output data of the virtio frontend device by means of a data plane at the virtio backend (S204).

Description

Data processing method, device, storage medium and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is based on Chinese patent application CN202310152011.0 filed on February 15, 2023, entitled “A data processing method, device, storage medium and electronic device”, and claims the priority of the patent application. All the contents disclosed therein are incorporated into the present disclosure by reference.

Technical Field

The embodiments of the present disclosure relate to the computer field, and in particular, to a data processing method, device, storage medium, and electronic device.

Background Art

Cloud computing has been deployed on a large scale in the industry. Assuming that each physical server runs 10 virtual machines, and each virtual machine has 1 virtual network console (VNC) and console device, it will require 1 million VNC and console devices. These devices will continue to simulate as long as there is output in the virtual machine. Assuming that the data plane simulation of each device only occupies 0.1% of the physical central processing unit (CPU), a total of 1,000 physical CPUs are required to simulate these devices. In most cases, these devices are not used by users and are idling in the background, resulting in a waste of computing resources.

There is no solution yet to the problem in related technologies that device simulation will continue as long as there is output in the virtual machine, resulting in a waste of computing resources.

Summary of the invention

The embodiments of the present disclosure provide a data processing method, apparatus, storage medium and electronic device to at least solve the problem in the related art that device simulation will continue as long as there is output in the virtual machine, resulting in a waste of computing resources.

According to an embodiment of the present disclosure, a data processing method is provided, the method comprising: initializing the virtio front-end device by interacting with the virtio front-end device through the control plane of the virtio back-end; processing the input data and/or output data of the virtio front-end device through the data plane of the virtio back-end.

According to another embodiment of the present disclosure, a data processing device is also provided, comprising: an initialization module, configured to initialize the virtio front-end device by interacting with the virtio front-end device through the control plane of the virtio backend; and a processing module, configured to process input data and/or output data of the virtio front-end device through the data plane of the virtio backend.

According to another embodiment of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

According to another embodiment of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a hardware structure block diagram of a computer device of a data processing method according to an embodiment of the present disclosure;

FIG2 is a flow chart of a data processing method according to an embodiment of the present disclosure;

FIG3 is a flow chart of a data processing method according to an optional embodiment of the present disclosure;

FIG4 is a block diagram of the separation of the data plane and the control plane according to an embodiment of the present disclosure;

FIG. 5 is a block diagram of a data processing device according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

The method embodiments provided in the embodiments of the present disclosure can be executed in a computer device or a similar computing device. Taking running on a computer device as an example, FIG. 1 is a hardware structure block diagram of a computer device of the data processing method of the embodiment of the present disclosure. As shown in FIG. 1 , the computer device may include one or more (only one is shown in FIG. 1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device) and a memory 104 for storing data, wherein the above-mentioned computer device may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those of ordinary skill in the art that the structure shown in FIG. 1 is only for illustration, and it does not limit the structure of the above-mentioned computer device. For example, the computer device may also include more or fewer components than those shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data processing method in the embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

In this embodiment, a data processing method running on the above-mentioned computer device is provided. FIG. 2 is a flow chart of the data processing method according to an embodiment of the present disclosure. As shown in FIG. 2 , the flow chart includes the following steps:

Step S202, initializing the virtio front-end device by interacting with the virtio front-end device through the control plane of the virtio back-end;

Virtio in this embodiment is an abstraction layer located above the device in the paravirtualized Hypervisor.

In the embodiment of the present disclosure, the above step S202 may specifically include: receiving a configuration message of the virtio front-end device through the control plane; and returning corresponding data to the virtio front-end device to initialize the virtio front-end device.

Step S204: Process the input data and/or output data of the virtio front-end device through the data plane of the virtio back-end.

Through the above steps S202 to S204, the virtio front-end device is initialized by interacting with the virtio front-end device through the control plane of the virtio back-end; the input data and output data of the virtio front-end device are processed through the data plane of the virtio back-end, which can solve the problem that device simulation will continue as long as there is output in the virtual machine, resulting in waste of computing resources in the related technology. The virtio front-end device is initialized by the control plane to process the input data and/or output data of the virtio front-end device, so as to realize the separation of the control plane and the data plane of the virtio back-end device, and only the control plane of the virtio device and the data plane without function simulation are realized in the absence of user access, thereby avoiding unnecessary waste of resources.

FIG3 is a flow chart of a data processing method according to an optional embodiment of the present disclosure. As shown in FIG3 , the above step S204 specifically includes:

Step S302: receiving and discarding output data of the virtio front-end device through the first sub-data plane when no target user accesses the virtio front-end device;

Step S304: When a target user accesses the virtio front-end device, input data and/or output data of the virtio front-end device is received through the first sub-data plane, and the input data and/or output data is processed through the first sub-data plane and a second sub-data plane corresponding to the first sub-data plane.

The data plane in the disclosed embodiment includes the above-mentioned first sub-data plane and second sub-data plane. The second sub-data plane is added when it is detected that the target user accesses the virtio front-end device. The function set of the control plane and the first sub-data plane in this embodiment can be called an intermediate layer, which can be located in the same process of the Hypervisor or in other processes.

In one embodiment, after the above step S302, the method may further include: receiving a configuration message of the virtio front-end device through the control plane; returning corresponding data to the virtio front-end device to initialize the virtio front-end device, and driving the virtio front-end device to generate output data.

In this embodiment, the above step S304 may specifically include: sending the output data to the second sub-data plane through the first sub-data plane and the virtio backend transport layer in sequence, and sending the output data to the target user through the second sub-data plane; and/or detecting the input data of the target user accessing the virtio front-end device through the second sub-data plane, and sending the input data to the virtio front-end device through the second sub-data plane, the virtio backend transport layer, and the first sub-data plane in sequence.

In one embodiment, before the above step S302, the method further includes: determining whether it is detected that the target user accesses the virtio front-end device; and if the determination result is yes, adding a second sub-data plane corresponding to the first sub-data plane.

In another embodiment, the method further includes: deleting the second sub-data plane when the target user stops accessing the virtio front-end device through the second sub-data plane.

This embodiment adds an intermediate layer (a functional set of the control plane and the first sub-data plane) to the virtio backend to separate the control plane and data plane of the virtio backend device. For the guest, the virtio device is still loaded. The guest has no perception of the actual separation of the control plane and data plane of the backend.

The control plane is mainly responsible for communicating with the guest virtual machine, initializing and loading the device, and ensuring that the corresponding virtio device in the guest can be loaded normally. Each device must implement the control plane.

The data plane is mainly responsible for communicating with the guest and for data transmission of virtio devices, such as the output of the console, the graphic frames of VNC, and other data. The data plane implements an empty device by default, and the sent and received data can be directly discarded. When the user has actual device access needs, the corresponding virtio backend data plane simulation is dynamically added; when the user's access is over, the corresponding virtio backend data plane simulation is dynamically deleted. In this way, when the user does not access devices such as VNC/Console, the computing resource overhead of the actual device data plane simulation can be saved.

Furthermore, the data plane can be based on multiple transport layers, including but not limited to pcie/tcp/udp/vsock, etc. Device simulation can select the appropriate transport layer to meet various business requirements, such as security reinforcement, security auditing, and cost savings.

If the data plane is based on the IP transport layer, the data plane can be simulated on the remote host. If there are 100 virtual machines on the computing nodes in the cluster, but the operation and maintenance personnel or cloud platform users only access the VNC/Console of 20 virtual machines at the same time, it is only necessary to simulate the data plane of 20 VNC/Console devices on a certain management node; at night, holidays and other idle time, the virtio backend device can basically not occupy the computing resources of the server. Figure 4 is a block diagram of the separation of the data plane and the control plane according to an embodiment of the present disclosure, as shown in Figure 4, including:

The Hypervisor prepares the control plane and the first sub-data plane corresponding to the virtio backend middle layer. Each virtio backend middle layer has and only has one control plane and one first sub-data plane.

The GuestOS (the operating system of the client virtual machine) starts to start, and the device manager in the Guest traverses the virtio front-end devices and sends the configuration message of the virtio front-end devices to the control plane.

After receiving the device configuration message, the control plane returns the corresponding data to the device manager in the Guest according to the type of virtio device.

The device manager in the Guest initializes the virio front-end device based on the data returned by the control plane and successfully loads the virtio front-end device driver.

The virtio front-end device starts generating output data and sends it to the first sub-data plane.

After the first sub-data plane receives the output data, the second sub-data plane corresponding to the first sub-data plane does not exist yet, so the data is directly discarded and the correct response data is returned according to the virtio front-end device type.

The user needs to access the virtio front-end device of the Guest. At this time, the second sub-data plane corresponding to the first sub-data plane is added according to the front-end device currently accessed. The first sub-data plane and the second sub-data plane of each front-end device are in one-to-one correspondence.

If the user wants to input data to the virtio front-end device, the second sub-data plane will send the input data to the first sub-data plane through the virtio back-end transport layer, and then the first sub-data plane will send the input data to the virtio front-end device according to the virtio protocol.

If the virtio front-end device has output data, the virtio front-end device sends the output data to the first sub-data plane according to the virtio protocol, and then the first sub-data plane sends the output data to the second sub-data plane through the virtio back-end transport layer.

The user stops accessing the virtio front-end device of the Guest, and the second sub-data plane corresponding to the first sub-data plane is deleted and cleared.

The device manager in the guest includes but is not limited to the operating system kernel program, user-mode program, etc. The virtio backend intermediate layer can be implemented as, but is not limited to, kernel-mode and user-mode programs. Hypervisors include but are not limited to qemu, cloud-Hypervisor, etc. Virtio protocols include but are not limited to virtiolagacy, virtio 1.0, virtio 1.1, virtio 1.2 and later virtio protocol versions released by OASIS, as well as other related protocols compatible with the standard virtio protocol. Virtio device types include but are not limited to Console, VNC and other device types. In theory, any virtio device type can be supported. Users can realize the separation of the control plane and data plane of the backend of only some virtio devices according to their own needs. Guest os only needs to support the driver of virtio devices, including but not limited to linux, windows, etc. The virtio backend transport layer only needs to support reading and writing data, including but not limited to pcie, rdma, dma, tcp, udp, vsock, etc. The virtio device backend simulation can run any location accessible by the virito backend transport layer (such as physical machines, dedicated devices, virtual machines, etc.).

Through this embodiment, the computing resources of the virtual machine platform are saved. When the user does not access the corresponding device, the device does not The corresponding virtio backend data plane does not occupy computing resources. Improve the security of the hypervisor. By selecting a suitable transport layer, such as TCP, the data plane of the virtio device can run on a server other than the hypervisor where the current virtual machine is located, thereby effectively reducing the threat of virtio device vulnerabilities and GuestOS vulnerabilities to the hypervisor. Reduce the server cost of the cloud platform. By selecting a suitable transport layer, such as TCP, the data plane of the virtio device can run on a server other than the hypervisor where the current virtual machine is located. The data plane of the virtio device can be run on a cheaper server, saving costs.

According to another aspect of the embodiment of the present disclosure, a data processing device is further provided. FIG. 5 is a block diagram of the data processing device according to the embodiment of the present disclosure. As shown in FIG. 5 , the device includes:

An initialization module 52 is configured to initialize the virtio front-end device by interacting with the virtio front-end device through a virtio back-end control plane;

The processing module 54 is configured to process the input data and/or output data of the virtio front-end device through the data plane of the virtio back-end.

In one embodiment, the initialization module is further configured to receive a configuration message of the virtio front-end device through the control plane; and return corresponding data to the virtio front-end device to initialize the virtio front-end device.

In one embodiment, the processing module 54 includes:

a discarding submodule, configured to receive and discard output data of the virtio front-end device through the first sub-data plane when no target user accesses the virtio front-end device;

a processing submodule, configured to receive the input data and/or the output data of the virtio front-end device through a first sub-data plane when a target user accesses the virtio front-end device, and process the input data and/or the output data through the first sub-data plane and a second sub-data plane corresponding to the first sub-data plane;

The data plane includes the first sub-data plane and the second sub-data plane, and the second sub-data plane is added when it is detected that the target user accesses the virtio front-end device.

In one embodiment, the device further comprises:

The return module is configured to return corresponding response data according to the device type of the virtio front-end device.

In one embodiment, the processing submodule is further configured to send the output data to the second sub-data plane through the first sub-data plane and the virtio backend transport layer in sequence, and send the output data to the target user through the second sub-data plane; and/or detect through the second sub-data plane that the target user accesses the input data of the virtio front-end device; and send the input data to the virtio front-end device through the second sub-data plane, the virtio backend transport layer, and the first sub-data plane in sequence.

In one embodiment, the device further comprises:

A judgment module is configured to judge whether the target user is detected to access the virtio front-end device;

The adding module is configured to add the second sub-data plane corresponding to the first sub-data plane if the judgment result is yes.

In one embodiment, the device further comprises:

The deletion module is configured to delete the second sub-data plane when the target user stops accessing the virtio front-end device through the second sub-data plane.

The embodiment of the present disclosure also provides a computer-readable storage medium storing a computer-readable storage medium. A computer program, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

In an exemplary embodiment, the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.

For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

Obviously, those skilled in the art should understand that the above modules or steps of the present disclosure can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims (10)

A data processing method, the method comprising: Initializing the virtio front-end device by interacting with the virtio front-end device through the control plane of the virtio back-end; The input data and/or output data of the virtio front-end device is processed through the data plane of the virtio back-end. The method according to claim 1, wherein initializing the virtio front-end device by interacting with the virtio front-end device through the control plane of the virtio back-end comprises: Receiving a configuration message of the virtio front-end device through the control plane; Return corresponding data to the virtio front-end device to initialize the virtio front-end device. The method according to claim 1, wherein processing the input data and/or output data of the virtio front-end device through the data plane of the virtio back-end comprises: In the case where there is no target user accessing the virtio front-end device, receiving and discarding output data of the virtio front-end device through the first sub-data plane; In the case where a target user accesses the virtio front-end device, receiving the input data and/or the output data of the virtio front-end device through the first sub-data plane, and processing the input data and/or the output data through the first sub-data plane and a second sub-data plane corresponding to the first sub-data plane; The data plane includes the first sub-data plane and the second sub-data plane, and the second sub-data plane is added when it is detected that the target user accesses the virtio front-end device. The method according to claim 3, wherein, after receiving and discarding the output data of the virtio front-end device through the first sub-data plane, the method further comprises: Return corresponding response data according to the device type of the virtio front-end device. The method according to claim 3, wherein processing the input data and/or the output data through the first sub-data plane and a second sub-data plane corresponding to the first sub-data plane comprises: Sending the output data to the second sub-data plane through the first sub-data plane and the virtio backend transport layer in sequence, and sending the output data to the target user through the second sub-data plane; and/or The target user accesses the input data of the virtio front-end device through the second sub-data plane, and the input data is sent to the virtio front-end device through the second sub-data plane, the virtio back-end transport layer, and the first sub-data plane in sequence. The method according to any one of claims 3 to 5, wherein, before processing the input data and/or output data of the virtio front-end device through the data plane of the virtio back-end, the method further comprises: Determining whether the target user is detected to access the virtio front-end device; If the judgment result is yes, the second sub-data plane corresponding to the first sub-data plane is added. The method according to claim 6, wherein the method further comprises: When the target user stops accessing the virtio front-end device through the second sub-data plane, the second sub-data plane is deleted. A data processing device, comprising: An initialization module, configured to initialize the virtio front-end device by interacting with the virtio front-end device through a virtio back-end control plane; A processing module is configured to process input data and/or output data of the virtio front-end device through a data plane of the virtio back-end. A computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the method described in any one of claims 1 to 7 when run. An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method described in any one of claims 1 to 7.