CN106775949A - A kind of Application of composite feature that perceives migrates optimization method online with the virtual machine of the network bandwidth - Google Patents

Abstract

The invention discloses a virtual machine online migration optimization method for sensing compound application characteristics and network bandwidth, and belongs to the field of software technology. The method of the present invention is as follows: 1) perceiving the virtual machine application feature environment and the network bandwidth environment, and collecting the number of dirty pages in the memory; 2) predicting the number of dirty pages in the memory using a gray prediction model; 3) calculating the dirty page rate of the iterative cycle of the virtual machine; 4) Collect network bandwidth usage; 5) According to the network bandwidth required by the application in the virtual machine, determine whether the virtual machine is a network-intensive virtual machine, and then reserve the network bandwidth. When facing the virtual machine migration of network-intensive applications or memory-intensive applications, the present invention can reduce extra expenses in the migration process, improve transmission efficiency in the migration process, and effectively reduce migration time.

Description

A virtual machine online migration optimization based on perception of compound application characteristics and network bandwidth method

technical field

The invention designs an online migration optimization method of a virtual machine that perceives application characteristics and network environment, especially an application characteristic-aware network bandwidth reservation adjustment algorithm, which belongs to the field of software technology.

Background technique

With the wide application of virtualization technology, the dynamic management of virtual machines has become more and more important, and online migration is an important means of dynamic management of virtual machines. Live migration is a network-intensive activity that requires the transfer of several gigabytes or even tens of gigabytes of virtual machine memory state from the source host to the destination host. In addition to occupying network resources, online migration consumes additional physical resources such as memory and CPU.

Currently, traditional online migration methods include pre-copy (pre-copy), post-copy (post-copy) and hybridcopy (mixed copy) online migration methods. Pre-copy online migration is the mainstream virtual machine dynamic migration technology. Pre-copy is divided into three stages: first copy, iterative copy, and downtime copy. The specific migration process is shown in Figure 1. First, copy all the memory of the virtual machine to the destination host without interrupting the operation of the virtual machine. This stage is called the "first copy" stage; then, enter the "iterative copy stage", and copy the memory generated during the previous copy process. Dirty pages (memory pages that were modified last time) are iteratively copied to the destination host, and the process does not interrupt the running of the virtual machine. After each round of iterative copying, it is necessary to judge whether the current conditions for entering the stop-and-coy stage (that is, the stop-and-coy stage) are met. If it is satisfied, it will enter the third stage - "downtime copy" stage, otherwise continue to iterative copy. During the shutdown copy phase, the virtual machine on the source host is suspended, and then the remaining dirty memory pages of the virtual machine are synchronized to the destination host. At the same time, it also synchronizes the virtual machine system information, including CPU and network status, to the destination. After the information is transmitted, the virtual machine on the destination host restores the system according to the transmitted virtual machine system information. Different from pre-copy online migration, the memory synchronization of post-copy online migration is after the virtual machine resumes running on the destination host. Hybrid copy online migration is a special case of post-copy online migration. It combines the characteristics of the pre-copy online migration method and the post-copy online migration method. Before the virtual machine system information is transmitted to the destination host, the most frequently accessed subset of memory pages is transferred to the destination host. The remaining infrequently used memory pages will be obtained from the source host when the virtual machine needs to run.

The mainstream virtualization platforms (KVM, Xen, VMware, etc.) all support the pre-copy algorithm for online migration. Pre-copy online transfer methods are widely used and developed in commercial and academic fields. This method can effectively shorten the migration time and improve migration performance, but in practical applications, due to the influence of iterative convergence, different application characteristics of virtual machines and resource constraints, using the pre-copy online migration method to migrate network-intensive and memory-intensive virtual machines Machine-time migration performance is not ideal. For example, if a memory-intensive application is running in a virtual machine, the memory of the virtual machine is constantly being modified rapidly, and when the modification speed is greater than the transmission speed of dirty pages in the memory, this will prolong the iterative copying time of pre-copy and occupy physical resources The time also increases and it is difficult to enter the downtime copy phase, which can seriously affect other services and even cause the migration to fail. None of the methods for predicting virtual machine memory dirty pages takes into account the impact of virtual machine application characteristics and environments on virtual machine memory dirty pages. At present, pre-copy online migration improvement methods often reduce the data transmission of memory dirty pages by deleting duplicate memory page data, compressing memory pages, and suppressing the generation of similar memory pages, so as to reduce migration time and downtime. However, the above method often increases a large amount of extra CPU consumption, and does not take into account the impact of the network bandwidth environment on the dirty pages of the transmitted memory.

Contents of the invention

The purpose of the present invention is to overcome the influence of application features on the number of memory dirty pages in existing related achievements, ignore the combined effect of memory dirty pages and network bandwidth, and iterate in network-intensive or memory-intensive virtual machine migration scenarios Migration performance problems such as long cycle time and increased downtime, provide an online virtual machine migration optimization method that perceives application characteristics and network environment, which can effectively reduce competition for network resources, improve network transmission efficiency, reduce migration time, and achieve provision of virtual machines. Purpose of migration performance.

In the present invention, in order to obtain the specific value of the dirty pages in the memory, a gray prediction model is used to predict the dirty pages in the memory. The reason for choosing this method is that it is difficult to collect dirty page information during the memory copy process, and the number of collected pages is relatively small, while the gray prediction model can make better predictions with fewer data samples.

As shown in Figure 2, the virtual machine online migration optimization method for perceiving application characteristics and network environment of the present invention is as follows:

(1) Perceiving the application characteristics and network environment of the virtual machine, and collecting the number of dirty pages in the memory; the perceiving virtual machine application characteristics refers to dynamically obtaining the usage of application resources in the virtual machine, including memory usage, CPU usage and network bandwidth, And be able to grasp the change trend of the resource usage rate of the application; sensing the network environment refers to dynamically obtaining the network bandwidth usage rate of the cloud data center; memory dirty pages refer to the memory pages that are modified during the virtual machine migration process;

(2) According to the application characteristics and the network environment of the perceived virtual machine in the step (1) and obtaining the number of dirty pages in the memory as sample data, use the gray prediction model to predict the number of dirty pages in the memory;

(3) According to the number of memory dirty pages predicted in step (2), calculate the dirty page rate of the iterative cycle of the virtual machine, dirty page rate=iterative cycle produces dirty page number/iterative cycle time;

(4) obtain network bandwidth usage;

(5) According to the network bandwidth usage obtained in step (4), it is judged whether the virtual machine is a network-intensive virtual machine, and then the network bandwidth is reserved. In the process of network bandwidth reservation, bandwidth reservation is performed according to the network bandwidth and dirty page rate required by the virtual machine application, so as to ensure that the network bandwidth is sufficient in each iteration cycle during the migration process and reduce network congestion. For network-intensive applications, the method reserves network bandwidth in combination with the network environment, which can reduce network bandwidth competition during the migration process, improve migration efficiency, reduce iteration time, and reduce migration time, thereby achieving the purpose of improving migration performance. At the same time, priority is given to ensuring the migration of non-network-intensive virtual machines, which rationally allocates network resources and improves network utilization.

The process of the step (2) using the gray prediction model to predict the number of dirty pages in the next period is as follows:

(21) To collect the data in step (1), that is, X ₁ : the number of memory dirty pages generated in different iteration cycles of migration; X ₂ : the memory usage rate of virtual machines in different iteration cycles of migration; X ₃ : the CPU usage of virtual machines in different iteration cycles X ₄ : Network bandwidth of virtual machines in different iteration cycles of migration; X ₅ : Time of last iteration; X ₆ : Network usage of cloud data centers in different iteration cycles of migration. These data are transformed into matrix, as gray prediction sequence X ⁽⁰⁾ , and described sequence X ⁽⁰⁾ is accumulated, generate AGO sequence order to be X ⁽¹⁾ ;

(22) The sequence that obtains according to step (21) X ⁽¹⁾ solves the sequence that neighbor value generates is mean value sequence is;

(23) In the gray prediction model, assume that there is a certain relationship between X ⁽¹⁾ of step (21) and step (22), so the gray differential equation is established;

(24) according to the establishment of step (22) whitening differential equation;

(25) according to step (23) equation group, set up based on GM (1, N) equation group, utilize least square method to solve GM (1, N) equation parameter sequence;

(26) Substituting the parameters into the whitening differential equation in (24), solving to obtain the GM (1, N) discrete response function, and solving the predicted value sequence through the cumulative reduction formula;

(27) In order to reduce the error of the gray prediction model, the prediction accuracy is improved by correcting the prediction value by the residual.

The network bandwidth reservation process in the described step (5) is as follows:

(31) Collect the network bandwidth information required by the application in the virtual machine, and judge whether the virtual machine belongs to a network-intensive application virtual machine. If the migration virtual machine is not a network-intensive application virtual machine, skip to step (32); if it is a network-intensive application virtual machine virtual machine, skip to step (34);

(32) According to calculating the virtual machine memory dirty page rate in step (3), judge whether the current idle bandwidth meets the transmission bandwidth of the next memory dirty page quantity, that is, whether the network bandwidth is greater than the memory dirty page rate; if satisfied, do not adjust; Otherwise, adjust the network bandwidth, that is, skip to step (33);

(33) Prioritize the migration of non-network-intensive applications and delay the migration of network-intensive virtual machines at the same time;

(34) The application bandwidth is reserved according to the historical network transmission traffic data, and the current idle network bandwidth is subtracted from the application reserved bandwidth to obtain the bandwidth allocated for the current virtual machine migration.

The advantage of the present invention compared with prior art is:

(1) In the prior art, methods such as deleting duplicate memory page data, compressing memory pages, and suppressing the generation of similar memory pages will bring large additional resource overheads such as CPU resource overheads. At the same time, the efficiency of these methods also depends on the deletion algorithm, compression and The efficiency of the decompression algorithm does not consider the impact of the network bandwidth environment on the dirty pages of the transmitted memory. Compared with the traditional pre-copy strategy, the present invention can predict the number of dirty pages in the memory of the virtual machine under the perception of different application characteristics. Next, adjust the network bandwidth reservation.

(2) Compared with the traditional pre-copy online migration method, during the migration process of virtual machines containing network-intensive applications or memory-intensive applications, the network bandwidth reservation adjustment not only considers the network traffic during the virtual machine migration process but also The computer contains the network bandwidth required by network-intensive applications, and reserves the required network bandwidth for applications. The invention can effectively reduce the competition of network resources, improve network transmission efficiency and reduce migration time.

Description of drawings

Figure 1 is a schematic diagram of the existing pre-copy online migration process;

Fig. 2 is the process diagram of the virtual machine online migration optimization method of perception compound application characteristics and network environment of the present invention;

Figure 3 shows the number of dirty pages in memory predicted by GM(1, N);

FIG. 4 is a schematic diagram of a network reservation adjustment process.

detailed description

The present invention will be described in detail below in conjunction with specific embodiments and accompanying drawings.

The virtual machine online migration optimization strategy for perceiving application characteristics and network environment proposed by the present invention collects virtual machine application characteristic data and network environment characteristic data, and adjusts network bandwidth reservation according to these data, so as to achieve the purpose of optimizing virtual machine migration performance, especially It plays a significant role in the virtual machine migration scenarios involving network-intensive applications and memory-intensive applications.

After collecting certain samples of virtual machine application characteristics and network environment, these samples will be input into the memory dirty page number prediction model based on GM (1, N) (ie gray prediction model).

For the selection of predictors of virtual machine memory dirty pages, the present invention considers the following factors that can reflect the characteristic environment of virtual machine applications: X1: the number of memory dirty pages generated in different iteration cycles of migration; X2: the memory usage rate of virtual machines in different iteration cycles of migration; X3 : CPU usage of virtual machine in different iteration cycles of migration; X4: network bandwidth of virtual machine in different iteration cycles of migration; X5: time of last iteration; X6: network usage of cloud data center in different iteration cycles of migration. It is assumed that a total of S iterations are performed during the migration process, and the number of memory dirty pages generated each time is related to the application characteristics of the last iteration cycle and the last iteration time.

The establishment process of the memory dirty page prediction model based on G(1, N) is shown in Figure 3, and the specific steps are as follows:

Step 1: Given the gray prediction sequence X ⁽⁰⁾ , let X ⁽¹⁾ be the AGO sequence of X ⁽⁰⁾ :

Among them, the elements of X1 represent the number of memory dirty pages generated in each cycle of S iteration cycles respectively; the elements of X2 represent the memory usage rate of each iteration virtual machine in S iteration cycles respectively; Represents the CPU usage rate of each iterative virtual machine in S iteration cycles; X4 elements represent the network bandwidth of each iteration virtual machine in S iteration cycles; X5 elements represent the last iteration cycle; X6 elements represent Represents the network usage of the cloud data center of each iteration in the S iteration cycles;

Step 2: Solving

for The sequence generated by the neighbor values of , let for mean sequence of , so but

for the matrix Elements, for with The mean value of , where k is 2～S;

Step 3: Establish the gray differential equation:

Among them, k=1,2,...,S; a is the development coefficient; b _i (i=2,3,...,6) is the driving coefficient.

Step 4: Establish whitening differential equation:

Among them, k=1,2,...,S; a is the development coefficient; bi(i=2,3,...,6) is the driving coefficient.

Step 5: Establish a system based on GM(1,N) equations:

Let the parameters be listed as Then the GM(1,N) gray differential equation is: Among them, k=1,2,...,S; a is the development coefficient; bi(i=2,3,...,6) is the driving coefficient.

Solve using the method of least squares

Step 6: Solve the GM(1,N) discrete response function:

Substituting the solved parameters into the differential equation, the GM(1,N) discrete response function is:

Wherein k=1,2,3,...,6; e is a mathematical constant. also because The cumulative reduction formula is:

Step 7: Residual Correction

The gray prediction model sometimes predicts that the data with large fluctuations will affect the development of the data series, so it is necessary to modify the model of GM(1,N). make is the original residual sequence, If there exists k ₀ that satisfies the following conditions:

(1) have the same sign;

(2) When Sk ₀ ≥ 4, it can be called is the tail segment of the modelable residual, denoted as

Establish a GM(1,1) model for ￡ ⁽⁰⁾ , and find its parameter list Calculate The simulated value of:

Then the modified formula of GM(1,N) discrete response function is:

After predicting the number of memory dirty pages, calculate the memory dirty page rate. According to the memory dirty page rate generated by different application characteristics, combined with the network environment to adjust the network bandwidth, an application characteristic-aware network bandwidth reservation adjustment algorithm is proposed.

The network reservation adjustment algorithm is shown in Figure 4. It can be seen from Figure 4 that judging whether the virtual machine is a network-intensive application based on the network bandwidth required by the virtual machine application, that is, if the average load of the network bandwidth of the virtual machine is greater than the set network bandwidth-intensive threshold, For network-intensive application virtual machines; anyway, not network bandwidth-intensive application virtual machines. Specific steps are as follows:

Step 1: Determine whether the virtual machine is a network-intensive application virtual machine by collecting the network bandwidth information required by the application in the virtual machine. If the migrated virtual machine is not a network-intensive application virtual machine, skip to step 2; if it is a network-intensive virtual machine, skip to step 4;

Step 2: According to the predicted number of memory dirty pages, determine whether the current free bandwidth meets the transmission bandwidth of the next number of memory dirty pages (that is, whether the network bandwidth is greater than the memory dirty page rate). If it is satisfied, do not adjust; otherwise, adjust the network bandwidth, that is, skip to step 3;

Step 3: Prioritize the migration of non-network-intensive applications, and delay the migration of network-intensive virtual machines at the same time. Step 4: Reserve application bandwidth based on historical network transmission traffic data. The bandwidth allocated for virtual machine migration is obtained by subtracting the application reserved bandwidth from the current idle network bandwidth.

Traffic adjustment and control is mainly realized by the traffic control module, that is, the traffic control (Traffic control) module, which is mainly divided into input interfaces (Input Interfaces), traffic limitation (Ingress Policing), input multiplexing (InputDe-Multiplexing), forwarding (Forwarding) , queue scheduling (Queue Scheduling) and output interface (OutputInterface) 6 modules. The input interface is responsible for receiving data packets and sending the data packets to the flow limiting module. The flow limiting module is responsible for screening data packets, transmitting the qualified data packets to the input multiplexing module and discarding the non-compliant data packets. The input multiplexing module judges and analyzes the input data packets, and determines the flow direction of the data packets. If the data packet belongs to the host, the data packet is delivered to the upper layer (Upper Layer); otherwise, the data packet is transmitted to the forwarding module for forwarding processing: by checking the routing table flow, the next hop of the data packet is determined. Then, the data packets will be transmitted to the queue scheduling module, which will queue the data packets and transmit the data packets to the output interface according to the order of the queues. The output interface transmits the packet to the next hop in accordance with the Flow control is mainly processed and implemented during queue scheduling.

In a word, the present invention can reduce the extra overhead in the migration process, improve the transmission efficiency in the migration process, and effectively reduce the migration time when facing the virtual machine migration of network-intensive applications or memory-intensive applications.

Claims (3)

1. a kind of Application of composite feature that perceives moves optimization shifting method online with the virtual machine of the network bandwidth, it is characterised in that step For：

(1) the application feature and network environment of virtual machine are perceived, the dirty page number of internal memory is collected；The perception virtual machine application feature Refer to application resource service condition in dynamic access virtual machine, including memory usage, CPU usage and the network bandwidth, and Will appreciate that the trend of application resource utilization rate change；Sensing network environment refers to the dynamic access cloud data center network bandwidth Utilization rate；Internal memory containing dirty pages face refers to the memory pages changed during virtual machine (vm) migration；

(2) the application feature and network environment of the perception virtual machine in step (1) and the acquisition dirty page number conduct of internal memory Sample data, the dirty page number of internal memory is predicted using grey forecasting model；

(3) the dirty page number of internal memory obtained according to prediction in step (2), calculates the containing dirty pages face rate of the iteration cycle of virtual machine, dirty Page rate=iteration cycle produces dirty page number/iteration cycle time, is that step (5) carries out the reserved of the network bandwidth；

(4) network bandwidth service condition is obtained；

(5) network bandwidth service condition obtained according to step (4), judges whether virtual machine is network-intensive virtual machine, so After carry out the network bandwidth reserve；In network bandwidth reservation procedure, the network bandwidth and containing dirty pages face according to needed for virtual machine application Rate carries out RSVP, to ensure that each iteration cycle network bandwidth is sufficient in transition process, reduces network congestion.

2. method according to claim 1, it is characterised in that：The step (2) is predicted next using grey forecasting model The process of the dirty page number of period internal memory is as follows：

(21) to collection step (1) data, i.e. X₁：Migrate the dirty page quantity of internal memory that different iteration cycles are produced；X₂：Migration is not With iteration cycle virutal machine memory utilization rate；X₃：Migrate different iteration cycle virtual machine CPU usages；X₄：Migrate different iteration Cycle virtual machine network bandwidth；X₅：The time of last iteration；X₆：Migrate the Web vector graphic of different iteration cycle cloud data centers These data are converted into matrix by situation, used as gray prediction sequence X⁽⁰⁾, and to the sequence X⁽⁰⁾Added up, generated AGO Sequence order is X⁽¹⁾；

(22) X obtained according to step (21)⁽¹⁾The sequence for solving the generation of neighbour's value is that equal value sequence is；

(23) in grey forecasting model, it is assumed that the X of step (21)⁽¹⁾With the certain relation of presence of step (22), grey differential is set up Equation；

(24) albinism differential equation is set up according to step (22)；

(25) according to step (23) equation group, set up and be based on GM (1, N) equation group, GM (1, N) sides are solved using least square method Journey argument sequence；

(26) parameter is substituted into the albinism differential equation in (24), solution obtains GM (1, N) discrete receptance function, by regressive also Former equations go out to predict value sequence；

(27) be reduce grey forecasting model error, by forecasting with residual error modification value improve precision of prediction.

3. method according to claim 1, it is characterised in that：Network bandwidth reservation procedure in the step (5) is as follows：

(31) judge whether virtual machine belongs to network-intensive application virtual using required network bandwidth information in collection virtual machine Machine, if migration virtual machine is not network-intensive application virtual machine, jumps to step (32)；It is virtual if network-intensive Machine, jumps to step (34)；

(32) according to virutal machine memory containing dirty pages face rate is calculated in step (3), judge whether current idle bandwidth is met in next Whether the transmission bandwidth of dirty page quantity, the i.e. network bandwidth are deposited more than internal memory containing dirty pages face rate；It is not adjusted if meeting；It is no Then, the network bandwidth is adjusted, that is, jumps to step (33)；

(33) it is preferential to ensure the migration of non-network intensive applications, postpone same time network intensity virtual machine (vm) migration；

(34) carry out application bandwidth according to web-based history transmission data on flows to reserve, present idle network bandwidth is subtracted using pre- The bandwidth for staying bandwidth just to be distributed by present virtual machine (vm) migration.