CN117521778B - A cost-optimization approach for split federated learning - Google Patents

Abstract

The invention discloses a cost optimization method for split federal learning, which comprises the following steps: s1: constructing a split federal learning system model, wherein the model comprises a server and a plurality of clients, and each client is provided with a local data set; s2: calculating the time delay and the energy consumption of data calculation and communication of all clients; s3: calculating time delay and energy consumption of communication by using client data, analyzing an objective function and related constraints by taking model splitting layers and bandwidths as decision variables, and establishing a cost optimization problem of a splitting federal learning system; s4: solving an optimization problem to obtain a model splitting and bandwidth allocation strategy; s5: and organizing all clients to train the given complete model by matching with the server based on the obtained model splitting and bandwidth allocation strategy, and obtaining the trained complete model. The invention calculates the optimal model splitting and bandwidth allocation strategy, so that the weighted sum of the total time and the total energy consumption of all clients in the splitting federal learning process is minimum, and the cost of time and energy consumption is reduced.

Description

A cost-optimization approach for split federated learning

Technical Field

The present invention relates to the field of artificial intelligence, and more specifically, to a cost optimization method and system for split federated learning.

Background Art

Federated Learning (FL) and Split Learning (SL) are distributed model training technologies in deep learning that do not share user data. In federated learning, the server first initializes a global model. In each round of global model training, the server sends the global model to all clients. Each client trains the model based on local data on the local side, and then uploads the updated model parameters to the server. The server aggregates and updates the global model. The above process continues to iterate until the accuracy of the global model converges or the global model training round reaches the preset value. In split learning between the server and a client, the complete model is first split into two parts at the specified position (i.e., the split layer), and the front part sub-model is assigned to each client, and the back part sub-model is distributed to the server. In each round of model training, the client performs forward propagation to the split layer based on local data, obtains the crushed data and uploads it to the server. The server completes the forward propagation of the remaining layers. The server also performs back propagation to the split layer, obtains the gradient information of the above crushed data and transmits it back to the client, and the client then completes the back propagation of the remaining layers. The above two processes of forward and backward propagation are iterated continuously until the client completes all local data calculations, and then the updated client sub-model parameters are sent to the next client participating in model training, and the above steps are repeated until the model converges. The advantage of federated learning is that it allows multiple clients to train models in parallel while protecting data privacy. Split learning allows the client to train only sub-models separately. In order to combine the advantages of both, the existing literature [Thapa, Chandra, et al. "Splitfed: When federated learning meets split learning." Proceedings of the AAAIConference on Artificial Intelligence. Vol. 36. No. 8. 2022.] proposed split federated learning (Splitfed Learning, SFL), in which all clients and servers use split learning to complete model training, and the server uses federated learning to update the unified client sub-model. However, considering the participation of energy-constrained IoT devices in split federated learning, although split federated learning distributes most of the computational load involved in model training to the server, each client still needs to continuously calculate, send and receive data during the training process, which brings huge time and communication overhead, which puts a lot of pressure on IoT devices. Therefore, in order to ensure the practicality of split federated learning, it is crucial to optimize the cost management of all clients participating in split federated learning, including time and energy consumption.

Summary of the invention

In order to overcome the defects of the above-mentioned prior art in that the model training time and energy consumption are relatively high, the present invention provides a cost optimization method for split federated learning.

The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

A first aspect of the present invention provides a cost optimization method for splitting federated learning, comprising the following steps:

S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

S2: Calculates the latency and energy consumption of all client data computation and communication;

S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

Furthermore, step S2 calculates the latency and energy consumption of all client data computation and communication, firstly calculating the client local computation energy consumption and computation latency, and then calculating the split federated learning system client energy cost and computation communication latency.

Furthermore, the specific process of calculating the energy consumption and the calculation delay of the local calculation client is as follows:

The local computing load of client i for unit data client sub-model training is:

Among them, γ(·) and ξ(·) represent the number of operations required for the forward and reverse calculations of the client sub-model and server sub-model samples, respectively. The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration.

Client local computing energy consumption Calculation delay They are

Among them, the computing frequency of client i is _fi , the unit is cycle/s, _ki is the computing intensity of client i, the unit is FLOPs/cycle, then the computing speed of the client is defined as _φi = _fi × _ki , the unit is FLOPS; the client computing energy consumption coefficient is _εi , the unit is J/FLOPs; _Di is the local data set owned by client i; the computing speed of the server collaborating with client i is ψ.

Furthermore, the calculation splits the energy cost of the federated learning system client and the calculation communication delay. The specific process is as follows:

When transmitting the shredded data and updated client sub-model parameters, the uplink data transmission rate _ri ^up of client i is:

Where, _Piup is ^the transmission power when client i uploads data, _bi is the bandwidth allocated by the server to client i, ^σ2 is the Gaussian channel noise, is the channel gain between the server and the client, where α ₀ represents the channel gain at a distance of d = 1 m, and d _i represents the Euclidean distance between the server and the client;

The downlink data transmission rate r _i ^down of the server transmitting the crushing data gradient information to the client i is:

Where ^BD is the fixed bandwidth used by the server to broadcast to each user, _PB is the average transmission power, and the client upload delay is Download delay They are

Among them, the size of a single sample crushed data is D(C), the size of the gradient is G(C), the size of the data label is β, and the size of the client sub-model parameter is

The communication delay _Ti ^comm is:

Client communication energy consumption for

Where, _Pi ^down is the client receive power, and _Pi ^{up is} the average transmit power of client i;

Client i time cost _Ti ^total and energy cost They are:

T _i ^total =T _i ^comm +T _i ^comp

In a specified M global model training rounds, the split federated learning system client energy cost E and computational communication delay T are:

Furthermore, in step S3, the system cost optimization problem established is as follows:

subject to：

C1 _{: Cmin≤C≤Cmax}

C2: _bmin ≤b _{i ≤bmax}

C3:

Wherein, B _total is the available uplink bandwidth for all clients; C1 indicates the range of model splitting layers, constraining the number of client sub-model layers; C2 indicates the bandwidth value range that can be allocated to each client, which is pre-set; C3 ensures that the sum of bandwidth resources allocated to the client does not exceed B _total ; represents the weight between total energy consumption and total time delay,

Further, in step S4, solving the optimization problem specifically includes: discretizing the bandwidth variable and setting the discrete interval to Δ, then Where _bi is the bandwidth allocated by the server to client i, _bmin and _bmax represent the minimum and maximum bandwidth values that can be allocated to the client, respectively.

Furthermore, the optimal solution to the integer programming problem is solved using the Scipy library tool in Python or an existing planning problem solver.

Furthermore, in step S5, all clients are organized to cooperate with the server to train a given complete model based on the obtained model splitting and bandwidth allocation strategy. The specific process is as follows:

The given complete model is split into a global client sub-model and a global server sub-model. In each round of global model training, the global client sub-model and the global server sub-model are trained, which includes two aspects: first, each client receives the global client sub-model sent by the server, trains this sub-model by jointly training the server, and then uploads the updated client sub-model parameters to the server to aggregate and update the global client sub-model; in addition, the server internally assigns an associated server sub-model to each client, and updates this associated server sub-model when coordinating each client to train its client sub-model. Finally, the global server sub-model is updated by aggregating all associated server sub-models.

Furthermore, the specific process of step S5 is as follows:

S51: The server selects a model splitting strategy to determine the number of layers C included in the client sub-model, splits the given complete model, and sends the client sub-model W ^client to each client;

S52: Each client receives W ^client , and updates this sub-model based on local data in cooperation with the server, and the server is responsible for updating the server sub-model;

S53: Any client i has a local dataset D _i , and the training batch size is H. The local dataset is divided into batches;

S54: Any client i uses the current batch data to perform forward propagation of the client sub-model, and sends the crushed data and the data label corresponding to the current batch to the server, where the crushed data is the output of the splitting layer;

S55: The server uses the received crushed data to perform forward propagation of the server sub-model associated with client i, and calculates the loss function according to the received data label, performs back propagation calculation on the associated server sub-model to obtain the gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are executed in parallel inside the server;

S56: The server sends the gradient information of all crushed data to the corresponding clients respectively;

S57: Each client receives the gradient information of the crushed data, performs back propagation in the client sub-model and updates its client sub-model;

S58: Repeat S54 to S57 until all local data is used up;

S59: Each client uploads the updated local client sub-model parameters to the server. The server uses weighted averaging to aggregate and update the global client sub-model, and then re-sends the global client sub-model to all clients for a new round of local client sub-model training. The server aggregates all associated server sub-models through weighted averaging to update the global server sub-model.

A second aspect of the present invention provides a cost optimization system for split federated learning, the system comprising: a memory, a processor, the memory comprising a cost optimization method program for split federated learning, the cost optimization method program for split federated learning being executed by the processor to implement the following steps:

S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

S2: Calculates the latency and energy consumption of all client data computation and communication;

S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

The present invention first constructs a split federated learning system model, which includes a server and several clients, each of which has its own local data set; secondly, the delay and energy consumption of data calculation and communication of all clients are calculated; then, the delay and energy consumption of data calculation and communication of the clients are used, the model splitting layer and bandwidth are used as decision variables, the objective function and related constraints are analyzed, and the cost optimization problem of the split federated learning system is established; then, the optimization problem is solved to obtain the model splitting and bandwidth allocation strategy, and the optimal model splitting and bandwidth allocation strategy is obtained; finally, in the split federated learning system, based on the obtained model splitting and bandwidth allocation strategy, all clients are organized to cooperate with the server to train a given complete model, and the trained complete model is obtained, so as to protect the data privacy of the client and enable the client to complete the model training task in a training cost optimization manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a cost optimization method for split federated learning provided by an embodiment of the present invention.

FIG2 is a diagram of a split federated learning system model provided in an embodiment of the present invention.

FIG3 is a workflow diagram of a split federated learning system provided in an embodiment of the present invention.

FIG4 is a flow chart of model splitting and bandwidth allocation strategy calculation provided by an embodiment of the present invention.

FIG5 is a schematic diagram of splitting a ResNet18 model provided in an embodiment of the present invention.

FIG. 6 is a diagram showing simulation results according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

Example 1

As shown in FIG1 , the first aspect of the present invention provides a cost optimization method for splitting federated learning, comprising the following steps:

S1: Build a split federated learning system model, which includes a server and I clients. Each client has its own local data set. Figure 2 shows the split federated learning system model diagram.

More specifically, a split federated learning system model is constructed, as shown in Figure 2, which includes a server and I clients. The server performs server sub-model training, aggregates and updates the client sub-model shared by all clients, and allocates different communication bandwidths to different clients. Each client has its own local data set to train its own client sub-model and communicates data with the server through a wireless channel.

S2: Calculate the latency and energy consumption of all client data calculations and communications.

More specifically, the specific process of calculating the client's local computing energy consumption and computing latency is as follows:

The local computing load of client i for unit data client sub-model training is γ(·) and ξ(·) respectively calculate the number of operations required for the forward and reverse calculations of a single sample of the network model. The unit is FLOPs, where C represents the number of layers in the client submodel; similarly, it corresponds to the computing load that requires server collaboration. The computing frequency of client i is _fi , in cycle/s, and _ki is the computing intensity of client i, in FLOPs/cycle. Then the computing speed of the client is defined as _φi = _fi × _ki , in FLOPS. The computing energy consumption coefficient of the client is _εi , in J/FLOPs. _Di is the local data set owned by client i. The computing speed of the server in collaboration with client i is ψ, and the local computing energy consumption of the client is The calculated delays _Ti ^comp are

The calculation splitting federated learning system client energy cost and calculation communication delay are as follows:

P _i ^up is the transmission power when client i uploads data, b _i is the bandwidth allocated by the server to client i, σ ² is the Gaussian channel noise, is the channel gain between the server and the client, where α ₀ represents the channel gain at a distance of d = 1 m, and d _i represents the Euclidean distance between the server and the client; when transmitting the pulverized data and the updated client sub-model parameters, the uplink data transmission rate r _i ^up of client i is:

Similarly, the downlink data transmission rate r _i ^down at which the server transmits the crushed data gradient information to the client i is:

Where ^BD is the fixed bandwidth used by the server to broadcast to each user, and _PB is the average transmission power;

The size of a single sample crushing data is D(C), the size of the gradient is G(C), the size of the data label is β, and the size of the client sub-model parameter is The client upload delay Download delay They are

The communication delay _Ti ^comm is:

_Pi ^down is the client receiving power, _Pi ^{up is} the average transmitting power of client i, and the client communication energy consumption is for

Client i time overhead and energy consumption They are:

T _i ^total =T _i ^comm +T _i ^comp

In a specified M global model training rounds, the split federated learning system client energy cost E and computational communication delay T are:

S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system.

More specifically, in step S3, the system cost optimization problem established is as follows:

subject to：

C1 _{: Cmin≤C≤Cmax}

C2: _bmin ≤b _{i ≤bmax}

C3:

Wherein, B _total is the available uplink bandwidth for all clients; C1 indicates the range of model splitting layers, constraining the number of client sub-model layers; C2 indicates the bandwidth value range that can be allocated to each client, which is pre-set; C3 ensures that the sum of bandwidth resources allocated to the client does not exceed B _total ; represents the weight between total energy consumption and total time delay,

S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy.

More specifically, in order to simplify the problem solving in practical applications, consider discretizing the bandwidth variable and setting the discrete interval to Δ, then At this point, the original problem will be transformed into a traditional integer programming problem, and then the optimal solution to the integer programming problem will be solved using the Scipy library tool in Python or an existing planning problem solver (such as Lingo).

S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

More specifically, the given complete model is split into a global client sub-model and a global server sub-model. In each round of global model training, the global client sub-model and the global server sub-model are trained, which includes two aspects: first, each client receives the global client sub-model sent by the server, trains this sub-model by jointly training the server, and then uploads the updated client sub-model parameters to the server to aggregate and update the global client sub-model; in addition, the server internally assigns an associated server sub-model to each client, and updates this associated server sub-model when coordinating the training of each client's client sub-model. Finally, the global server sub-model is updated by aggregating all associated server sub-models.

More specifically, as shown in FIG3 , a workflow diagram of the split federated learning system is shown, the specific process of step S5 is as follows:

S51: The server selects a model splitting strategy to determine the number of layers C included in the client sub-model, splits the given complete model, and sends the client sub-model W ^client to each client;

S52: Each client receives W ^client , and updates this sub-model based on local data in cooperation with the server, and the server is responsible for updating the server sub-model;

S53: Any client i has a local dataset D _i , and the training batch size is H. The local dataset is divided into batches;

S54: Any client i uses the current batch data to perform forward propagation of the client sub-model, and sends the crushed data and the data label corresponding to the current batch to the server, where the crushed data is the output of the splitting layer;

S55: The server uses the received crushed data to perform forward propagation of the server sub-model associated with client i, and calculates the loss function according to the received data label, performs back propagation calculation on the associated server sub-model to obtain the gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are executed in parallel inside the server;

S56: The server sends the gradient information of all crushed data to the corresponding clients respectively;

S57: Each client receives the gradient information of the crushed data, performs back propagation in the client sub-model and updates its client sub-model;

S58: Repeat S54 to S57 until all local data is used up;

S59: Each client uploads the updated local client sub-model parameters to the server. The server uses weighted averaging to aggregate and update the global client sub-model, and then re-sends the global client sub-model to all clients for a new round of local client sub-model training. The server aggregates all associated server sub-models through weighted averaging to update the global server sub-model.

A second aspect of the present invention provides a cost optimization system for split federated learning, the system comprising: a memory, a processor, the memory comprising a cost optimization method program for split federated learning, the cost optimization method program for split federated learning being executed by the processor to implement the following steps:

S1: Build a split federated learning system model, which includes a server and several clients, each of which has its own local dataset;

S2: Calculates the latency and energy consumption of all client data computation and communication;

S3: Using the latency and energy consumption of client data computing and communication, taking the model splitting layer and bandwidth as decision variables, analyzing the objective function and related constraints, and establishing the cost optimization problem of the split federated learning system;

S4: Solve the optimization problem and obtain the model splitting and bandwidth allocation strategy;

S5: In the split federated learning system, all clients are organized to cooperate with the server to train the given complete model based on the obtained model splitting and bandwidth allocation strategy to obtain the trained complete model.

Example 2

The flowchart for implementing split federated learning cost optimization is shown in Figure 4. The following is a specific experimental simulation to demonstrate the effect of the method. Consider a split federated learning system with 5 clients, the server is located in the center, and the clients are evenly distributed 100 to 200 meters around the server. The training task is the classic CIFAR10 dataset classification task, and the global iteration number M=100. The CIFAR10 dataset consists of 60,000 32×32 color images in 10 categories, each category has 6,000 images, and 50,000 training images and 10,000 test images are selected. The model selects the convolutional neural network model ResNet-18, and the number of model layers is defined as 10. The candidate positions for model splitting are shown in Figure 5. Considering heterogeneity, the client local data is evenly distributed in [8000, 10000] sheets, the computing speed φ _i satisfies the uniform distribution of [0.01, 0.03] TFLOPS, the client energy consumption coefficient ε _i satisfies the uniform distribution of [0.5, 0.7] J/TFLOPs, and the transmission power _Pi ^up satisfies the uniform distribution of [0.1, 0.2] W. For simplicity, let _Pi ^up = _Pi ^down . The computing speed ψ of the server cooperating with each client is 2 TFLOPS, the total bandwidth of the system B _total is 8 MHz, the channel gain α ₀ is set to -50 dB, the server transmission power is 1 W, and the bandwidth used for broadcasting ^BD is 15 MHz. The weight coefficient in the objective function

The bandwidth discrete interval is Δ=0.1MHz, the minimum bandwidth b _min allocated to the user is 0.5MHz, and the maximum bandwidth b _max =3.5MHz, then b _i ∈{0.5+k×Δ,|k=0,1,2，…，30}. The simulation results are shown in FIG6 .

When C=10, the original federated learning scheme is executed. As can be seen from the table, this patent obtains the optimal solution at the split layer C=6, at which the bandwidth allocation is (1.3, 2.6, 1.8, 1.1, 1.2), and the objective function Objective function that is better than the original federated learning solution

Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims (8)

1. A cost optimization method for split federal learning, comprising the steps of:

S1: constructing a split federal learning system model, wherein the model comprises a server and a plurality of clients, and each client is provided with a local data set;

s2: calculating the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of all client data;

S3: using the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of the client data, taking the model splitting layer and the bandwidth as decision variables, and establishing an optimization target for minimizing the cost of the splitting federal learning system according to an objective function and constraint conditions;

S4: solving an optimization problem to obtain a model splitting and bandwidth allocation strategy;

s5: in the split federal learning system, organizing all clients to train a given complete model by matching with a server based on the obtained model split and bandwidth allocation strategy to obtain a trained complete model;

step S2, calculating the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of all client data, firstly calculating the local calculation energy consumption and calculation time delay of the client, and then calculating the communication energy consumption and the communication time delay of the split federal learning system client;

The specific process of calculating the local energy consumption and the calculating time delay of the client side is as follows:

The local computational load of the client i for training the unit data client terminal model is as follows:

Wherein γ ^client (C) represents the required operands for forward computation of the model for a single sample of the client at split point C; ζ ^client (C) represents the required operands for forward computation of the model for a single sample of the client at split point C; The unit is FLOPs, and C represents the number of layers contained in the client terminal model; similarly, the corresponding computing load that requires server collaboration

Client-side local computing energy consumptionThe calculation time delays T _i ^comp are respectively

The calculation frequency of the client i is f _i, the unit is cycle/s, k _i is the calculation intensity of the client i, the unit is FLOPs/cycle, the calculation speed of the client is phi _i＝f_i×k_i, and the unit is FLOPS; the client calculates the energy consumption coefficient as epsilon _i and the unit is J/FLOPs; d _i is the local dataset owned by client i; the calculation speed of the server cooperative client i is psi.

2. The cost optimization method for splitting federal learning according to claim 1, wherein the calculating the communication energy consumption and the communication time delay of the client of the splitting federal learning system comprises the following specific processes:

In transmitting the shredding data and the updated client terminal model parameters, the uplink data transmission rate r _i ^up of the client i is:

Where Pi ^up is the transmission power at which the client i uploads data, bi is the bandwidth allocated to the client i by the server, σ ² is gaussian channel noise, Is the channel gain between the server and the client, where α ₀ represents the channel gain at distance d=1m, di represents the euclidean distance between the server and the client;

The downlink data transmission rate r _i ^down of the server transmitting the crushed data gradient information to the client i is:

wherein B ^D is a fixed bandwidth used by the server for broadcasting to each user, and P _B is an average transmission power;

Client upload time delay Download time delayRespectively is

Wherein, the size of the crushed data of a single sample is D (C), the gradient size is G (C), the size of the data label is beta, and the parameter size of the client terminal model is

Communication delayThe method comprises the following steps:

client communication energy consumption Is that

Pi ^down is the client receiving power, and Pi ^up is the average transmitting power of the client i;

Client i time overhead T _i ^total and energy consumption overhead The method comprises the following steps of:

T_i ^total＝T_i ^comm+T_i ^comp

In the appointed M global model training rounds, the total time expenditure T and the total energy expenditure E of all clients of the split federal learning system are respectively as follows:

T＝Mmax(T₁ ^total,T₂ ^total,...,T_I ^total)

3. The method for optimizing the cost of split federation learning according to claim 2, wherein in step S3, the optimization objective for minimizing the cost of the split federation learning system is established according to the objective function and the constraint condition as follows:

The objective function is:

The constraint conditions are as follows:

subject to:

C1:C_min≤C≤C_max

C2:b_min≤b_i≤b_max

C3:

Wherein, B _total is the available uplink bandwidth of all clients; c1 is the range of model splitting layers, and the number of layers of the client terminal model is constrained; c2 is the constraint of the value range of the bandwidth value which can be allocated to each client; c3 is a constraint on the sum of bandwidth resources allocated to the client; representing a weight between total energy consumption overhead and total latency overhead; b _i is the bandwidth allocated to the client i by the server; c is the number of layers the client terminal model contains.

4. The method for cost optimization of split federal learning according to claim 1, wherein in step S4, solving the optimization problem specifically includes: discretizing the bandwidth variable, setting the discretization interval as delta, thenWherein b _i is the bandwidth allocated to the client i by the server, and b _min and b _max respectively represent the minimum value and the maximum value of the bandwidth values that the client can allocate to; k is the kth discrete interval.

5. The method of claim 4, wherein the optimal solution of the integer programming problem is solved using a Scipy library tool in Python or an existing programming problem solver.

6. The method for optimizing cost of split federal learning according to claim 1, wherein in step S5, all clients are organized to train a given complete model with a server based on the obtained model split and bandwidth allocation policy, and the specific process is as follows:

Splitting a given complete model into a global client terminal model and a global server sub-model, and training the global client terminal model and the global server sub-model in each round of training of the global model comprises the following two aspects: firstly, each client receives a global client terminal model issued by a server, trains the sub-model through a joint server, and then uploads updated client terminal model parameters to the server to aggregate and update the global client terminal model; in addition, the server internally assigns each client an associated server sub-model that is updated as each client trains its client sub-model in conjunction with it, and finally updates the global server sub-model by aggregating all of the associated server sub-models.

7. The method for cost optimization for split federal learning according to claim 6, wherein the specific process of step S5 is as follows:

s51: the server selects a model splitting strategy to determine the layer number C contained in the client terminal model, splits a given complete model and then issues a global client terminal model W ^client to each client terminal;

S52: each client receives W ^client, updates the sub-model based on local data under the cooperation of a server, and the server is responsible for updating the server sub-model;

S53: any client i owns a local dataset D _i with training batch size H, dividing the local dataset into Each batch;

s54: any client i uses the current batch data to perform forward propagation of the client terminal model, and sends the data labels of the crushed data and the current batch to a server, wherein the crushed data is output of a split layer;

s55: the server utilizes the received crushed data to conduct forward propagation of a server sub-model associated with the client i, calculates a loss function according to the received data label, executes backward propagation calculation on the associated server sub-model to obtain a gradient of the crushed data, and updates the associated server sub-model based on the crushed data gradient; the above steps for all clients are performed in parallel inside the server;

S56: the server respectively transmits gradient information of all the crushing data to the corresponding client;

S57: each client receives gradient information of the crushed data, performs back propagation on the client terminal model and updates the client terminal model thereof;

S58: repeating S54 to S57 until all the local data is exhausted;

S59: each client uploads the updated local client terminal model parameters to a server, the server utilizes weighted average to aggregate and update the global client terminal model, and then the global client terminal model is re-issued to all clients for a new round of client terminal model local training; the server aggregates all associated server sub-models by weighted average to update the global server sub-model.

8. A cost optimization system for split federal learning, the system comprising: the system comprises a memory and a processor, wherein the memory comprises a cost optimization method program for splitting federal learning, and the cost optimization method program for splitting federal learning realizes the following steps when being executed by the processor:

S1: constructing a split federal learning system model, wherein the model comprises a server and a plurality of clients, and each client is provided with a local data set;

s2: calculating the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of all client data;

S3: using the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of the client data, taking the model splitting layer and the bandwidth as decision variables, and establishing an optimization target for minimizing the cost of the splitting federal learning system according to an objective function and constraint conditions;

S4: solving an optimization problem to obtain a model splitting and bandwidth allocation strategy;

s5: in the split federal learning system, organizing all clients to train a given complete model by matching with a server based on the obtained model split and bandwidth allocation strategy to obtain a trained complete model;

step S2, calculating the calculation time delay, the calculation energy consumption, the communication time delay and the communication energy consumption of all client data, firstly calculating the local calculation energy consumption and calculation time delay of the client, and then calculating the communication energy consumption and the communication time delay of the split federal learning system client;

The specific process of calculating the local energy consumption and the calculating time delay of the client side is as follows:

The local computational load of the client i for training the unit data client terminal model is as follows:

Wherein γ ^client (C) represents the required operands for forward computation of the model for a single sample of the client at split point C; ζ ^client (C) represents the required operands for forward computation of the model for a single sample of the client at split point C; The unit is FLOPs, and C represents the number of layers contained in the client terminal model; similarly, the corresponding computing load that requires server collaboration

Client-side local computing energy consumptionThe calculation time delays T _i ^comp are respectively

The calculation frequency of the client i is f _i, the unit is cycle/s, k _i is the calculation intensity of the client i, the unit is FLOPs/cycle, the calculation speed of the client is phi _i＝f_i×k_i, and the unit is FLOPS; the client calculates the energy consumption coefficient as epsilon _i and the unit is J/FLOPs; d _i is the local dataset owned by client i; the calculation speed of the server cooperative client i is psi.