CN117034738A - Event-driven-based deep reinforcement learning building control method - Google Patents

Abstract

The invention discloses an event-driven deep reinforcement learning building control method, which includes: Step S1, obtain weather data, generate a data set, and divide the data set into a training set and a test set; Step S2, select a multi-region The residential building is used as an architectural model, and a Python-based experimental platform is created according to the thermodynamic formula; Step S3: Design an event-driven MDP model, and design two types of events based on expert knowledge and prior knowledge; Step S4 , design an event-driven DQN algorithm to optimize the control strategy when an event is triggered. The present invention provides an event-driven deep reinforcement learning building control method, which can realize model-free optimal control under the needs of multi-region residential buildings, and can maintain better performance while accelerating learning speed and reducing the frequency of HVAC replacement actions. Performance control minimizes building energy consumption while meeting people's needs for indoor thermal comfort.

Description

An event-driven deep reinforcement learning building control method

Technical field

The invention relates to an event-driven deep reinforcement learning building control method, belonging to the field of building energy conservation.

Background technique

Currently, with the increasing global climate change, it is particularly urgent to reduce building energy consumption and improve thermal comfort. According to the International Energy Agency, residential buildings account for a significant share of global energy consumption, consuming 35% of building energy in 2020 alone. In building systems, the energy consumption of HVAC systems accounts for more than 50%. Therefore, reducing the energy consumption of HVAC systems has become one of the important research directions for optimizing building control. However, while pursuing building energy conservation, we cannot sacrifice thermal comfort. Especially during the epidemic, people stay indoors more. Therefore, how to minimize energy consumption while maintaining thermal comfort in residential buildings has become the focus of researchers and related practitioners.

In the existing technology, most HVAC systems adopt methods such as rule-based control RBC, proportional integral derivative PID, Lagrarangian relaxation method, and model predictive control MPC. However, RBC is limited by rule settings and has limited control accuracy, making it difficult to adapt to complex actual environments; PID controllers rely on fixed parameters and may not provide the best performance when the environment changes; although MPC has better performance, it It is not easy to create a simplified and sufficiently accurate building model in practice because the indoor environment is affected by many factors, such as building structure, building layout, building internal heat and outdoor environment, etc. When the model is not enough to accurately describe the building thermal dynamics and When large deviations exist, control performance may deviate from expectations.

Reinforcement learning is an effective data-driven control method. Compared with traditional control methods, reinforcement learning does not require precise thermodynamic models and can better adapt to changes and uncertainties in the environment. Although reinforcement learning has shown great potential in HVAC systems, it still faces several challenges. First, traditional reinforcement learning methods learn at fixed time steps, while there are similarities between consecutive time steps in HVAC control, which may lead to data redundancy and inefficient utilization. Secondly, the choice of time interval also affects the control performance. A longer time interval will affect the control accuracy, while a shorter time interval will cause too many action switches. In addition, slow changes in building thermal dynamics can also slow down reinforcement learning learning. And HVAC control problems often involve high-dimensional state spaces, which further increases the complexity of reinforcement learning methods. Deep reinforcement learning methods have the potential to solve more complex HVAC control problems by combining the advantages of deep learning and reinforcement learning. However, deep reinforcement learning methods still face the above challenges. Therefore, it is necessary to explore new methods to improve the efficiency and performance of HVAC control, which will help promote technological innovation in the field of building energy conservation and contribute to the realization of sustainable building development and global energy conservation.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide an event-driven deep reinforcement learning building control method that can achieve model-free optimal control under the needs of multi-regional residential buildings, and can accelerate the learning speed, Reduce the frequency of HVAC replacement actions while maintaining better performance control. By optimizing the control of HVAC systems, we can minimize building energy consumption while meeting people's needs for indoor thermal comfort.

In order to solve the above technical problems, the technical solution of the present invention is:

An event-driven deep reinforcement learning building control method, which includes:

Step S1: Obtain weather data, generate a data set, and divide the data set into a training set and a test set;

Step S2: Select a residential building with multiple areas as the building model, and create an experimental platform based on Python based on the thermodynamic formula;

Step S3: Design an event-driven MDP model, and design two types of events based on expert knowledge and prior knowledge;

Step S4: Design an event-driven DQN algorithm to optimize the control strategy when an event is triggered;

Step S5: Verify the effectiveness of the event settings through ablation experiments, and adjust event parameters according to the results;

Step S6: Prove the control effect and superiority of the event-driven DQN algorithm through comparative experiments.

Further, in step S3, an event-driven MDP model is designed, which specifically includes the following steps:

Construct an event-driven six-tuple event-driven MDP model consisting of state, action, transition, reward function, discount factor and event trigger function, expressed as:

Among them, S represents the state set; A represents the action set that the agent can take; P represents the transfer of the system; R represents the reward function; γ represents the discount factor, which is used to measure the importance of the agent to future rewards; e represents the event trigger function ;

When the value of the event triggering function exceeds the preset threshold, the agent is triggered and updates the strategy, and a state transition occurs at the same time. The transition function is:

P{s(t+1)|s(t),a,e};

Using multi-region residential buildings as the environment and event-driven deep reinforcement learning as the intelligent agent, the status, actions and rewards of the HVAC system are designed.

Further, the status is expressed as:

[T _in,z (t),K _z (t),T _out (t),λ ^electricity (t)]

Among them, T _{in, z} (t) is the indoor temperature of each room, K _z (t) is the occupancy rate, T _out (t) is the outdoor temperature, λ ^electricity is the electricity price, z represents the room number, and t represents the current time step. .

Further, the action is the temperature set point of the HVAC system, and the action is expressed as:

A(t)＝[Sp _z (t)]

where, SP _z (t) represents the temperature set point of room z; where the set point of each zone is a discrete variable and follows the following logic:

If the indoor temperature is higher than the temperature value of the temperature set point, the HVAC system is turned on; if the indoor temperature is lower than the temperature value of the temperature set point, the HVAC system is turned off.

Further, the reward function is expressed as:

Among them, λ ^retail (t') represents the retail price, E _HVAC (t') represents energy consumption, R ^comfort (t') represents the reward for keeping the temperature within the comfort range, and SW ^penalty (t') represents the penalty for switching on and off HVAC. item;

Given comfort range When the indoor temperature T _in (t) is higher than the minimum comfort temperature th ^min , the agent adjusts the settings of the HVAC system according to the reward size; when T _in (t) is lower than the minimum comfort temperature th ^mn , the HVAC is turned off;

When the execution action deviates from the threshold, a negative reward is added, which is specifically expressed as follows:

When the indoor temperature is within the thermal comfort threshold, a positive reward is given, and an optimal comfort temperature th ^best is defined. The closer the indoor temperature is to th ^best , the higher the positive reward. The positive reward is expressed as:

Further, the two types of events in step S3 include state transition events and combination events.

Further, the state transition event includes a first event and a second event, and the first event is defined as follows:

Among them, λ ^retail (t) and λ ^retail (t') are both within the price range [λ ^low , λ ^high ], λ ^low represents the lowest retail price, and λ ^high represents the highest retail price.

The second event is defined as follows:

Among them, K _z (t') and K _z (t) are within [-1, 1], -1 means there is no one at the moment, and 1 means there is someone at the moment.

Further, the combined event is a third event, and the third event is composed of TH (t'), K _z (t') and λ ^retail (t');

When room occupancy or price changes, if When it is greater than the thermal comfort value χ, the third event is triggered, and the third event is defined as follows:

in, Indicates thermal comfort value.

Further, in step S4, an event-driven DQN algorithm is designed, which specifically includes the following steps:

Step S41: Initialize the Q network parameter ω, assign the target Q network value ω ^* =ω, and initialize all states and actions and corresponding q values based on the random Q network parameter ω;

Step S42: Initialize the minibatch size B and initialize the experience buffer M;

Step S43: Iterate.

Further, the iterative process of step S43 is as follows:

Step S431: Obtain the current state quantity of the environment initialization, and perform preliminary processing to obtain the characteristic state parameters s (T _out (0), T _{in, z} (0), λ ^electricity (0), K _z (0));

Step S432. Use s as input in the Q network. If none of the events are triggered, the previous action is executed; if one of the events is triggered, all action outputs corresponding to the q value of the Q network are obtained, and the current q value is calculated through the ε greedy algorithm. Select the corresponding action a from , and action a is expressed as:

Among them, s represents a certain state, and A is the action set;

Step S433: The current state s(t) executes action a, and obtains the processed characteristic state vector s(t+1) of the new environmental state and the reward R of the action;

Step S434: Put the tuple [s(t), Setpt _z (t), r(t), s(t+1)] into the experience buffer M;

Step S435: Randomly select B samples from the experience buffer M [s ⁽ⁱ⁾ (t), Setpt _z ⁽ⁱ⁾ (t), r ⁽ⁱ⁾ (t), s ⁽ⁱ⁾ (t+1)] , where i is the sample number;

Step S436: When a sufficient number of samples are collected, a batch of samples are randomly selected, and the q values of these samples are calculated to update the Q network. The calculation formula is as follows:

Step S437: Use the mean square error formula as the loss function to correct the weight ω. The calculation formula is as follows:

Step S438: According to the delay strategy, update the target Q network after U steps and copy the weight parameter ω ^* = ω of the network;

Step S439: The environment simulation stops and the algorithm stops.

Adopting the above technical solution, the present invention has the following beneficial effects:

1. The present invention first refers to the event-driven method and designs a new event-driven MDP model to enable the agent to learn more efficiently. Then, when designing the trigger rules, important state changes are selected as events based on prior knowledge, and reasonable trigger conditions are designed so that the agent can only perform optimal control when necessary. Finally, an event-driven deep reinforcement learning algorithm improved based on the DQN algorithm is proposed. Compared with traditional reinforcement learning algorithms, event-driven deep reinforcement learning algorithms can make better use of events, accelerate the learning process and improve learning effects.

2. Traditional reinforcement learning methods control at fixed time steps, which may lead to waste of resources and low learning efficiency. In contrast, the event-driven deep reinforcement learning method of the present invention is based on the concept of "intermittent" and only updates decisions after important events occur, improving data utilization.

3. The event-driven deep reinforcement learning method of the present invention can capture and utilize some infrequently occurring states by learning dynamic nonlinear characteristics, such as indoor temperatures in different areas, and through predefined events.

4. The event-driven deep reinforcement learning method of the present invention can combine prior knowledge and assign variable weights in the event definition stage, and can flexibly adapt to unknown environments and improve learning speed.

5. The event-driven deep reinforcement learning method of the present invention only requires historical data of the building system as input, without establishing a complete building system model.

Description of the drawings

Figure 1 is a flow chart of an event-driven deep reinforcement learning building control method of the present invention;

Figure 2 is a framework diagram of the event-driven deep reinforcement learning method of the present invention;

Figure 3 is an event triggering flow chart of the present invention;

Figure 4 shows the backtracking diagram of traditional reinforcement learning;

Figure 5 is a traceback diagram of the event-driven deep reinforcement learning method of the present invention.

Detailed ways

In order to make the content of the present invention easier to understand clearly, the present invention will be described in further detail below based on specific embodiments and in conjunction with the accompanying drawings.

As shown in Figure 1, this embodiment provides an event-driven deep reinforcement learning building control method, which includes:

Step S1: Obtain weather data, generate a data set, and divide the data set into a training set and a test set. In the present invention, weather data comes from the Meteorological Bureau, part of which is used for training and part of which is used for testing. Considering that the research focus is on refrigeration, the hotter season was chosen for the experiment. Additionally, a simulated electricity price sequence was created in which electricity prices alternated between high and low values every four hours. The purpose of this simulated pricing sequence is to test whether the agent is able to detect the impact of price signals on the reward function and adjust the control strategy accordingly. Additionally, occupancy levels within residences can vary based on time of the week, with higher occupancy levels on weekends, depending on people's work schedules.

Step S2: Select a residential building with multiple areas as the building model, and create an experimental platform based on Python based on the thermodynamic formula. In this invention, a multi-region residential building is selected as the building model and implemented using a Python-based experimental platform. The building model is constructed based on thermodynamic formulas to simulate the impact of the HVAC system on indoor temperature. Through this experimental platform, the temperature of the building and other related parameters can be adjusted, providing a real environment for event-driven deep reinforcement learning methods.

Step S3: Design an event-driven MDP model, and design two types of events based on expert knowledge and prior knowledge.

Step S4: Design an event-driven DQN algorithm to optimize the control strategy when an event is triggered.

Step S5: Verify the effectiveness of the event settings through ablation experiments, and reasonably adjust event parameters based on the results. The present invention verifies the effectiveness of event settings through ablation experiments, observes which factors have the greatest impact on the results, and finally adjusts corresponding parameters in order to balance energy consumption and thermal comfort.

Step S6: Prove the control effect and superiority of the event-driven DQN algorithm through comparative experiments. Step 5: Result analysis and optimization. Compared with traditional methods and general reinforcement learning methods, event-driven DQN can learn faster, work less frequently, and can better balance thermal comfort and energy consumption. After testing, it has been verified that the event-driven DQN has good adaptability and generalization capabilities for different building thermal environments.

In step S3 of this embodiment, an event-driven MDP model is designed, which specifically includes the following steps:

Construct an event-driven six-tuple event-driven MDP model consisting of state, action, transition, reward function, discount factor and event trigger function, expressed as:

Among them, S represents the state set; A represents the action set that the agent can take; P represents the transfer of the system; R represents the reward function; γ represents the discount factor, which is used to measure the importance of the agent to future rewards; e represents the event trigger function ;

When the value of the event triggering function exceeds the preset threshold, the agent is triggered and updates the strategy, and a state transition occurs at the same time. The transition function is:

P{s(t+1)|s(t),a,e};

Under this setting, as shown in Figure 2, multi-region residential buildings are used as the environment and event-driven deep reinforcement learning is used as the intelligent agent to design the status, actions and rewards of the HVAC system.

Specifically, the status of this embodiment is determined by the environment. In the present invention, the indoor environmental state (indoor temperature and occupancy rate of each room) and the outdoor environmental state (outdoor temperature) are considered, which affect the energy consumption state (electricity price). The state is expressed as:

[T _in,z (t),K _z (t),T _out (t),λ ^electricity (t)]

Among them, T _{in, z} (t) is the indoor temperature of each room, K _z (t) is the occupancy rate, T _out (t) is the outdoor temperature, λ ^electricity is the electricity price, z represents the room number, and t represents the current time step. .

Specifically, the actions of this embodiment can be defined as control variables in the HVAC system. In the present invention, the action is defined as the temperature set point of the HVAC system, and the action is expressed as:

A(t)＝[Sp _z (t)]

where, SP _z (t) represents the temperature set point of room z; where the set point of each zone is a discrete variable and follows the following logic:

It can be seen that if the indoor temperature is higher than the temperature value of the temperature set point, the HVAC system is turned on; if the indoor temperature is lower than the temperature value of the temperature set point, the HVAC system is turned off; in other cases, the status remains unchanged.

Specifically, as a bonus to this embodiment, the primary goal of optimizing an HVAC system is to maintain occupant comfort while consuming as little energy as possible. For the multi-objective problem of balancing energy consumption and thermal comfort, two weight factors α and β can be added to it, and the reward function is expressed as:

Among them, λ ^retail (t') represents the retail price, E _HVAC (t') represents energy consumption, R ^comfort (t') represents the reward for keeping the temperature within the comfort range, and SW ^penalty (t') represents the penalty for switching on and off HVAC. item;

Given comfort range When the indoor temperature T _in (t) is higher than the lowest comfort temperature th ^min , the agent adjusts the settings of the HVAC system according to the reward size; when T _in (t) is lower than the lowest comfort temperature th ^min , in order to avoid the loss of energy consumption waste, consider turning off the HVAC;

When the execution action deviates from the threshold, a negative reward is added, which is specifically expressed as follows:

When the indoor temperature is within the thermal comfort threshold, a positive reward is given, and an optimal comfort temperature th ^best is defined. The closer the indoor temperature is to th ^best , the higher the positive reward. The positive reward is expressed as:

As shown in Figure 3, the trigger process of this embodiment presets a series of events. It is assumed that the current HVAC system environment is relatively stable, which means that the trigger conditions are not met, and the agent does not need to conduct strategy search and can continue to execute the current Actions. However, if the environment changes and trigger conditions are met, the agent needs to update its strategy. In the event-driven MDP model, the key is to design triggering rules. After the agent completes the observation, it can determine whether an event needs to be triggered based on the change rate of the last moment's observation and the current observation.

For example, when the indoor temperature exceeds a certain threshold, an event is triggered and the system automatically adjusts the temperature to maintain comfort. By determining these events in advance, the system can more easily capture a priori factors that affect environmental response, thereby improving the efficiency and adaptability of the system.

As shown in Figure 1, the two types of events in step S3 of this embodiment include two types of events designed in the present invention, state transition events and combination events. In addition, other types of events can be easily added to the event-driven deep reinforcement learning framework if needed.

Specifically, state transition events: certain state changes have a great impact on the operation of the system. Considering that the retail price λ ^retail (t) and the room occupancy rate K _z (t) have a great impact on energy consumption and thermal comfort, therefore List these two state changes as the first event and the second event. The state transition event includes a first event and a second event; specifically, when the current retail price is λ ^retail (t), and it is different from the previous moment λ ^retail (t'), the first event is triggered. The first event is defined as follows:

Among them, λ ^retail (t) and λ ^retail (t') are both within the price range [λ ^low , λ ^high ], λ ^low represents the lowest retail price, and λ ^high represents the highest retail price.

Similarly, the second event is defined as follows:

Among them, K _z (t') and K _z (t) are within [-1, 1], -1 means there is no one at the moment, and 1 means there is someone at the moment.

Specifically, combined events: When different states change at the same time, they can be listed as combined events. Considering that thermal comfort and energy consumption are optimization goals and change according to changes in room occupancy and price, the combined event is the third event. The third event consists of TH(t'), K _z (t') and λ ^retail (t ') combined;

When room occupancy or price changes, if When it is greater than the previously defined thermal comfort value χ, the third event is triggered. The third event is defined as follows:

in, Indicates thermal comfort value;/> The size of is equal to r ^comfort (t'), which allows the event to better respond to changes in reward and ensures that the indoor temperature remains within a comfortable range.

As shown in Figures 4 and 5, they are backtracking diagrams of reinforcement learning and event-driven deep reinforcement learning. Each solid point represents a state-action pair, and the hollow dot represents a state. In the traditional reinforcement learning process, the agent completes each learning step from the state-action pair in sequence. In event-driven deep reinforcement learning, the status and rewards are still periodic, but the actions are converted to non-periodic. If an event is triggered, the optimal strategy (s', a') is obtained under s'. If the event is not triggered, the optimal strategy (s", a') will be continued directly. It is worth mentioning that non-periodic actions It does not mean that the action is not performed sometimes, but that the previous action is directly used without conducting a strategy search.

As shown in Figure 1, in step S4 of this embodiment, an event-driven DQN algorithm is designed, which specifically includes the following steps:

Step S41: Initialize the Q network parameter ω, assign the target Q network value ω ^* =ω, and initialize all states and actions and corresponding q values based on the random Q network parameter ω;

Step S42: Initialize the minibatch size B and initialize the experience buffer M;

Step S43: Iterate (the number of iterations T is determined by the environment until the end of the simulation environment).

Specifically, the iterative process of step S43 is as follows:

Step S431: Obtain the current state quantity of the environment initialization, and perform preliminary processing to obtain the characteristic state parameters s (T _out (0), T _{in, z} (0), λ ^electricity (0), K _z (0));

Step S432. Use s as input in the Q network. If none of the events are triggered, the previous action is executed; if one of the events is triggered, all action outputs corresponding to the q value of the Q network are obtained, and the current q value is calculated through the ε greedy algorithm. The corresponding action a is selected from , the random action is selected with probability ε, and the action with the largest value function is selected with probability 1-ε. It is worth noting that ε gradually decreases during the training process until it reaches the minimum value, and the action a is expressed as:

Among them, s and A are elements in the ED-MDP algorithm model, s represents a certain state, and A is an action set;

Step S433: The current state s(t) executes action a, and obtains the processed characteristic state vector s(t+1) of the new environmental state and the reward R of the action;

Step S434: Put the tuple [s(t), Setpt _z (t), r(t), s(t+1)] into the experience buffer M;

Step S435: Randomly select B samples from the experience buffer M [s ⁽ⁱ⁾ (t), Setpt _z ⁽ⁱ⁾ (t), r ⁽ⁱ⁾ (t), s ⁽ⁱ⁾ (t+1)] , where i is the sample number;

Step S436: When a sufficient number of samples are collected, a batch of small samples are randomly selected, and the q values of these samples are calculated for updating the Q network. The calculation formula is as follows:

Step S437: Use the mean square error formula as the loss function to correct the weight ω. The calculation formula is as follows:

Step S438: According to the delay strategy, update the target Q network after U steps and copy the weight parameter ω ^* = ω of the network;

Step S439: The environment simulation stops and the algorithm stops.

The specific embodiments described above further describe in detail the technical problems, technical solutions and beneficial effects solved by the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the invention shall be included in the protection scope of the invention.

Claims (10)

1. The method for controlling the deep reinforcement learning building based on the event driving is characterized by comprising the following steps of:

step S1, acquiring weather data, generating a data set, and dividing the data set into a training set and a testing set;

s2, selecting a residential building with multiple areas as a building model, and creating an experimental platform based on Python according to a thermodynamic formula;

s3, designing an MDP model based on event driving, and designing two types of events under the condition of being based on expert knowledge and priori knowledge;

s4, designing an DQN algorithm based on event driving, and optimizing a control strategy when the event is triggered;

s5, verifying the effectiveness of event setting through an ablation experiment, and adjusting event parameters according to the result;

and S6, comparing experiments prove the control effect and superiority of the DQN algorithm based on event driving.

2. The event-driven deep reinforcement learning building control method according to claim 1, wherein in the step S3, an event-driven MDP model is designed, and the method specifically comprises the following steps:

constructing an event-driven six-tuple based event-driven MDP model consisting of states, actions, transitions, rewards functions, discount factors, and event-triggered functions, expressed as:

wherein S represents a state set; a represents a set of actions that an agent can take; p represents a transfer of the system; r represents a return function; gamma represents a discount factor for measuring the degree of importance of an agent to future rewards; e represents an event trigger function;

when the value of the event triggering function exceeds a preset threshold, the agent is triggered and updates the strategy, and state transition occurs at the same time, wherein the transition function is as follows:

P{s(t+1)|s(t),a,e}；

the state, action and rewards of the heating, ventilation and air conditioning system are designed by taking a multi-region residential building as an environment and event-driven deep reinforcement learning as an intelligent agent.

3. The event-driven deep reinforcement learning building control method of claim 2, wherein the states are expressed as:

[T _in,z (t),K _z (t),T _out (t),λ ^electricity (t)]

wherein T is _in,z (t) is the indoor temperature of each room, K _z (T) is the occupancy of personnel, T _out (t) is the outdoor temperature, lambda ^electricity For electricity price, z represents a roomThe number t indicates the current time step.

4. The event-driven deep reinforcement learning building control method of claim 2, wherein the action is a temperature set point of a hvac system, the action being expressed as:

A(t)＝[Sp _z (t)]

wherein SP is _z (t) represents a temperature set point for the z-room; wherein the set point for each zone is a discrete variable and follows the logic:

if the indoor temperature is higher than the temperature value of the temperature set point, the heating ventilation air conditioning system is started; if the indoor temperature is lower than the temperature value of the temperature set point, the heating ventilation air conditioner is turned off.

5. The event-driven deep reinforcement learning building control method of claim 2, wherein the reward function is expressed as:

wherein lambda is ^retail (t') represents retail price, E _HVAC (t') represents energy consumption, R ^comfort (t') represents a prize having a temperature in the comfort range, SW ^penalty (t') represents a punishment item of the switch heating ventilation air conditioner;

given comfort rangeWhen the indoor temperature T _in (t) is higher than the minimum comfort temperature th ^min When the intelligent agent adjusts the setting of the heating, ventilation and air conditioning system according to the rewarding size; when T is _in (t) below the minimum comfort temperature th ^min When the air conditioner is turned off;

when the action is performed away from the threshold, then a negative prize is increased, specifically as follows:

when the indoor temperature is within the thermal comfort threshold, a positive prize is awarded, defining an optimal comfort temperature th ^best The closer the indoor temperature is to th ^best The higher the positive prize, the positive prize is expressed as:

6. the event-driven deep reinforcement learning building control method of claim 1, wherein: the two types of events in step S3 include a state transition event and a combination event.

7. The event-driven deep reinforcement learning building control method of claim 6, wherein: the state transition event includes a first event and a second event, the first event being defined as follows:

wherein lambda is ^retail (t) and lambda ^retail (t') are all in the price range [ lambda ] ^low ,λ ^high ]In, lambda ^low Represents the lowest retail price, lambda ^high Representing the highest retail price.

The second event is defined as follows:

wherein K is _z (t') and K _z (t) is at [ -1,1]In, -1 indicates that no person is present at this time, and 1 indicates that a person is present at this time.

8. The event-driven deep reinforcement learning building control method of claim 6, wherein: the combined event is a third event, and the third event is composed of TH (t'), K _z (t') and lambda ^retail (t') are combined;

when the occupancy or price of the room changes, ifAbove the thermal comfort value χ, a third event is triggered, said third event being defined as follows:

wherein,indicating a thermal comfort value.

9. The event-driven deep reinforcement learning building control method of claim 1, wherein: in the step S4, designing an DQN algorithm based on event driving, which specifically includes the following steps:

step S41, initializing Q network parameters omega and assigning omega to the target Q network ^* =ω, initializing all states and actions based on the random Q network parameter ω and the corresponding Q value;

step S42, initializing the size B of the miniband and initializing an experience buffer area M;

step S43, iteration is carried out.

10. The event-driven deep reinforcement learning building control method according to claim 9, wherein the iterative process of step S43 is as follows:

step S431, obtaining the current state quantity of the environment initialization, and performing preliminary processing to obtain the characteristic state parameter S (T _out (0),T _in,z (0),λ ^electricity (0),K _z (0))；

Step S432, taking S as input in the Q network, and executing the previous action if the event is not triggered; if one of the events is triggered, obtaining action outputs of all corresponding Q values of the Q network, selecting a corresponding action a from the current Q values through an epsilon greedy algorithm, wherein the action a is expressed as:

wherein s represents a certain state, and A is an action set;

step S433, executing action a in the current state S (t) to obtain a feature state vector S (t+1) processed by the new environmental state and a reward R of the action;

step S434, tuple [ S (t), setpt _z (t),r(t),s(t+1)]Put into experience buffer M;

step S435, randomly extracting B samples [ S ] from the experience buffer M ⁽ⁱ⁾ (t),Setpt _z ⁽ⁱ⁾ (t),r ⁽ⁱ⁾ (t),s ⁽ⁱ⁾ (t+1)]Where i is the sample number;

step 436, when a sufficient number of samples are collected, randomly selecting a batch of samples, and calculating Q values of the samples for updating the Q network, wherein the calculation formula is as follows:

step S437, the weight omega is corrected by using a mean square error formula as a loss function, wherein the calculation formula is as follows:

step S438, updating the target Q network after the step U according to the delay strategy, and copying the weight parameter omega of the network ^* ＝ω；

And step S439, stopping the environment simulation and stopping the algorithm.