WO2025010050A2

WO2025010050A2 - A system for controlling the data synchronization process involved in the integration of iot (internet of things) devices and digital twin technology in smart city applications and operation method thereof

Info

Publication number: WO2025010050A2
Application number: PCT/TR2024/050940
Authority: WO
Inventors: Gökhan YURDAKUL; Özgür PALANTÖKEN; Lal Verda ÇAKIR; Kübra DURAN; Berk Canberk; Craig THOMSON; Matthew BROADBENT
Original assignee: Bts Kurumsal Bilisim Teknolojileri AS
Current assignee: Bts Kurumsal Bilisim Teknolojileri AS
Priority date: 2024-06-07
Filing date: 2024-08-13
Publication date: 2025-01-09
Anticipated expiration: 2026-12-07
Also published as: WO2025010050A3

Abstract

The invention relates to a system for controlling the data synchronization process involved in the integration of IoT (internet of things) devices and Digital Twin technology in smart city applications, and methods for operating the system.

Description

A SYSTEM FOR CONTROLLING THE DATA SYNCHRONIZATION PROCESS INVOLVED IN THE INTEGRATION OF IOT (INTERNET OF THINGS) DEVICES AND DIGITAL TWIN TECHNOLOGY IN SMART CITY APPLICATIONS AND OPERATION METHOD THEREOF

Technical Field

The invention relates to a system for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications, and methods for operating the system.

State of the Art

A digital twin is defined as a real-time virtual model of an object or system. A digital twin uses real data about a real-life object or system as input and generates predictions and control strategies about how the real object or system will react. Digital twin systems created with these models are used in the development of smart city applications.

In the state of the art, digital twins are used in smart cities to control rules in transportation systems, manage emergencies, plan resources and monitor environmental conditions. These systems are used to create a digital twin model of the data coming from the physical environment frequently in order to create a digital twin model, and this process is called digital twin data synchronization. With this model, the environment can be monitored and decision mechanisms can be operated. These digital twin models are mostly based on modeling semantic, spatial and/or temporal relationships by applying graph-based techniques. These graphs are usually stored using relational databases and file systems based on JavaScript Object Notation, and other graphical databases are also found in new studies. At this point, in the previous techniques developed using graphical databases, the synchronization processes of data are excluded. As a result, it hinders the reproducibility and applicability of the systems to real world.

At this point, data synchronization is achieved at high frequencies in order to increase the realism of the digital twin model. If there is no change in the physical environment between the two synchronization processes, the value in the model will not change as a result of the synchronization. This leads to the accumulation of data that does not contribute to the operation of the system in the digital twin model and consumes memory and processor resources.

Therefore, it is necessary to develop a method that minimizes or eliminates the above- mentioned disadvantages in the state of the art and a system that works according to said method.

Summary of the Invention

One of the advantages of the invention is that the need for definitions of data synchronization processes can be met in the existing digital twin data synchronization techniques mentioned in the state of the art. Unnecessary consumption of resources can be prevented. In the invention, a data synchronization system based on reinforcement learning has been developed, and at this point, by using reinforcement learning, the necessary decision about which data to transfer or not to transfer to the model can be made easily with the decision mechanism it contains.

Another advantage of the invention is that a method will be developed on how the graphs used in digital twin modeling will be updated during the synchronization process.

Description of the Drawings

Figure 1 : A representative schematic illustration of the system developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications of the invention.

Figure 2: A representative flow diagram illustration of the method developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications of the invention.

Description of the References in the Drawings

For a better understanding of the invention, the description of the numbers in the figures is given below:

100. System

1. loT device

2. Physical layer

3. Digital twin layer 4. loT data queue

5. Data transfer controller

6. Data synchronization module

7. Digital twin data storage unit

8. Digital twin modeling

9. Data transmitter

10. Step function module

11. Q neural network module

12. Optimization module

13. Target neural network module

14. State module

15. Reward calculation module

16. Replay memory storage unit

1001. Generating s_t with the State Module (14), entering s_t into the Q Neural Network Module (11) and transferring it to the Replay Memory Storage Unit (16)

1002. Calculating Q values with the Q Neural Network Module (11), entering Q values in the Step Function module (10) and transferring Q values to the Optimization Module (12)

1003. Generating a_t with Step Function (10)

1004. Checking that there is at least one data in each of at least one loT Data queue (4), if there is, transmitting a_t to the data transmitter (9) and transferring it to the replay memory storage unit (16)

1005. Transferring the data at the top of the queue in the loT data queues (4) that corresponds to one in a_t to the data synchronization module (6) using the Data Transmitter (9), and deleting the data at the top of the queue in the loT data queues (4) that corresponds to value of zero in a_t

1006. Arrival of data in the data synchronization module (6)

1007. Creating (8) nodes in digital twin modeling for each data

1008. Creating the temporal relationship in the digital twin modeling (8) with the most recently created node for the loT device (1) they represent

1009. For each node created in step (1007), determining the loT devices (1) that the node should be in a spatial relationship with

1010. For each determined one in step (1009), creating the spatial relationship in the digital twin modeling (8) with the most recently created node in the digital twin modeling (8) of the loT device (1 ) 1011. Generating s_t+1 with State Module (14) and transferring it to the replay memory storage unit (16)

1012. Generating r_t with reward calculation module (15) and transferring it to the replay memory storage unit (16)

1013. Saving (s_t, a_t, r_t, s_t+1) bundle with replay memory storage unit (16)

1014. Checking the number of bundles in the replay memory storage unit (16), and if sufficient, inputting its content to the target neural network module (13)

1015. Calculating the Q' values with the target neural network module (13) using the data from the replay memory storage unit (16) and entering the Q' values into the optimization module (12)

1016. Calculating the loss function with the optimization module (12) and transmitting the result to the Q neural network module (11 )

1017. Updating the weights of the Q neural network module (12)

1018. Checking whether the iteration limit has been completed

1019. If completed, copying the target neural network module (13) to the Q neural network module (11 )

Detailed Description of the Invention

The example embodiments are described in more detail below with reference to the accompanying descriptions. Furthermore, the embodiments can be established in different forms and should not be interpreted as being limited to the embodiments specified herein. Rather, these example embodiments are provided so that this description will be thorough, and will fully convey the scope to those skilled in the art.

The terminology used in this description is intended to describe a particular example embodiment only and is not intended to be limiting. As used herein, the forms "a", "at least", and "preferably" are intended to include plural forms as well, unless the context clearly states otherwise.

The system (100) of the invention developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications comprises the following elements; - At least one loT device (1 ) that collects data from the physical layer (2) and regularly sends this data to the Digital twin layer (3),

- At least one physical layer (2) comprising the environment to be to be digitally twinned

- At least one digital twin layer (3) with at least one digital twin modeling (8) that can project the physical layer (1 ) in real time,

- At least one loT data queue (4) where data from loT devices (1 ) is queued,

- At least one data transfer controller (5) that controls the transfer of data waiting in loT data queues (4) to the digital twin modeling (8),

- At least one data synchronization module (6) that processes the data to be transferred to the digital twin modeling (8) and writes it to the digital twin data storage unit (7),

- At least one digital twin data storage unit (7) that houses digital twin modeling (8),

- At least one digital twin modeling (8), which is the data from the physical layer (2) modeled according to its spatial and temporal properties,

- At least one data transmitter (9) that transfers to the data synchronization module (6) or deletes data according to the action generated by the step function module (10),

- At least one step function module (10) that generates action by applying a step function to the Q values calculated in the Q neural network module (1 1 ),

- At least one Q neural network module (11 ) that calculates Q values using the state data of the environment,

- At least one optimization module (12) that optimizes the weights in the Q neural network module (11 ),

- At least one target neural network module (13), which has the same deep neural network structure as the Q neural network module (1 1 ) and is used in the evaluation of decisions made by the Q neural network module (1 1 ),

- At least one state module (14) that creates the state rotation of the environment by using data from digital twin modeling (8),

- At least one reward calculation module (15) that calculates a reward for the applied action using data from digital twin modeling (8),

- At least one replay memory storage unit (16) that retains the past experience of the data transfer controller (5). The operation method of the system of the invention developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications comprises the process steps of;

- Generating s_t with the State Module (14), entering s_t into the Q Neural Network Module (11 ) and transferring it to the Replay Memory Storage Unit (16) (1001 ),

- Calculating Q values with the Q Neural Network Module (11 ), entering Q values in the Step Function module (10) and transferring Q values to the optimization module (12) (1002),

- Generating a_t with Step Function (10) (1003),

- Checking that there is at least one data in each of at least one loT Data queue (4), if there is, transmitting a_t to the data transmitter (9) and transferring it to the replay memory storage unit (16) (1004),

- Transferring the data at the top of the queue in the loT data queues (4) that corresponds to value of one in a_t to the data synchronization module (6) with the Data Transmitter (9), and deleting the data at the top of the queue in the loT data queues (4) that corresponds to value of zero in a_t (1005)

- Arrival of data in the data synchronization module (6) (1006),

- Creating (8) nodes in digital twin modeling for each data (1007),

- Creating the temporal relationship in the digital twin modeling (8) with the most recently created node for the loT device (1 ) they represent (1008),

For each node created in step (1007), determining the loT devices (1 ) that the node should be in a spatial relationship with (1009),

For each determined one in step (1009), creating the spatial relationship in the digital twin modeling (8) with the most recently created node in the digital twin modeling (8) of the loT device (1 ) (1010),

- Generating s_t+1 with State Module (14) and transferring it to the replay memory storage unit (16) (101 1 ),

- Generating r_t with reward calculation module (15) and transferring it to the replay memory storage unit (16) (1012),

- Saving (s_t, a_t,r_t, s_t+1) bundle with replay memory storage unit (16) (1013), - Checking the number of bundles in the replay memory storage unit (16), and if sufficient, inputting its content to the target neural network module (13) (1014),

- Calculating the Q' values with the target neural network module (13) using the data from the replay memory storage unit (16) and entering the Q' values into the optimization module (12) (1015),

- Calculating the loss function with the optimization module (12) and transmitting the result to the Q neural network module (1 1 ) (1016), Updating the weights of the Q neural network module (12) (1017), Checking whether the iteration limit has been completed (1018),

If completed, copying the target neural network module (13) to the Q neural network module (1 1 ) (1019).

The invention relates to a system for controlling the data synchronization process involved in the integration of lot (internet of things) devices and Digital Twin technology in smart city applications, and methods for operating the system. At this point, the environment in which the smart city application is developed forms the Physical Layer (2), and the Digital Twin is located in the Digital Twin Layer (3). In the Physical Layer (2), there are loT devices (1 ) deployed for the purpose of collecting data, and these devices provide streaming data flow to the Digital Twin Layer (3). The data in these data flows is generated by each loT Device (1 ) by reading from the environment at the same time. The Digital Twin Layer (3) creates a digital twin modeling (8) in the form of a spatial- temporal graph using the data transmitted from the Physical Layer (2) and stores it in the digital twin data storage unit (7), which is a graph database. In this process, the data that reaches the Digital Twin Layer (3) is held in loT Data Queues (4), and there is one for each loT Device (1 ) of these loT Data Queues (4). Data synchronization process is carried out by transferring the data in loT Data Queues (4) to Digital Twin Modeling (8). While performing this transfer, the invention reduces the resource consumption of the digital twin Data Storage Unit (7) by preventing the transfer of data to the digital twin Modeling (8) that will not change the accuracy of the digital twin Modeling (8) with the Reinforcement Learning-based Data Transfer Controller (5).

The data decided to be transferred by the Reinforcement Learning-Based Data Transfer Controller (5), the Reinforcement Learning-Based Data Transfer Controller (5) works as shown in Figure 2. The decision mechanism in it works using deep Q learning and includes two deep neural networks. These neural networks are kept in the Q Neural Network Module (1 1 ) and the Target Neural Network Module (13). Q Neural Network Module (1 1 ), calculates Q values using s_t created by State Module (14). These Q values are passed through the Step Function (10) and a_t is created. At the same time, the Q values are transmitted to the Optimization Module (12) for later use. a_t is a vector containing values one and zero with a length equal to the number of loT devices (1 ) in the system. Data transmitter transfers or deletes front-row data in loT Data Queues (4) using values in a_t. This transfer takes place as detailed above. Then, from the updated digital twin modeling (8), new state information s_t+1 by the state module (14), and reward r_t by reward calculation module (15) are calculated. The reward calculation module (15) works according to the following formula;

Wherein, N is the number of loT Devices (1 ), F is the number of items represented by the data received from each loT device (1 ), X is the digital twin modeling (8), and E is the matrix of spatial relationships.

As a result of these operations, (s_t, a_t,r_t, s_t+1) bundle is obtained and stored in the replay memory storage unit (16). This storage process continues until a limit is reached, and when the limit is reached, input is given to the target neural network module (13). The target neural network module (13) calculates Q' with these past experiences and transmits it to the Q' optimization module (12). The optimization module (12) calculates the loss function given below.

Wherein 0; is the weights of the Q neural network module (11) and 0' is the weights of the target neural network (13). Using the loss function, the weights of the Q neural network (1 1 ) are optimized by the optimization module (12). Furthermore, the Q neural network is copied to the target neural network module (13) at certain iterations. Using these two different networks, the training process and copying are used to stabilize the training process. All modules in the system and method of the invention perform the operations mentioned in the invention over the processor included in the computer through a software.

Industrial Applicability of the Invention The invention is a system for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications and methods for operating the system, and is industrially applicable.

The invention is not limited to the example embodiments above, and the person skilled in the art can readily present other different embodiments of the invention. These should be considered within the protection scope of the invention claimed by the claims.

Claims

1. The system (100) developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications, characterized in that it comprises;

- At least one loT device (1 ) that collects data from the physical layer (2) and regularly sends this data to the Digital twin layer (3),

- At least one physical layer (2) comprising the environment to be digitally twinned

- At least one loT data queue (4) where data from loT devices (1 ) is queued,

- At least one Q neural network module (1 1 ) that calculates Q values using the state data of the environment,

- At least one optimization module (12) that optimizes the weights in the Q neural network module (1 1 ),

- At least one target neural network module (13), which has the same deep neural network structure as the Q neural network module (11 ) and is used in the evaluation of decisions made by the Q neural network module (1 1 ),

- At least one state module (14) that creates the state rotation of the environment by using data from digital twin modeling (8), - At least one reward calculation module (15) that calculates a reward for the applied action using data from digital twin modeling (8),

- At least one replay memory storage unit (16) that retains the past experience of the data transfer controller (5).

2. A system (100) according to claim 1 , characterized in that the reward calculation module (15) operates according to the following formula;

3. A system (100) according to claim 1 , characterized in that the optimization module (12) calculates the loss function using the following formula; \²

(r_t + ymaxQ(s_t+1, ^at+i> 0') ) - Q s, a; 0) I

\ ^at+i / J

4. The operation method of the system (100) developed for controlling the data synchronization process involved in the integration of loT (internet of things) devices and Digital Twin technology in smart city applications according to claim 1 , characterized in that it comprises the process steps of;

- Calculating Q values with the Q Neural Network Module (11), entering Q values in the Step Function module (10) and transferring Q values to the optimization module (12) (1002),

- Generating a_t with Step Function (10) (1003),

- Checking that there is at least one data in each of at least one loT Data queue (4), if there is, transmitting a_t to the data transmitter (9) and transferring it to the replay memory storage unit (16) (1004), - Transferring the data at the top of the queue in the loT data queues (4) that corresponds to value of one in a_t to the data synchronization module (6) with the Data Transmitter (9), and

- deleting the data at the top of the queue in the loT data queues (4) that corresponds to value of zero in a_t (1005),

- Arrival of data in the data synchronization module (6) (1006),

- Creating nodes in digital twin modeling (8) for each data (1007),

- For each node created in step (1007), determining the loT devices (1 ) that the node should be in a spatial relationship with (1009),

- For each determined one in step (1009), creating the spatial relationship in the digital twin modeling (8) with the most recently created node in the digital twin modeling (8) of the loT device (1 ) (1010),

- Saving (s_t, a_t, r_t, s_t+1) bundle with replay memory storage unit (16) (1013),

- Checking the number of bundles in the replay memory storage unit (16), and if sufficient, inputting its content to the target neural network module (13) (1014),

- Calculating the loss function with the optimization module (12) and transmitting the result to the Q neural network module (1 1 ) (1016),

- Updating the weights of the Q neural network module (12) (1017),

- Checking whether the iteration limit has been completed (1018),

- If completed, copying the target neural network module (13) to the Q neural network module (1 1 ) (1019).