WO2025042425A1 - Semantic integration of opc ua information models in a process control system - Google Patents

Abstract

Methods for semantic integration of OPC UA information models in a process control system and corresponding systems and computer-readable mediums. A method (300) performed by a federated information system (200) includes receiving (306) a query from a calling process, by a first node (202a) of the federated information system (200), for required information. The method includes propagating (308) the query, by the first node (202a) to a plurality of other nodes (202b, 202c) of the federated information system (200). The method includes identifying (310), by each of the plurality of other nodes (202b, 202c), responsive information corresponding to the query and aggregating (312) the responsive information from each of the plurality of other nodes (202b, 202c). The method includes storing (312) the aggregated responsive information in a knowledge graph of the first node (202a) and returning (314) the aggregated responsive information to the calling process by the first node (202a).

Description

SEMANTIC INTEGRATION OF OPC UA INFORMATION MODELS IN A PROCESS CONTROL SYSTEM

CROSS-REFERENCE TO OTHER APPLICATION

[0001] This application claims the benefit of the filing date of U.S. Provisional Patent Application 63/578,269 filed August 23, 2023, which is hereby incorporated by reference.

TECHNICAL FIELD

[0002] The present disclosure is directed, in general, to process control systems, including manufacturing systems. In particular, improved systems and methods are disclosed for controlling such systems using integrated data models.

BACKGROUND OF THE DISCLOSURE

[0003] Process control systems, including manufacturing systems, must receive data from and/or send data to any number of components. These components can include any number of sensors, actuators, motors, subsystems, and others. These can include analog components with analog-to-digital converters and digital component. Each of these components may operate using different data and signaling protocols, making efficient and unified control of the systems difficult. Improved systems are desirable.

SUMMARY OF THE DISCLOSURE

[0004] Various disclosed embodiments include a method performed by a federated information system. The method includes receiving a query from a calling process, by a first node of the federated information system, for required information. The method includes propagating the query, by the first node of the federated information system, to a plurality of other nodes of the federated information system. The method includes identifying, by each of the plurality of other nodes, responsive information corresponding to the query. The method includes aggregating, by the federated information system, the responsive information from each of the plurality of other nodes. The method includes storing the aggregated responsive information in a knowledge graph of the first node. The method includes returning the aggregated responsive information to the calling process by the first node.

[0005] In various embodiments, each of the plurality of other nodes has a knowledge graph. Various embodiments also include aggregating data from a plurality of control level devices and transforming the aggregated data into a knowledge graph of at least one node of the federated information system.

[0006] In various embodiments, propagating the query includes federating the query such that subqueries are sent only to other nodes that have responsive information corresponding to the subquery. In various embodiments, each node of the federated information system has a schema that represents an information model and information stored in a knowledge graph of that node. In various embodiments, the first node maintains a configuration file that describes information stored in each of the other nodes. In various embodiments, aggregating the responsive information is performed by merging the responsive information using a recursive process to create a single object an object that includes aggregated responsive information.

[0007] Various embodiments also include integrating a new node into the federated information system by introspecting the new node and adding configuration information of the new node to a configuration file of the first node. [0008] Disclosed embodiments also include a federated information system having a plurality of nodes, each node implemented by at least one computer system having a processor and an accessible memory. The federated information system particularly configured to execute a process as described herein.

[0009] Disclosed embodiments also include a process control system comprising a plurality of physical resources connected to communicate with a federated information system as described herein.

[0010] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detailed description that follows. Additional features and advantages of the disclosure will be described hereinafter that form the subject of the claims. Those skilled in the art will appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art will also realize that such equivalent constructions do not depart from the spirit and scope of the disclosure in its broadest form.

[0011] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words or phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or” is inclusive, meaning and/or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, whether such a device is implemented in hardware, firmware, software or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior as well as future uses of such defined words and phrases. While some terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0013] FIG. 1 illustrates a block diagram of a data processing system in which an embodiment can be implemented;

[0014] FIG. 2 illustrates an example of a federated information system in accordance with disclosed embodiments;

[0015] FIG. 3 illustrates a flowchart of a process performed by a federated information system as disclosed herein;

[0016] FIG. 4 illustrates a non-limiting example of schema definitions in accordance with disclosed embodiments;

[0017] FIG. 5 is a non-limiting example of a configuration file in accordance with disclosed embodiments;

[0018] FIG. 6 is a non-limiting example of an algorithm that can be used to federate queries in accordance with disclosed embodiments; and

[0019] FIG. 7 illustrates a process that can be performed by the federated information system to integrate new nodes.

DETAILED DESCRIPTION

[0020] FIGURES 1 through 7, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged device. The numerous innovative teachings of the present application will be described with reference to exemplary non-limiting embodiments.

[0021] The Fourth Industrial Revolution or “Industry 4.0” refers to the trend towards automation and data exchange in manufacturing technologies and processes which include cyber-physical systems (CPS), Internet of Things (loT), Industrial Internet of Things, cloud computing, cognitive computing, and artificial intelligence. The convergence of operational and information technology in the era of Industry 4.0 has significantly increased the volume of data being generated at manufacturing shop floors.

[0022] These techniques require the production and management of large amounts of data, which can advantageously be “vertically integrated” with other information present throughout the different systems in manufacturing facilities, such as from enterprise resources planning and manufacturing execution systems software. When possible, data is often normalized to conform to the Open Platform Communication (OPC) Unified Architecture (UA) specifications, which may be obtained at time of filing opcfoundation.org. Some systems focus on aggregating this data through utilizing centralized aggregation servers to create an information system.

[0023] Disclosed embodiments include a framework, implemented with and in a process control system, which adopts a federated information system capable of connecting to different OPC UA servers and storing the data within knowledge graph-based information models. In various embodiments, servers supporting the GraphQL query language, from The GraphQL Foundation, can be used for easy accessibility of information by users. Disclosed systems provide modularity, scalability, and interoperability to that expand and improve the capabilities of current information systems.

[0024] FIG. 1 illustrates a block diagram of a data processing system 100 (or computer system 100) in which an embodiment can be implemented, for example as a computer system particularly configured by software or otherwise to perform the processes as described herein, and in particular as each one of a plurality of interconnected and communicating systems as described herein, such as any of the nodes described below.

[0025] In particular, data processing system 100 is simply one example of a computer system that can be used as one of many communicably connected systems in a process control system that includes vertically-integrated systems and a federated information system as described in more detail below. Those of skill in the art will

[0026] The data processing system depicted includes a processor 102 connected to a level two cache/bridge 104, which is connected in turn to a local system bus 106. Local system bus 106 may be, for example, a peripheral component interconnect (PCI) architecture bus. Also connected to local system bus in the depicted example are a main memory 108 and a graphics adapter 110. The graphics adapter 110 may be connected to display 111.

[0027] Other peripherals, such as local area network (LAN) / Wide Area Network (WAN) / Wireless (e.g. WiFi) adapter 112, may also be connected to local system bus 106. Expansion bus interface 114 connects local system bus 106 to input/output (I/O) bus 116. I/O bus 116 is connected to keyboard/mouse adapter 118, disk controller 120, and I/O adapter 122. Disk controller 120 can be connected to a storage 126, which can be any suitable machine usable or machine readable storage medium, including but not limited to nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), magnetic tape storage, and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs), and other known optical, electrical, or magnetic storage devices. [0028] Also connected to I/O bus 116 in the example shown is audio adapter 124, to which speakers (not shown) may be connected for playing sounds. Keyboard/mouse adapter 118 provides a connection for a pointing device (not shown), such as a mouse, trackball, trackpointer, touchscreen, etc.

[0029] I/O adapter 122 can be connected to any number of physical components 150 that may be present in a process control or manufacturing system, including but not limited to sensors, actuators, motors, subsystems, and others.

[0030] Storage 126 can include any data or other information necessary or useful for performing the processes described herein, including executable code 162, query 163, information 166, schemas 168, configurations 170, and other data 172.

[0031] Those of ordinary skill in the art will appreciate that the hardware depicted in Figure 1 may vary for particular implementations. For example, other peripheral devices, such as an optical disk drive and the like, also may be used in addition or in place of the hardware depicted. The depicted example is provided for the purpose of explanation only and is not meant to imply architectural limitations with respect to the present disclosure.

[0032] A data processing system in accordance with an embodiment of the present disclosure includes an operating system employing a graphical user interface. The operating system permits multiple display windows to be presented in the graphical user interface simultaneously, with each display window providing an interface to a different application or to a different instance of the same application. A cursor in the graphical user interface may be manipulated by a user through the pointing device. The position of the cursor may be changed and/or an event, such as clicking a mouse button, generated to actuate a desired response.

[0033] One of various commercial operating systems, such as a version of Microsoft Windows™, a product of Microsoft Corporation located in Redmond, Wash, may be employed if suitably modified. The operating system is modified or created in accordance with the present disclosure as described. [0034] LAN/WAN/Wireless adapter 112 can be connected to a network 130 (not a part of data processing system 100), which can be any public or private data processing system network or combination of networks, as known to those of skill in the art, including the Internet. Data processing system 100 can communicate over network 130 with other systems 140, which is also not part of data processing system 100, but can be implemented, for example, as a separate data processing system 100 as a server system and can represent any number of other intercommunicating systems as described herein.

[0035] In some embodiments, the data processing system 100 can communicate with the physical components 150 via LAN/WAN/Wireless adapter 112 and network 130, particularly in Internet of Things (loT) or Industrial Internet of Things (IIoT) implementations.

[0036] To achieve more holistic decision making, disclosed embodiments integrate data with enterprise resource planning systems, the product lifecycle management, computer- aided design, and manufacturing software, creating an enterprise-wide system. Vertical integration is used to tackle the issue of interoperability between different software, equipment, and processes.

[0037] Vertical integration is a cornerstone of Industry 4.0 aiming at integrating all the vertical layers within an organization. This integration allows strategic and tactical decisions-making since the relevant data is transparent and accessible among all the layers of the manufacturers' system, increasing the company’s competitiveness in the market.

[0038] Disclosed embodiments include systems and methods for management and use of data within a vertically-integrated control system. Particular embodiments apply to the vertical integration of “shop floor” equipment or, more generally, data generated at the production level.

[0039] FIG. 2 illustrates an example of a federated information system 200 in accordance with disclosed embodiments, used to manage data at the production level of a vertically integrated system 270. [0040] Vertically integrated system 270 can be used as a “big data” architecture to implement Industry 4.0 techniques and includes a hierarchy of data and control levels. Vertically integrated system 270 represents, for example, a process control system configured and capable of controlling a physical system. Such a vertically integrated process control system, as vertically integrated system 270, can be, for example, a manufacturing system, an HVAC system, an energy generation system, a fuel refining system, an oil or gas collection system, an electric, gas, water, or other utility system, a wastewater processing system, a building control system, a building management system, a rail transport system, a traffic control system, an industrial automation system, or other system.

[0041] Field Level 272 includes field-level components and other physical resources such as sensors and actuators. These physical resources can include any number of sensors, actuators, motors, and other physical devices connected and configured to monitor or control physical conditions. For example, the physical resources can include (but are not limited to) the physical sensors and devices used to monitor and control various aspects of any number of physical systems, including manufacturing systems, boiler systems, HVAC systems, energy generation systems, traffic control systems, and others.

[0042] Control level 274 includes low-level physical and software controllers, such as programmable logic controllers, application-specific integrated circuits, and connected devices. Any of these control level components can communicate with the field-level components as described above, and can communicate with the control-level devices 204 described below.

[0043] Production level 276 can include IIoT devices and Supervisory Control and Data Acquisition (SCADA) systems and processes, and additional systems and processes as described in more detail below. Operations Level 278 can include manufacturing execution systems, cloud storage systems, and others. Enterprise Planning Level 280 can include Enterprise Resource Planning (ERP) systems and processes and other application-level systems for managing the vertically integrated system 270 overall. [0044] Disclosed embodiments include a federated information system 200, implemented by one or more computer systems 202a-202c (collectively, computer systems 202 or computers 202), each of which can be implemented for example using a data processing system 100, configured to perform processes as described herein. Federated information system 200, in various embodiments, operates in cooperation with or as a part of production level 276, as described herein. Computer systems 202 can intercommunicate, for example by using the MQTT communications protocol documented at mqtt.org.

[0045] As described herein each component in the federated information system 200 can undergo semantic integration, information storage, and information federation. Computer systems 202 are able to integrate a large volume of heterogeneous data sources. A federated information system 200 as disclosed herein provides a significant technical improvement over existing systems in that it is fault tolerant, modular, scalable, and interoperable. Further, federated information system 200 is capable of real time scalability by adding and removing components during run time.

[0046] Computer systems 202 can implement knowledge graphs (KGs) 208 as described herein, and can support query languages such as the GraphQL query language from graphql.org or the REST API from restdb.io.

[0047] In various embodiments, computer systems 202a-202c can communicate with one or more Control Level 274 devices 204a-204c (collectively, devices 204). Devices 204 can process, store, and communicate data using the OPC UA protocols, including communicating with the computer systems 202. Devices 204 can be implemented as OPC UA servers. Devices 204 can include and communicate with other low-level physical and software controllers, such as programmable logic controllers, application-specific integrated circuits, and connected devices, as may be used in Control Level 274.

[0048] At the shop floor or production level, PLCs and similar devices provide the logic to proceed with the manufacturing process while also reading data from the different sensors, transducers, and other assets at the field level. This data is then transmitted through OPC UA servers, such as devices 204. OPC UA is a service-oriented architecture adopted by manufacturers as a means of data exchange between machines. Some of the key features of OPC UA include platform independence, secure communication, scalability, and information modelling. Some of the key research trends for OPC UA include integration of devices, interoperability with legacy systems, and integration with other protocols/architectures.

[0049] Disclosed embodiments can aggregate or integrate OPC UA information models from multiple servers into one model. Aggregating components 206 (illustrated as aggregating components 206a-206c) connect the servers via OPC UA services and aggregates the types, instances, and structure information. Aggregating components 206 can centralize information, can control distributed cyber-physical systems, information and data processing, and device integration, and can superimpose security mechanisms. The architecture of an aggregating component 206 can be thought of as an aggregation layer above data sources which can consolidate the information under a single platform.

[0050] A number of different implementations can be used for aggregating component 206. Different examples of aggregation servers have been implemented such as an aggregation server in which OPC UA was used for information retrieval and distribution from databases to clients. This server unifies all the information in the databases to be accessed by the clients. Another example is an aggregating server design utilizing OPC UA technology. The design can be used, for example, for production control of a flexible manufacturing system or for remote monitoring of mobile work machines.

[0051] Aggregating components 206 can perform a number of different aggregations. One approach, for example, is unification aggregation, which ingests all the information from an OPC UA server and stores them in the aggregators local model as-is. This type of aggregation allows all the information models found in source servers to be unified into one model at the aggregation layer. Another type of aggregation is abstraction where logic is applied on the information models coming from the source servers and the extracted insight from the logic is stored at the aggregation layer. This type of aggregation leads to unique information models at the aggregation layer.

[0052] Some systems use a single server for aggregation so that all aggregation is done at one component of the developed systems. This approach may introduce issues such as excessive workload and low resilience as these systems expand. If all aggregation processes need to be accomplished using one device, then an increase of data sources will lead to an increased need for processing power. At a certain point, the aggregating component might reach its load capacity limiting the system’s scalability. Should the workload for that server prove too large for the server to handle, the server would fail, shutting down the entire system. Disclosed embodiments, use the federated information system 200 the aggregation of OPC UA information models, ensuring scalability and flexibility not found in existing systems.

[0053] Federated information system 200 solves the issue of providing the intermediary step between the OPC UA servers and an enterprise knowledge graph (KG). The KG organizes information in a structured manner and enables enhanced search, browsing, integration, and data analysis capabilities. The KG can include two main components: nodes and edges. Nodes represent assets, people, places, or objects, while edges establish relationships between different nodes. A KG can include two sections: the Schema/Ontology and the Instantiations. The Schema/Ontology defines the classes used in the KG, specifying the domain of discourse through a finite list of terms and relationships. A Resource Description Framework (RDF) serves as an international standard for Semantic Web data, representing web resources and is used to structure data into a KG. It offers a common framework for information representation, enabling interoperability between applications. RDF resources are documented using Uniform Resource Identifiers (URI) and consist of three types: subject, object, and predicate.

[0054] The disclosed federated information system 200 also provides the added technical advantage of using multiple aggregating components 206, implemented as separate applications or systems, providing the aggregation capability.

[0055] FIG. 3 illustrates a flowchart of a process 300 performed by a federated information system as disclosed herein.

[0056] At 302, the federated information system aggregates data from control level devices, such as PLCs, using different instances of an aggregating component 206, such as different instances of an aggregation application. Each instance can be connected to a different set of sources at the control level and hence ingesting different data. For example, each aggregating component 206 can collect and aggregate control level data from one or more OPC UA servers, such as devices 204. This process can be considered “ingesting” the information from the control level devices. During this process, the federated information system may transform the data according to an RDF.

[0057] At 304, the federated information system transforms the aggregated data to KGs. This can be performed, for example, using technologies such as the RDFlib Python library described at rdflib.readthedocs.io or the OWL API application programming interface described at owlapi.sourceforge.net. These tools can be connected to the control level 274 or devices 204 using, for example, OPC UA communication protocol. The federated information system, using computers 202, can also implement higher-level applications to connected to the KBs and use the data from multiple applications to reason and extract implicit abstract information about the production line. These reasoning capabilities can be achieved using technologies such as Jena, RDFlib, or OWL API. The different information can be relayed between the applications in the federation using MQTT protocol or the gRPC remote procedure call framework. This process can include creating any necessary schemas or ontologies for the KGs.

[0058] At 306, the federated information system can receive a query for required information (“required” in the sense that it is the information requested by the query). “Receiving,” as used herein, can include loading from storage, receiving from another device or process, receiving via an interaction with a used, or otherwise. The query may be received by any computer system in the federated information system or information system application running on such computer system. For reference herein, each such system or application is referred to as a “node” of the federated information system and the receiving node is referred to as the “queried node.” In specific embodiments with hierarchical nodes, the queried node may be considered the “parent” node and the other nodes are its “children” nodes. A user, device, or process (the “calling process”) can send a query to the queried node/application in the federated system and the query is then be propagated to the appropriate nodes until all the required information is retrieved. This enables a user to use one “point of contact” to the federation. Each node can function as a GraphQL server.

[0059] At 308, the federated information system propagates the query to at least one other node (and, typically, to a plurality of other nodes). For example, if computer system 202a is the queried node, it can propagate the query to computer systems 202b and 202c.

[0060] At 310, the federated information system identifies responsive information at the at least one other node. Since different applications/nodes have different information models and as such hold different data from multiple control levels, the systems must work together to retrieve the data required. Each node identifies the responsive information in its own KG. Each node can use a GraphQL API as the point of communication.

[0061] At 312, the federated information system aggregates responsive information from each of the other nodes and stores it in the KG at the queried node. This can be performed using the query resolution and merging techniques discussed below.

[0062] At 314, the federated information system returns the aggregated responsive information to the calling process. The queried node returns the response, having stored the aggregated responsive information in its own KG from throughout the federated information system.

[0063] A Knowledge Graph (KG) as used herein organizes information in a structured manner. KGs offer enhanced search, browsing, integration, and data analysis capabilities. The KG is made up of two main components: nodes and edges. Nodes represent assets, people, places, or objects, while edges establish relationships between different nodes. A KG can include two sections: the Schema/Ontology and the Instantiations. The Schema/Ontology defines the classes used in the KG, specifying the domain of discourse through a finite list of terms and relationships.

[0064] The use of KGs in various embodiments can help facilitate the integration and storage of data from various sources. KGs allow the integration of large volumes of heterogenous data sources. Additionally, introducing reasoning mechanisms allows deduction of insights and knowledge about the manufacturing system that were previously inaccessible. This allows for different abstraction logic to be applied on the KG.

[0065] The transformation of OPC UA models to KGs provides advantages in interoperability, including increased capabilities for OPC UA information models such as querying and increased formal semantics. Disclosed embodiments can use KGs for information storage within each component in the federated information system. Alongside its capability for storing information, KGs introduce semantic definitions behind the data and can be used for both unification and abstraction.

[0066] Various embodiments employ the GraphQL query language, from The GraphQL Foundation, to improve accessibility of the information to end users. Data in GraphQL, as used in disclosed embodiments, is structured as a graph and defined by a schema. It provides a simple interface for data retrieval which enables users to input intuitive queries based of the defined types in the GraphQL schema. GraphQL is compatible, as disclosed, with OPC UA and can be a more viable and accessible alternative than REST due to its more intuitive interface and queries and performance and interoperability.

[0067] As described in the process of FIG. 3, the control level data can be ingested by different nodes in the federation before it is translated into RDF format to be stored in the form of KGs in each separate node. Communication channels between them can be used to exchange information, and new communication channels can be established whenever applicable. Each node can have its own GraphQL server running independently for the data to be queried by end users. A GraphQL schema is also established for each node. Once that has been determined, a query is sent to the queried node in the federated system. This query will request information that is not stored in the queried node, consequently, it will have to communicate with the different nodes within the federation to obtain the information. As such, upon receiving the query, the federated information system can federate the query so that only the appropriate subqueries are sent to the different nodes. That is, propagating the query can include federating the query such that subqueries are sent only to other nodes that have responsive information corresponding to the subquery. Once all the subqueries are resolved, all the results are sent back to the parent node through the different GraphQL servers to be merged into one object and shown to the user.

[0068] As described above, different communication channels are used and can be created for the federated information system. These can include a number of different communication protocols as illustrated in FIG. 2. OPC communication can be used with the control-level devices, and MQTT channels can be used between the nodes. These channels are used for the exchange of information in the different models that these nodes contain. As an example, when a node in the bottom layer receives a new device reading, this data will be stored in that node and then published onto the common MQTT broker where the higher-level nodes (i.e., parent node in this prototype) can read it and perform reasoning before storing the result as an insight in its own information model. Another means of communication is through GraphQL Servers, which allows easy accessibility to the information being stored within the nodes. Clients can send a query to the parent node to retrieve any information in the federation. These servers are solely used for query federation and resolution, not for exchange of information for storage within each node.

[0069] As described above, the federated information system can “federate” the queries before propagating to other notes. This can be performed by creating and using appropriate GraphQL schemas in each node. This schema can be exposed to the users to form the basis of queries. In a hierarchical implementation, each child node’s schema represents its own information model, and the parent node’s schema includes the information stored within it and its children. In this way, each node of the federated information system has a schema that represents an information model and information stored in a knowledge graph of that node.

[0070] FIG. 4 illustrates a non-limiting example of schema definitions in accordance with disclosed embodiments. The children each have a schemas 402 and 404 made up of one object type which is hasSensor. This object type has two fields that can be queried: sensorName which allows the user to see which sensor that specific node is connected to, and hasValue which allows the user to retrieve the current value of the sensor. On the other hand, the parent node schema 400 has a main object type called Devices which allows the user to send a query to uncover all the devices that are connected to the parent. This class has an id field which returns a device ID alongside the hasSensor object which has the same structure seen in the child schema. This repetition is required since the user sending the query to the parent must also be exposed to the children’s schema.

[0071] Disclosed embodiments can autonomously federate a single query. In that sense, the parent node (or queried node) will take in the query sent by the user and create multiple subqueries that can be sent to its children to resolve every part of the query. For this to be possible, a configuration file needs to be available. This configuration file will contain the metadata about the schemas and system.

[0072] FIG. 5 is a non-limiting example of a configuration file 502 in accordance with disclosed embodiments. This file contains the schemas of every node within the query object in each node object. This file also has metadata such as the URL of each node’s GraphQL server and a list of all the nodes connected to the parent node. Most importantly, this file also has information on what data is present at each of its child nodes. To generalize, each node in the federation will have a similar configuration file to allow queries to propagate down till they reach a node in which they can be resolved. The parent node maintains the confirmation file and schema that describes information stored in each of the other nodes.

[0073] FIG. 6 is a non-limiting example of an algorithm 600 that can be used to federate queries in accordance with disclosed embodiments. The function used creates one query at a time for the nodes direct children. Hence, the function will take the required node as input alongside the configuration file, the original query keys or words, and the subquery that it is creating. This subquery is required as input since this a recursive algorithm that needs to continually add or remove objects to the subquery. The configuration file that is passed as an input is only the object that is related to the node that the subquery is being created for (i.e., object with key DI if creating a subquery for DI). This function will then iterate through all the keys in that object and the words of the original query. Once a key and word match, should that key’s value be another object instead of a value (i.e., nested object), then the function is called recursively with the new object being passed as the configuration. This will repeat until the function reaches a key with a value. If the value is equal to the node ID that the subquery is being created for (i.e., the node has the value stored in its model) then that key is added to the subquery. However, if no key from the object is added, then that object is removed completely from the subquery as that indicates that device does not have any data for that object. Once the subquery is finalized, it is returned as an output of the function. All the subqueries are then added to a list creating the final list of queries.

[0074] As described above, the federated information system can perform query resolution and merging. The system can do so to resolve all the subqueries and merge them into one object to be sent back to the requesting device, process, or user. Once the parent resolver receives the initial GraphQL query from the user, it can parse the configuration file and take in the list of all the direct child nodes. Based on that list, a subquery will be created for each node as described above.

[0075] For example, a first query can be resolved locally as the IDs of the different nodes in the federation are stored in the parent node. To maintain the same structure of the original query, it can be transformed into a JSON object with the value for the keys defaulting to null at first. All the subqueries are then sent to their corresponding GraphQL servers using the URLs found in the configuration file such as illustrated in FIG. 5. The subqueries are then resolved in the separate nodes. Once their responses are received, they are integrated with the data that was resolved locally. This is done through iterating through the JSON object created and writing in the appropriate values to the keys present in it. The function created for this part is similar to the one created for federating the query in that it uses recursiveness to iterate through nested dictionaries and compares keys with the keys in the query responses. This formatted response is finally sent back to the calling process to view the results. In this way, aggregating the responsive information is performed by merging the responsive information using a recursive process to create a single object an object that includes aggregated responsive information. [0076] FIG. 7 illustrates a process 700 that can be performed by the federated information system to integrate new nodes. Using such as process, the federated information system can seamlessly introduce new nodes into the federation during run time with minimal configuration effort, providing a significant technical advantage in scalability of the system. The federation can integrate new nodes autonomously by inspecting the schema that the new node utilizes, integrating that schema within the larger schema present in the parent node, and obtaining the metadata required to undergo regular functions.

[0077] As shown in this figure, at 702, the federated information system sends a “mutation” (described below) to the parent node containing information about the new node to be integrated. In various embodiments, the GraphQL Server running at the parent node is the primary or only poiont of contact between the client and the system. As such, any information needed to configure the new node can be provided through a GraphQL query. In various embodiments, a mutation is used, which is a GraphQL query that can modify the server-side data.

[0078] For example, a mutation can be defined in the parent node called AddDeviceMutation. This mutation takes in two parameters as input. The first input is the address of the GraphQL server running on the new child node being introduced, and the second being a device ID to be specified by the client. The ID is used to create a new node instance in the federation which will be used as the identifier for the new child. The address is used to initiate the communication with the new node.

[0079] At 704, the parent node introspects the schema of the new node to determine configuration information. This can be performed, for example, by sending an introspection query on behalf of the parent node to the new child node. The child then responds by projecting its schema to the parent node.

[0080] At 706, the configuration information for the new node is added to the configuration file. Here, the parent node parses the schema of the child node and adds it as a new object in the configuration JSON object. At this point, should the ID be present already in the configuration file, the mutation is voided and that is relayed back to the client. Similarly, a RemoveDeviceMutation can be used, which requires an ID as an input. This mutation removes all information about that node from the configuration file so that no subqueries can be federated to it.

[0081] At 708, the schema of the new node is integrated into the schema for the parent node. After the information about the new node was configured into the system, the next step was to expose the schema of the new node for the client to query the information within it. For this to take place, the GraphQL schema of the new node must be merged with the schema of the parent node. As such the schemas will be merged without the disruption of the server.

[0082] The schema merging also takes place when the AddDeviceMutation is called by the client. After the configuration file is updated, the parent node will then communicate with the child node to retrieve its specific schema file within the child node through MQTT. This step can be performed when the introspection done for populating the configuration file does not provide enough information about each object type in the schema that this method provides. This interchange begins when the parent node publishes a request on the designated MQTT broker topic. When the new node reads the message, the full schema file is loaded as message and returned to the parent node through the same topic.

[0083] Once the parent node receives the new schema it integrates it with its own preexisting schema. Since the federation follows a hierarchical system, the schema also follows suit. The parent node will always have the topmost object type defined within its schema with the child’s topmost object type defined under the parent’s. In some embodiments, the main query begins with the “devices” key. Within this key, further queries can be formed such as projecting the different device IDs and the information in each device. Hence, whenever a new node is introduced, its schema will be integrated under the “devices” object type of the parent schema. This provides a standardized approach since each subquery represents one level down in the hierarchy.

[0084] An exemplary embodiment can use the graphene library to create the GraphQL schemas for each node. This library uses classes to define the objects in the schema and the relationships between objects. Hence, when merging the two schemas, all the classes need to be copied over to the schema file as is from the child node, then the objects defined in the “Query” class of the child node are copied into the “devices” class of the parent node to maintain the hierarchy.

[0085] Disclosed embodiments provide a number of technical advantages over current systems. The disclosed system is modular since each node can operate autonomously and scalable since new nodes can be easily introduced to the federation. This system is also scalable since there is no limit to how many nodes the federation can have. This system is also fault tolerant since it can still remain operational even if some nodes experience issues or failures unlike centralized systems.

[0086] This disclosed federated approach also enhances the systems security and privacy. Since the information is no longer centralized, the information can be distributed across multiple devices. Confidential data can be stored in specific devices with restricted access to key individuals only. On top of that, any data breach from one device can be handled by shutting down that device and its links with the system while the remainder of the system can remain operational as opposed to shutting down the full system in the case of a centralized system.

[0087] The disclosed system is also interoperable as it can use standardized technologies such as the Semantic Web and KGs to define its information models alongside the OPC UA model. With this standardization, data can be exchanged between different nodes through machine-to-machine communication. It is also accessible by users through its GraphQL interface which provides a simple and intuitive experience for data retrieval.

[0088] The following documents described features and aspects of various systems and methods that may be related to those described herein and which can be used to modify or implement various features herein. These documents are incorporated by reference:

• E. Auschitzky, M. Hammer and A. Rajagopaul, “How big data can improve manufacturing,” McKinsey & Company, 1 July 2014. [Online], Available: https://www.mckinsey.com/capabilities/operations/our-insights/how-big-data-can- improve-manufacturing. [Accessed August 2023],

• “The convergence of IT and OT,” Siemens, [Online], Available: https://www.siemens.com/global/en/products/automation/topic-areas/digital- enterprise/it-ot-convergence.html. [Accessed August 2023],

• F. Ocker, B. Vogel-Heuser and C. J.J. Paredis, “A framework for merging ontologies in the context of smart factories,” Computers In Industry, vol. 135, 2022.

• C. Li, Y. Chen and Y. Shang, “A review of industrial big data for decision making in intelligent manufacturing,” Engineering Science and Technology, an International Journal, vol. 29, 2022.

• K. Hanson, “Using Data to Deliver Results,” Society of Manufacturing Engineers,

19 June 2023. [Online]. Available: https://www.sme.org/technologies/articles/2023/june/using-data-to-deliver- results/?ite=6180&ito=2433&itq=d8851cl5-bf77-4dac-bf7d-

18f72400a035&itx%5Bidio%5D=4675426. [Accessed August 2023],

• S. Jasko, A. Skrop, T. Holczinger, T. Chovan and J. Abonyi, “Development of manufacturing execution systems in accordance with Industry 4.0 requirements: A review of standard- and ontology-based methodologies and tools,” Computers in Industry, vol. 123, 2020.

• “OPC Foundation,” [Online], Available: https://opcfoundation.org/about/opc- technologies/opc-ua. [Accessed July 2023],

• I. Gonzalez, A. J. Calderon, J. Figueiredo and J. M. C. Sousa, “A Literature Survey on Open Platform Communications (OPC) Applied to Advanced Industrial Environments,” Electronics, 2019. • D. GroBmann, M. Bregulla, S. Banerjee, D. Schulz and R. Braun, “OPC UA server aggregation — The foundation for an internet of portals,” in Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), Barcelona, 2014.

• C. P. latrou, L. Ketzel, M. Graube, M. Hafner and L. Urbas, “Design classification of aggregating systems in intelligent information system architectures,” in 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2020.

• S. Banerjee and D. GroBman, “Aggregation of Information Models-An OPC UA Based Approach to a Holistic Model of Models,” in 2017 4th International Conference on Industrial Engineering and Applications, 2017.

• S. G. Mathias, S. Schmied and D. Grossmann, “A framework for monitoring multiple databases in industries using OPC UA,” Journal of Ambient Intelligence and Humanized Computing, pp. 47-56, 2021.

• I. Seilonen, T. Tuovinen, J. Elovaara, I. Tuomi and T. Oksanen, “Aggregating OPC UA servers for monitoring manufacturing systems and mobile work machines,” in 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA), 2016.

• R. S. Crespi, A. Armentia, I. Sarachaga, O. Casquero, F. Perez and M. Marcos, “OPC UA Aggregation Server in the Fog,” in 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2019.

• H. Wang, Y. Ma and F. Yu, “An OPC UA Multi-Server Aggregator with Cache Management,” in 2018 Chinese Automation Congress (CAC), 2018.

• J. N. Weskamp and J. Tikekar, “An Industrie 4.0 compliant and self-managing OPC UA Aggregation Server,” in 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2021. • J. Bakakeu, M. Brossog, J. Zeitler, J. Franke, S. Tolksdorf, H. Klos and J. Peschke, “Automated Reasoning and Knowledge Inference on OPC UA Information Models,” in 2019 IEEE International Conference on Industrial Cyber Physical Systems (ICPS), 2019.

• S. Schmied, D. GroBmann, S. G. Mathias and S. Banerjee, “Vertical Integration via Dynamic Aggregation of Information in OPC UA,” in Asian Conference on Intelligent Information and Database Systems, 2020.

• S. Schmied, D. GroBmann, S. G. Mathias and R. K. Mueller, “An approach for aggregation and historicization of production entities in the graph,” in 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2020.

• A. Fernbach, W. Kastner, S. Matzler and M. Wollschlaege, “An OPC UA information model for cross-domain vertical integration in automation systems,” in Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), 2014.

• M. Lezoche, E. Yahia, A. Aubry, H. Panetto and M. Zdravkovic, “Conceptualising and structuring semantics in cooperative enterprise information systems models,” Computers in Industry, vol. 63, no. 8, pp. 775-787, 2012.

• S. Busse, R.-D. Kutsche, U. Leser and H. Weber, “Federated Information Systems: Concepts, Terminology and Architectures,” Forschungsberichte des Fachbereichs Informatik, pp. 1-38, 1999.

• R. Huber, A. M. Oberlander, U. Faisst and M. Rbglinger, “Disentangling Capabilities for Industry 4.0 - an Information Systems Capability Perspective,” Information Systems Frontiers, 2022.

• V. M. Tabim, N. F. Ayala and A. Frank, “Implementing Vertical Integration in the Industry 4.0 Journey: Which Factors Influence the Process of Information Systems Adoption?,” Information Systems Frontiers, 2021. • D. Romero and F. Vernadat, “Enterprise information systems state of the art: Past, present and future trends,” Computers in Industry, vol. 79, pp. 3-13, 2016.

• Y. Eslami, M. Lezoche, P. Kalitine and S. Ashouri, “How the Cooperative Cyber Physical Enterprise Information Systems (CCPEIS) improve the Semantic Interoperability in the domain of Industry 4.0 through the Knowledge formalization,” in International Federation of Automatic Control, 2021.

• M. Liebenberg and M. Jarke, “Information systems engineering with Digital Shadows: Concept and use cases in the Internet of Production,” Information Systems, vol. 114, 2023.

• G. Shukair, L. Nikolaos, V. Peristeras and S. SklarB, “Towards semantically interoperable metadata repositories: The Asset Description Metadata Schema,” Computers in Industry, vol. 64, no. 1, pp. 10-18, 2013.

• S. Gehring, “GraphQL Federation in Python,” Medium, 31 January 2020. [Online] . Available: https://atechnologistspov. com/graphql-federation-in-python- 46eaf8a4bl85. [Accessed June 2023],

• C. Constantinescu, U. Heinkel, R. Rantzau and B. Mitschang, “SIES: An Approach for a Federated Information System in Manufacturing,” in Proceedings of the International Symposium on Information Systems and Engineering (ISE), 2001.

• K.-D. Schmatz, K. Berwind and M. Hemmje, “An Interface to Heterogeneous Data Sources Based on the Mediator/Wrapper Architecture in the Hadoop Ecosystem,” in 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2018.

• H. Panetto, M. Zdravkovic, R. Jardim-Goncalves, D. Romer, J. Cecil and I. Mezgar, “New perspectives for the future interoperable enterprise systems,” Computers in Industry, vol. 79, pp. 47-63, 2016. • A. Sheth, S. Padhee and A. Gyrard, “Knowledge Graphs and Knowledge Networks: The Story in Brief,” IEEE Internet Computing, vol. 23, pp. 67-75, 2019.

• Parsons, S. (2009). A Semantic Web Primer, Second Edition by Antoniou Grigoris and Harmelen Frank van, MIT Press, 288 pp., $42.00. The Knowledge Engineering Review, 24(4), 415-415.

• “W3 Schools: XML RDF,” [Online]. Available: https://www.w3schools.com/xml/xml_rdf.asp. [Accessed August 2023],

• G. Buchgeher, D. Gabauer, J. Martinez-Gil and L. Ehrlinger, “Knowledge Graphs in Manufacturing and Production: A Systematic Literature Review,” IEEE Access, vol. 9, 2021.

• G. Steindl and W. Kastner, “Transforming OPC UA Information Models into Domain-Specific Ontologies,” in 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), 2021.

• R. Schiekofer, S. Grimm, M. M. Brandt and M. Weyrich, “A formal mapping between OPC UA and the Semantic Web,” in 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), 2019.

• L. Byron, “GraphQL A data query language,” Meta, 14 September 2015. [Online], Available: https://engineering.fb.com/2015/09/14/core-data/graphql-a- data-query-language/. [Accessed August 2023],

• “Introduction to GraphQL,” The GraphQL Foundation, [Online], Available: https://graphql.org/learn/. [Accessed 2023],

• E. Wittern, A. Cha and J. A. Laredo, “Generating GraphQL-Wrappers for REST(- like) APIs,” in International Conference on Web Engineering, 2018. • J. Hietala, R. Ala-Laurinaho, J. Autiosalo and H. Laaki, “GraphQL Interface for OPC UA,” in 2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), 2020.

• R. T. Fielding, “Architectural Styles and the Design of Network-based Software Architectures,” Ph.D. dissertation, University of California, Irvine, 2000.

• G. Brito and M. T. Valente, “REST vs GraphQL: A Controlled Experiment,” in 2020 IEEE International Conference on Software Architecture (ICSA), 2020.

• R. Ala-Laurinaho, J. Mattila, J. Autiosalo, J. Hietala, H. Laaki and K. Tammi, “Comparison of REST and GraphQL Interfaces for OPC UA,” Computers, vol. 11, no. 5, 2022.

• “Flask The Pallets Projects,” [Online], Available: https://palletsprojects.eom/p/flask/. [Accessed August 2023],

[0089] Of course, those of skill in the art will recognize that, unless specifically indicated or required by the sequence of operations, certain steps in the processes described above may be omitted, performed concurrently or sequentially, or performed in a different order.

[0090] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all data processing systems suitable for use with the present disclosure is not being depicted or described herein. Instead, only so much of a data processing system as is unique to the present disclosure or necessary for an understanding of the present disclosure is depicted and described. The remainder of the construction and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0091] It is important to note that while the disclosure includes a description in the context of a fully functional system, those skilled in the art will appreciate that at least portions of the mechanism of the present disclosure are capable of being distributed in the form of instructions contained within a machine-usable, computer-usable, or computer- readable medium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal bearing medium or storage medium utilized to actually carry out the distribution. Examples of machine usable/readable or computer usable/readable mediums include: nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs).

[0092] Although an exemplary embodiment of the present disclosure has been described in detail, those skilled in the art will understand that various changes, substitutions, variations, and improvements disclosed herein may be made without departing from the spirit and scope of the disclosure in its broadest form.

[0093] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: the scope of patented subject matter is defined only by the allowed claims. Moreover, none of these claims are intended to invoke 35 USC §112(f) unless the exact words “means for” are followed by a participle. The use of terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller,” within a claim is understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 U.S.C. §112(f).

Claims

WHAT IS CLAIMED IS:

1. A method (300) performed by a federated information system (200), comprising: receiving (306) a query from a calling process, by a first node (202a) of the federated information system (200), for required information; propagating (308) the query, by the first node (202a) of the federated information system (200), to a plurality of other nodes (202b, 202c) of the federated information system (200); identifying (310), by each of the plurality of other nodes (202b, 202c), responsive information corresponding to the query; aggregating (312), by the federated information system (200), the responsive information from each of the plurality of other nodes (202b, 202c); storing (312) the aggregated responsive information in a knowledge graph of the first node (202a); and returning (314) the aggregated responsive information to the calling process by the first node (202a).

2. The method of claim 1, wherein each of the plurality of other nodes (202b, 202c) has a knowledge graph.

3. The method of claim 1, further comprising: aggregating data (302) from a plurality of control level devices (274); and transforming (304) the aggregated data into a knowledge graph (208) of at least one node of the federated information system (200).

4. The method of claim 1, propagating the query includes federating the query such that subqueries are sent only to other nodes (202b, 202c) that have responsive information corresponding to the subquery.

5. The method of claim 1, each node (202a, 202b, 202c) of the federated information system (200) has a schema (400) that represents an information model and information stored in a knowledge graph (208) of that node.

6. The method of claim 1, wherein the first node (202a) maintains a configuration file (502) that describes information stored in each of the other nodes (202b, 202c).

7. The method of claim 1, wherein aggregating the responsive information is performed by merging the responsive information using a recursive process to create a single object that includes aggregated responsive information.

8. The method of claim 1, further comprising integrating a new node (202) into the federated information system (200) by introspecting the new node (202) and adding configuration information of the new node (202) to a configuration file (502) of the first node.

9. A federated information system (200) comprising a plurality of nodes (202a, 202b, 202c), each node (202a, 202b, 202c) implemented by at least one computer system (100) having a processor (102) and an accessible memory (108), the federated information system (200) particularly configured to execute a method (300) as in any of claims 1-8.

10. A process control system (270) comprising a plurality of physical resources (272) connected to communicate with a federated information system (200) as in claim 9.