CN120872436A - NodeRed-based data acquisition system and NodeRed-based data acquisition method - Google Patents
NodeRed-based data acquisition system and NodeRed-based data acquisition methodInfo
- Publication number
- CN120872436A CN120872436A CN202510747725.5A CN202510747725A CN120872436A CN 120872436 A CN120872436 A CN 120872436A CN 202510747725 A CN202510747725 A CN 202510747725A CN 120872436 A CN120872436 A CN 120872436A
- Authority
- CN
- China
- Prior art keywords
- data
- node
- nodered
- point table
- mqtt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Computer And Data Communications (AREA)
Abstract
本发明公开了一种基于NodeRed的数据采集系统及方法，通过扩展和增强NodeRed平台的现有功能，让点表配置更加简单，可适应多种设备数据格式；数据采集系统涉及NodeRed平台和MQTT服务器，所述NodeRed平台中设置有点表配置模块、Modbus通信模块、字节序调整模块以及MQTT客户端模块；所述点表配置模块用于配置待采集数据点的相关信息，形成点表配置；所述Modbus通信模块按照点表配置采集数据点的数据，形成第一数据；所述字节序调整模块将第一数据进行字节序转换，得到第二数据；所述MQTT客户端模块将第二数据发布至所述MQTT服务器，属于数据采集技术领域。
This invention discloses a data acquisition system and method based on NodeRed. By extending and enhancing the existing functions of the NodeRed platform, it simplifies point table configuration and adapts to various device data formats. The data acquisition system involves a NodeRed platform and an MQTT server. The NodeRed platform includes a point table configuration module, a Modbus communication module, a byte order adjustment module, and an MQTT client module. The point table configuration module configures relevant information of the data points to be acquired, forming a point table configuration. The Modbus communication module acquires data from the data points according to the point table configuration, forming first data. The byte order adjustment module converts the byte order of the first data to obtain second data. The MQTT client module publishes the second data to the MQTT server. This invention belongs to the field of data acquisition technology.
Description
Technical Field
The invention belongs to the technical field of data acquisition, and particularly relates to a NodeRed-based data acquisition system and method.
Background
Along with the rapid development of the industrial Internet of things, the acquisition, processing and remote transmission of industrial field data become more and more important, modbus is widely used as a communication protocol in the industrial automation field and is widely applied to various industrial equipment, MQTT is used as a lightweight publishing/subscribing message transmission protocol and is widely applied to the Internet of things, and in the data acquisition process, after the Modbus acquires equipment data, the equipment data is forwarded through the MQTT;
NodeRed is used as a popular visual programming tool, provides a rich node library comprising Modbus nodes and MQTT nodes, so that a developer can quickly construct a data stream processing application, however, the Modbus nodes based on the NodeRed platform and the MQTT nodes have some limitations that one is complicated in dot table configuration and needs to add or write codes one by one, and one is not capable of supporting byte order adjustment and cannot adapt to byte order formats.
Disclosure of Invention
The invention mainly aims to provide a NodeRed-based data acquisition system and a NodeRed-based data acquisition method, which can enable the configuration of a point table to be simpler and adapt to various equipment data formats by expanding and enhancing the existing functions of a NodeRed platform.
According to a first aspect of the invention, a NodeRed-based data acquisition system is provided, which relates to a NodeRed platform and an MQTT server, wherein a point table configuration module, a Modbus communication module, a byte order adjustment module and an MQTT client module are arranged in the NodeRed platform;
the point table configuration module is used for configuring related information of data points to be acquired to form point table configuration;
the Modbus communication module acquires data of data points according to the point table configuration to form first data;
The byte order adjustment module performs byte order conversion on the first data to obtain second data;
and the MQTT client module issues second data to the MQTT server.
In the data acquisition system based on NodeRed, a data collection module is arranged in the NodeRed platform;
and the data aggregation module reports second data to the MQTT client module according to a predefined strategy.
The data acquisition system based on NodeRed further comprises a Redis server, and a breakpoint continuous storage module is further arranged in the NodeRed platform;
The breakpoint continuous storage module temporarily stores second data to the Redis server, and then issues the second data to the MQTT client module from the Redis server.
In the data acquisition system based on NodeRed, the point table configuration module includes an object node, a template node, a CSV node, and a first function node;
the object node is used for triggering a point table configuration flow;
The CSV format text comprises the name, IP address, function code, address, target data type and byte order conversion rule of the equipment to be acquired;
The CSV node analyzes the CSV format text of the point table, and converts the CSV format text into data point configuration of a JSON format;
The first function node processes data point configuration, including defining data conversion into target data type rules, address resolution and point table ordering, to form point table configuration;
and saving the processed point table configuration into a flow context.
In the data acquisition system based on NodeRed, the Modbus communication module includes a node-red-control-Modbus node;
And the node-red-control-Modbus node establishes Modbus connection with industrial equipment according to the IP address in the point table configuration, reads data from the equipment and converts the data into a target data type to obtain first data.
In the data acquisition system based on NodeRed, the Modbus communication module includes a data point combining node, and the data point combining node combines the data points of adjacent addresses into a request.
In the data acquisition system based on NodeRed, the byte order adjustment module includes a second function node, and the second function node performs byte order conversion on the first data according to a byte order conversion rule, so as to obtain second data.
In the data acquisition system based on NodeRed, the MQTT client module includes a third function node and a MQTT out node;
The third function node converts the second data into an MQTT message format to obtain third data;
The MQTT out node issues the third data to the MQTT server.
In the data acquisition system based on NodeRed, the breakpoint continuous storage module includes a buffered-queue-pub node, a buffered-queue-sub node, a complex node and a buffered-queue-ack node;
The buffered-queue-pub node issues second data to a queue of the Redis server;
the buffered-queue-sub node subscribes second data from the queue, and publishes the subscribed second data to the MQTT client module;
the complex node is used for monitoring the completion state issued by the MQTT client module;
If the MQTT client module issues the second data to the MQTT server successfully, confirming that the second data is processed by using the buffered-queue-ack node, and removing the second data from a queue; if the MQTT client module issues second data to the MQTT server is unsuccessful, the second data will remain in the queue.
According to a second aspect of the present invention, there is provided a NodeRed-based data acquisition method, for data acquisition using a system as described in the first aspect, comprising the steps of:
acquiring a point table, and automatically analyzing the point table into a point table configuration according to the point table;
Configuring data of collected data points according to a data point table to form first data;
performing byte order conversion on the first data to obtain second data;
and publishing the second data to the MQTT server.
One of the above technical solutions of the present invention has at least one of the following advantages or beneficial effects:
According to the invention, the point table configuration module can be used for configuring the point tables in batches, automatically analyzing the point table configuration without adding or writing codes one by one, and the byte order conversion can be automatically carried out after the data is collected from the equipment based on the byte order adjustment module, so that a plurality of byte order formats are supported, the data formats of different equipment can be adapted, and the adjustment on the equipment side is not needed.
Drawings
The invention is further described below with reference to the drawings and examples;
FIG. 1 is a schematic diagram of a data acquisition system of the present invention;
fig. 2 is a flow chart of a data acquisition method of the present invention.
Wherein the drawings in the figures identify:
1. NodeRed platform, 11 point table configuration module, 12 Modbus communication module, 13 byte order adjustment module, 14 MQTT client module, 15 data collection module, 16 breakpoint continuous storage module, 2 MQTT server, 3 Redis server.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.
NodeRed is used as a popular visual programming tool, provides a rich node library, comprises Modbus nodes and MQTT nodes, and enables a developer to quickly construct data stream processing application, however, the Modbus nodes based on the NodeRed platform and the MQTT nodes have some limitations that the configuration of a point table is complex, codes need to be added or written one by one, and the node library does not support byte order adjustment and cannot adapt to byte order formats;
Therefore, the application provides a NodeRed-based data acquisition system, which enables the configuration of the point table to be simpler by expanding and enhancing the existing functions of the NodeRed platform, can easily configure the point table under the condition of large data volume, can realize byte order adjustment, is suitable for various equipment data formats, and enables the acquired equipment to be diversified.
In one embodiment of the present invention, referring to fig. 1, a NodeRed-based data acquisition system is provided, and relates to a NodeRed platform 1 and an MQTT server 2, wherein a point table configuration module 11, a Modbus communication module 12, a byte order adjustment module 13 and an MQTT client module 14 are arranged in a noded platform 1.
The point table configuration module 11 is used for configuring relevant information of data points to be acquired to form point table configuration, so that the subsequent Modbus communication module 12 can conveniently acquire data;
specifically, the point table configuration module 11 includes an object node, a template node, a CSV node, and a first function node;
Triggering a point table configuration flow through an object node;
In practice, the name, IP address, function code, address, target data type, multiplication, divisor, byte order conversion rule and other relevant information of the equipment to be acquired are listed in advance to form a point table, the point table is copied to the template node, the template node can convert the point table into CSV format text, and the CSV format text comprises the name, IP address, function code, address, data type, multiplication, divisor, byte order conversion rule and other relevant information of the equipment to be acquired;
then, analyzing a CSV format text of the point table through the CSV node, and converting the CSV format text into a data point configuration of a JSON format;
then, processing data point configuration through a first function node, including operations such as defining data conversion into a target data type rule, address resolution, point table ordering and the like, to form point table configuration;
defining a data conversion rule to be a target data type, wherein the rule defines how data read by the data point is converted to be the target data type according to the function code, the data types comprise Int16, uint16, int32, uint32, float, int64, uint64, double and the like, but the modbus is not defined in the way, but needs to be judged by combining the function code, so that which data type the data point needs to be converted to and how the data is converted can be defined according to the function code, and the data length of the data point and the byte order of the data point when the data point is output are further known;
Address resolution refers to mapping the address of a data point to a specific location in the device where the data is stored;
the sorting of the point table means sorting the addresses of the data points after analysis, if the addresses of the data points are adjacent, the adjacent addresses are sequentially sorted after sorting, so that the acquisition of the subsequent data is facilitated;
and saving the processed point table configuration in the flow context for other modules to use.
The Modbus communication module 12 collects data of data points according to the point table configuration to form first data;
Specifically, a node-red-control-Modbus node is installed in NodeRed platform 1 to realize Modbus protocol communication;
The node-red-control-Modbus node obtains point table configuration from the flow context, establishes Modbus connection with industrial equipment according to an IP address in the point table configuration, reads data from the equipment according to a function code, an address and a data type in the point table configuration, and obtains first data;
The Modbus communication module 12 is also provided with a data point combining node, so that data points of adjacent addresses in the point table configuration can be combined into a request, data can be read in batches, and the acquisition efficiency is improved;
specifically, before the node-red-control-modbus node reads the data of the data point, the data point merging node judges whether adjacent addresses exist, because the addresses are sequenced, the judgment is simpler, if the adjacent addresses exist, the starting address and the total length are remembered to initiate a request, the total length is the length of the adjacent addresses, after the data is returned, the values of the respective addresses are obtained according to the corresponding length splitting response result;
for example, two addresses 17404 and 17406 of the type UINT-32 (4 bytes ABCD) of the same device, which are adjacent addresses.
The byte order adjustment module 13 performs byte order conversion on the first data to obtain second data;
specifically, the byte order conversion rule is defined in the point table, and byte order conversion is automatically performed after the data of the corresponding data point is read;
and performing byte order conversion on the first data according to the byte order conversion rule by using the second function node, so as to obtain second data.
The MQTT client module 14 issues the second data to the MQTT server 2;
Specifically, the connection information such as the address, the port, the user name, the password and the like of the MQTT server 2 is pre-configured, and the MQTT theme is configured for issuing data;
converting the second data into a self-defined MQTT message format through a third function node to obtain third data, wherein the third data comprises a device identifier, a time stamp, a version number, load data and the like;
The third data is then published to the MQTT server 2 using the MQTT out node.
The system can configure the point tables in batches, automatically analyze the point table configuration, a user only needs to copy the whole point table into a template node, the system can automatically analyze and generate the required configuration without adding or writing codes one by one, after data are collected from equipment, byte order conversion can be automatically carried out, multiple byte order formats are supported, the system can adapt to the data formats of different equipment, adjustment is not needed at the equipment end, and the user can select the proper byte order format according to the characteristics of the equipment, so that correct analysis of the data is ensured.
In this embodiment, the NodeRed platform 1 is further provided with a data collecting module 15, and the data collecting module 15 reports the second data according to a predefined policy;
the data aggregation module 15 corresponds to a node, which is predefined with a first policy, a second policy and a third policy;
the first strategy is reporting when changing, and is suitable for scenes with infrequent data change and high real-time requirements. For example, the running state (such as running, stopping and alarming) of the equipment is uploaded by using the change, so that the real-time performance is ensured;
The second strategy is periodic reporting, and is suitable for scenes with frequent data changes and history data needs. For example, performance indexes (such as temperature and pressure) of the equipment are uploaded in a use period to generate historical data;
The third strategy is reporting both when changing and periodically, and combines the use of change uploading and periodic uploading to meet different demands. For example, the alarm signal is uploaded by using the change to ensure timely notification, and the state (such as the alarm times and the alarm types) of the alarm system is uploaded by using the period to generate a statistical report;
when running, a user selects one strategy for reporting data;
Collecting the second data according to topic, storing the second data in the context, and reporting the second data by the node according to the selected strategy;
Preferably, the second data is required to meet the output condition when being reported;
At the data aggregation module 15, output conditions may be defined, which may be none, contain a specific field, output a timestamp, or output only a changed portion of data;
When the second data is reported, one or more output conditions are required to be met, if the output conditions are none, the collected second data is output normally, if the output conditions are specific fields, only the second output containing the specific fields is reported, and if the output conditions are output time stamps, the time stamps are output when the second data is reported;
By defining reporting strategies and output conditions, the data transmission efficiency and instantaneity can be optimized according to application requirements, the first strategy can be used for data which is not frequent but needs to be monitored in real time, the second strategy can be selected for data which needs to be recorded regularly, the third strategy can be selected for data which needs to be monitored in real time and needs to be recorded regularly, and the output of the second data can be flexibly controlled by combining the output conditions.
In this embodiment, the method further includes a Redis server 3, a breakpoint continuous storage module 16 is further provided in the noded red platform 1, the breakpoint continuous storage module 16 temporarily stores second data to the Redis server 3, and then issues the second data from the Redis server 3 to the MQTT client module 14;
Specifically, the breakpoint resume module 16 includes a buffered-queue-pub node and a buffered-queue-sub node;
the buffered-queue-pub node issues the reported second data to a queue of the Redis server 3;
subscribing the second data from the queue by the buffered-queue-sub node, and publishing the subscribed data to the MQTT client module 14;
The system further comprises a complex node and a buffered-queue-ack node, wherein the complex node is used for monitoring the completion status issued by the MQTT client module 14, if the MQTT client module 14 issues second data to the MQTT server 2 successfully, the buffered-queue-ack node is used for confirming that the second data is processed, the second data is removed from the queue, and if the MQTT client module 14 issues second data to the MQTT server 2 unsuccessfully, the second data is reserved in the queue;
Based on the problem, the MQTT fails to issue or the network is interrupted, the data can be reserved in the queue and continue to be processed after the network is waited for recovery, the data can be reliably transmitted under the condition that the network is unstable, even if the network is interrupted, the data can be cached in the queue and continue to be transmitted after the network is waited for recovery, and the data cannot be lost.
In one embodiment of the present invention, referring to fig. 2, there is provided a data acquisition method based on NodeRed, including the steps of:
acquiring a point table, and automatically analyzing the point table into a point table configuration according to the point table;
Configuring data of collected data points according to a data point table to form first data;
performing byte order conversion on the first data to obtain second data;
reporting second data to a queue of the Redis server according to a predefined strategy;
subscribing second data from the queue, converting the second data into an MQTT message format, and obtaining third data;
and publishing the third data to the MQTT server.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510747725.5A CN120872436A (en) | 2025-06-05 | 2025-06-05 | NodeRed-based data acquisition system and NodeRed-based data acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510747725.5A CN120872436A (en) | 2025-06-05 | 2025-06-05 | NodeRed-based data acquisition system and NodeRed-based data acquisition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120872436A true CN120872436A (en) | 2025-10-31 |
Family
ID=97451340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510747725.5A Pending CN120872436A (en) | 2025-06-05 | 2025-06-05 | NodeRed-based data acquisition system and NodeRed-based data acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120872436A (en) |
-
2025
- 2025-06-05 CN CN202510747725.5A patent/CN120872436A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7941529B2 (en) | Periodic exporting of information over a flow protocol | |
| US7657624B2 (en) | Network usage management system and method | |
| CN101179452A (en) | Diagnostic method and diagnostic device for fieldbus systems | |
| US20110307219A1 (en) | Method for diagnostic monitoring | |
| CN111142487A (en) | Equipment data acquisition system based on OPC UA unified architecture protocol | |
| CN114430353B (en) | Modbus protocol-based data acquisition method, system, device and medium | |
| CN115865680B (en) | Method, system and device for accessing, controlling and transmitting data of distributed equipment | |
| CN112118174A (en) | Software Defined Data Gateway | |
| CN104219327A (en) | Distributed cache system | |
| CN111242807A (en) | Method for accessing data of transformer substation into ubiquitous power Internet of things | |
| CN102271052A (en) | Network system, network management device and gateway device | |
| CN119743349B (en) | Intelligent gateway system integrating communication protocol and thermal internet of things platform communication | |
| CN116055577A (en) | Protocol conversion method, device, equipment and storage medium | |
| CN115314572B (en) | A protocol conversion method, system and equipment for power communication protocol | |
| CN108873831A (en) | A kind of universal multi-channel workshop data acquisition method | |
| CN118642438A (en) | A data acquisition application method and system based on industrial data acquisition | |
| CN114201537B (en) | Method, system, equipment and storage medium for collecting and storing energy storage data | |
| EP4030737B1 (en) | Iot gateway with configurable data transmission mode | |
| CN120872436A (en) | NodeRed-based data acquisition system and NodeRed-based data acquisition method | |
| CN114417796A (en) | Dynamic report statistical method and system based on equipment sampling points | |
| JP2004062276A (en) | Device information collection / delivery device, method, program, and recording medium recording this program | |
| CN113381907A (en) | Log collection method and device, electronic equipment and storage medium | |
| CN115883639A (en) | Web real-time message pushing method and device, equipment and storage medium | |
| CN114201659A (en) | A message track transmission query method, device and system | |
| CN114817317A (en) | New energy power generation monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |