CN116599962A - A data transmission method and system based on a distributed server - Google Patents

Abstract

The invention discloses a data transmission method and system based on a distributed server. When an application instance needs to perform data transmission, an application identifier is configured for the data, and it is directly written into a local cache queue; the background of the server where the application instance is located The process program will regularly scan the data in the local cache queue, aggregate the scanned data and send it to the registration center; the registration center needs to unpack the data packet after receiving it, and obtain the data transmitted by each application instance according to the application identifier. Compared with the method in the prior art where each application instance needs to create a connection to interact with the registration center, the data transmission method of the present invention performs special management on the connection to the same registration center in one server, and one The application instance on the server will only create one connection with the same registry, so the number of connections to the registry can be reduced, and the number of instances of the registry service and service performance and efficiency can be improved.

Description

A distributed server-based data transmission method and system

technical field

The invention relates to the technical field of data transmission, in particular to a distributed server-based data transmission method and system.

Background technique

At present, in order to meet the requirements of high concurrency and high availability, most systems on the server side basically adopt distributed deployment. After adopting the distributed deployment architecture, more application instances need to be deployed on different machines. The current popular micro-service architecture also deploys many instances on multiple architectures, and under the micro-service architecture, there are A registration center, all microservice instances need to go to this registration center for registration processing. After the registration is successful, each client will establish a connection with the registration center, and transmit heartbeat and monitoring information through this connection, which will be collected by the registration center for subsequent analysis and processing.

As the number of service instances supported by microservices increases, each instance needs to establish a connection with the registration center to transmit heartbeats. After the number of connected microservices reaches a certain level, when a network exception or other abnormal situation occurs, the registration center and each client Information communication will increase with the increase of the number, and the large number of connections and heartbeat transmission will bring a great burden to the registration center.

Contents of the invention

The technical problem to be solved by the present invention is to provide a distributed server-based data transmission method and system, which can reduce the number of connections between instances on the server and the registration center, and increase the number of instances served by the registration center as well as service performance and efficiency.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

A data transmission method based on a distributed server, comprising the steps of:

The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client;

The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

In order to solve the above-mentioned technical problems, another kind of technical scheme that the present invention adopts is:

A data transmission system based on distributed servers, including application instances, servers and registration centers;

The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client;

The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

The beneficial effects of the present invention are: when the application instance needs to transmit data, configure the application identifier for the data, and write it directly into the local cache queue; the background process program of the server where the application instance is located will regularly scan the local cache queue The scanned data is aggregated and sent to the registration center; the registration center needs to unpack the data package after receiving it, and obtain the data transmitted by each application instance according to the application identification. Compared with the method in the prior art where each application instance needs to create a connection to interact with the registration center, the data transmission method of the present invention performs special management on the connection to the same registration center in one server, and one The application instance on the server will only create one connection with the same registry, so the number of connections to the registry can be reduced, and the number of instances of the registry service and service performance and efficiency can be increased.

Description of drawings

Fig. 1 is a flow chart of a data transmission method based on a distributed server according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a distributed server-based data transmission system according to an embodiment of the present invention.

Detailed ways

In order to describe the technical content, achieved goals and effects of the present invention in detail, the following descriptions will be made in conjunction with the embodiments and accompanying drawings.

Please refer to Fig. 1, the embodiment of the present invention provides a kind of data transmission method based on distributed server, comprises steps:

The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client;

The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

It can be seen from the above description that the beneficial effects of the present invention are: when the application instance needs to transmit data, configure the application identifier for the data and write it directly into the local cache queue; the background process program of the server where the application instance is located will regularly Scan the data in the local cache queue, aggregate the scanned data and send it to the registration center; the registration center needs to unpack the data packet after receiving it, and obtain the data transmitted by each application instance according to the application identifier. Compared with the method in the prior art where each application instance needs to create a connection to interact with the registration center, the data transmission method of the present invention performs special management on the connection to the same registration center in one server, and one The application instance on the server will only create one connection with the same registry, so the number of connections to the registry can be reduced, and the number of instances of the registry service and service performance and efficiency can be improved.

Further, it also includes:

When the registration center sends a task to the registration center client, configure the second data to be sent according to the application identifier of the application instance, aggregate the configured second data into a second data packet and send it to the server where the application instance is located;

The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier.

It can be seen from the above description that through the connection between the registration center and the server where the application is located, the tasks to be sent to the application instance can be aggregated and sent to the corresponding server. After the server unpacks the aggregated data packet, it can send the The second data is distributed to the local cache queue of the corresponding registration center client. Ensure that the data transmission between the registration center client and the registration center in the application instance can be realized through a connection between the server where the application instance is located and the registration center. At the same time, the registration center only needs to send a message to the server, which further improves the efficiency of data transmission. efficiency.

Further, the second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes:

The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

It can be seen from the above description that after receiving the task sent by the registration center, the registration center client can actively pull task data from the registration center to ensure the data transmission between the registration center client and the registration center.

Further, the server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, including:

The registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; establish connection.

It can be seen from the above description that the data transmission between the registration center client and the registration center is carried out through the background process program of the server where the application instance is located. The background process program can be created by the registration center client corresponding to the application instance to ensure the connection between the server and the registration center without interruption.

Further, establishing a connection with the registration center by the background process program includes:

When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection.

It can be seen from the above description that if there are multiple registration centers, there will be multiple connections, so the unique identifiers of the registration centers are used to distinguish them to ensure the normal operation of data transmission.

Referring to Fig. 2, another embodiment of the present invention provides a data transmission system based on a distributed server, including an application instance, a server, and a registration center;

The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client;

The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

It can be seen from the above description that the beneficial effects of the present invention are: when the application instance needs to transmit data, configure the application identifier for the data and write it directly into the local cache queue; the background process program of the server where the application instance is located will regularly Scan the data in the local cache queue, aggregate the scanned data and send it to the registration center; the registration center needs to unpack the data packet after receiving it, and obtain the data transmitted by each application instance according to the application identifier. Compared with the method in the prior art where each application instance needs to create a connection to interact with the registration center, the data transmission method of the present invention performs special management on the connection to the same registration center in one server, and one The application instance on the server will only create one connection with the same registry, so the number of connections to the registry can be reduced, and the number of instances of the registry service and service performance and efficiency can be improved.

Further, it also includes:

When the registration center sends a task to the registration center client, configure the second data to be sent according to the application identifier of the application instance, aggregate the configured second data into a second data packet and send it to the server where the application instance is located;

The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier.

It can be seen from the above description that through the connection between the registration center and the server where the application is located, the tasks to be sent to the application instance can be aggregated and sent to the corresponding server. After the server unpacks the aggregated data packet, it can send the The second data is distributed to the local cache queue of the corresponding registration center client. Ensure that the data transmission between the registration center client and the registration center in the application instance can be realized through a connection between the server where the application instance is located and the registration center. At the same time, the registration center only needs to send a message to the server, which further improves the efficiency of data transmission. efficiency.

Further, the second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes:

The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

It can be seen from the above description that after receiving the task sent by the registration center, the registration center client can actively pull task data from the registration center to ensure the data transmission between the registration center client and the registration center.

Further, the server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, including:

The registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; establish connection.

It can be seen from the above description that the data transmission between the registration center client and the registration center is carried out through the background process program of the server where the application instance is located. The background process program can be created by the registration center client corresponding to the application instance to ensure the connection between the server and the registration center without interruption.

Further, establishing a connection with the registration center by the background process program includes:

When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection.

It can be seen from the above description that if there are multiple registration centers, there will be multiple connections, so the unique identifiers of the registration centers are used to distinguish them to ensure the normal operation of data transmission.

The above-mentioned distributed server-based data transmission method and system of the present invention are suitable for micro-service architecture, reducing the number of connections between instances on the server and the registration center, and increasing the number of instances served by the registration center as well as service performance and efficiency, as follows To illustrate through specific implementation methods:

Embodiment one

Please refer to Figure 1, a distributed server-based data transmission method, including steps:

S1. The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client.

In this embodiment, it is assumed that a registration center already exists in the microservice architecture, and an application instance is deployed on a server. At this time, the application instance integrates the registration center client, and a proxy layer is added to the registration center client.

The registration center client on the server interacts with the background process program through the local cache queue. When the registration center client needs to send data, such as heartbeat information, it directly writes the heartbeat message to the local cache queue. The written content mainly includes the unique application identifier and the message type [heartbeat type]. The background process program will regularly scan the information in the local cache queue. After reading, it will aggregate related heartbeat packet messages at intervals, and send an aggregated heartbeat packet.

In this embodiment, heartbeat information is processed and sent, and other data such as monitoring data can also be transmitted.

S2. The server where the application instance is located regularly scans the data in the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center .

Specifically, the registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; Establish a connection with the registry.

In this embodiment, it is judged whether there is an independent background process program on the server through the proxy layer of the registration center client, and the program has started; if it does not exist, the background process program is actively created by the proxy layer, and the program Create a connection with the registry; if it exists, use the connection established by the program with the registry directly. Through this step, it has been ensured that only one connection on a server communicates with the same registry.

Wherein, when there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection.

In this embodiment, if there are multiple registration centers, there will be multiple connections, which are distinguished by unique identifiers of the registration centers.

S3. The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

In this embodiment, after receiving the heartbeat packet, the registration center needs to unpack the heartbeat packet, and obtain corresponding heartbeat messages according to unique identifiers of different applications, and perform subsequent processing.

Embodiment two

The difference between this embodiment and Embodiment 1 is that the data transmission method when the registration center needs to notify the relevant application instance to perform a task is further limited:

When the registration center sends a task to the registration center client, it configures the second data to be sent according to the application identifier of the application instance, aggregates the configured second data into a second data packet, and sends it to the server where the application instance is located.

The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier.

The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

Specifically, when the registration center needs to notify the relevant registration center client to perform a task, it can aggregate information in batches, send the aggregated message package to the registration center client, unpack it by the background process program, and send it to the local cache queue , read by the corresponding registry client client.

For example, the registration center may send a message requesting the client to reacquire the latest service list information from the registration center. At this time, it is only necessary to send a notification message. After receiving it, the client will actively pull the service list information from the registration center. At this time, the registration center only needs to send a copy of the latest service list information to the connection.

Embodiment three

Please refer to Figure 2, a data transmission system based on distributed servers, including application instances, servers and registration centers;

The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client;

The registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; Establishing a connection; wherein, when there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection;

The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center;

The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier.

In some embodiments, also include:

When the registration center sends a task to the registration center client, configure the second data to be sent according to the application identifier of the application instance, aggregate the configured second data into a second data packet and send it to the server where the application instance is located;

The server unpacks the second data packet through a corresponding background process program, and sends the second data obtained by unpacking to the local cache queue of the corresponding registration center client according to the application identifier;

The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data.

In summary, the present invention provides a distributed server-based data transmission method and system. When an application instance needs to perform data transmission, an application identifier is configured for the data, and it is directly written into the local cache queue; the application instance The background process program of the server where it is located will regularly scan the data in the local cache queue, aggregate the scanned data and send it to the registration center; the registration center needs to unpack the data packet after receiving it, and obtain the transmission of each application instance according to the application identification The data. Similarly, when the registration center needs to notify the relevant clients to perform tasks, it can also use the method of batch aggregation information to send the aggregated message package to the client. After the background process program unpacks it, it sends it to the local cache queue. The corresponding client to read. Compared with the method in the prior art where each application instance needs to create a connection to interact with the registration center, the data transmission method of the present invention performs special management on the connection to the same registration center in one server, and one The application instance on the server will only create one connection with the same registry, so the number of connections to the registry can be reduced, and the number of instances of the registry service and service performance and efficiency can be increased.

The above description is only an embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent transformations made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in related technical fields, are all included in the same principle. Within the scope of patent protection of the present invention.

Claims (10)

1. A data transmission method based on a distributed server, characterized in that, comprising steps: The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client; The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center; The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier. 2. A kind of data transmission method based on distributed server according to claim 1, is characterized in that, also comprises: When the registration center sends a task to the registration center client, configure the second data to be sent according to the application identifier of the application instance, aggregate the configured second data into a second data packet and send it to the server where the application instance is located; The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier. 3. A kind of data transmission method based on distributed server according to claim 2, it is characterized in that, the second data obtained by unpacking is sent to the local cache queue of corresponding registration center client, afterwards comprising: The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data. 4. The data transmission method based on a distributed server according to claim 1, wherein the server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, including : The registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; establish connection. 5. a kind of data transmission method based on distributed server according to claim 4, is characterized in that, setting up connection with registration center by described background process program comprises: When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection. 6. A data transmission system based on a distributed server, comprising an application instance, a server and a registration center; The registry client of the application instance obtains the first data to be transmitted, configures the corresponding application identifier for the first data to obtain the data to be transmitted, and writes the data to be transmitted into the local cache queue of the registry client; The server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, aggregates the scanned data into a first data packet, and sends the first data packet to the registration center; The registration center unpacks the first data packet, and obtains the first data transmitted by each application instance according to the application identifier. 7. A kind of data transmission system based on distributed server according to claim 6, is characterized in that, also comprises: When the registration center sends a task to the registration center client, configure the second data to be sent according to the application identifier of the application instance, aggregate the configured second data into a second data packet and send it to the server where the application instance is located; The server unpacks the second data packet through a corresponding background process program, and sends the unpacked second data to the local cache queue of the corresponding registration center client according to the application identifier. 8. A kind of data transmission system based on distributed server according to claim 7, it is characterized in that, the second data obtained by unpacking is sent to the local cache queue of the corresponding registration center client, and then includes: The registration center client obtains the second data from the local cache queue, and pulls task data from the registration center according to the second data. 9. A distributed server-based data transmission system according to claim 6, wherein the server where the application instance is located regularly scans the data of the local cache queue of the registration center client through the corresponding background process program, including : The registration center client judges whether there is a corresponding background process program on the server where the application instance is located, and if so, directly establishes a connection between the background process program and the registration center; establish connection. 10. A kind of data transmission system based on distributed server according to claim 9, is characterized in that, setting up connection with registration center by described background process program comprises: When there are multiple registration centers, the background process program establishes a connection with the registration center, and configures the unique identifier of the registration center for the connection.