CN109271259B - Enterprise service bus system, data processing method, terminal and storage medium - Google Patents

Abstract

An enterprise service bus system for communicatively coupling an enterprise system to a third party system, comprising: ESB-API, KAFKA, ESB-CONSUMER, ESB-PROXY-IN, ESB-PROXY-OUT and ESB-FILE-PROXY-OUT subsystems, wherein KAFKA is IN communication connection with ESB-API and ESB-CONSUMER, ESB-API is IN communication connection with ESB-PROXY-IN, ESB-CONSUMER is IN communication connection with ESB-PROXY-OUT and ESB-FILE-PROXY-OUT subsystems. The invention also provides an enterprise service bus data processing method, a terminal and a storage medium. The invention can convert the data requests sent by different requesters into uniform messages and request protocols, realizes the function of a synchronous-to-asynchronous processing mechanism and improves the data processing speed.

Description

Enterprise service bus system, data processing method, terminal and storage medium

technical field

The invention relates to the field of computer technology, in particular to a heavy-mode enterprise service bus system, a heavy-mode enterprise service bus data processing method, a terminal and a storage medium.

Background technique

With the development of computer information system, the information system is becoming larger and more complex. When there are many connection objects, the point-to-point connection method has high cost and low availability and maintainability. Therefore, the concept of bus is introduced into the architecture construction of information system. Following the concept of Service oriented architecture (SOA), the bus of an information system is usually called a service bus. The bus of its strategic layer is called enterprise service bus (Enterprise Service Bus, ESB). Enterprise service bus is an enterprise-level information system basic platform with standard interfaces, realizing interconnection, communication, service routing, and supporting SOA.

In some enterprises that choose the heavy model (self-operated model), because there are enough intermediate links involved and strong control, but if you want to do a good job in each link, the operating cost is very high, so it is necessary to establish an enterprise The architecture of the service bus reduces the operating cost and management cost of the enterprise.

SUMMARY OF THE INVENTION

In view of the above, it is necessary to propose a heavy-mode enterprise service bus system, a heavy-mode enterprise service bus data processing method, a terminal and a storage medium, which can convert data requests sent by requesters of different data requests into unified messages and Request protocol, and realize the function of converting synchronous processing mechanism into asynchronous processing mechanism.

A first aspect of the present invention provides an enterprise service bus system, the enterprise service bus system communicates and connects multiple enterprise systems and multiple third-party systems, and the enterprise service bus system includes: ESB-API subsystem, KAFKA subsystem , ESB-CONSUMER subsystem, ESB-PROXY-IN subsystem, ESB-PROXY-OUT subsystem, ESB-FILE-PROXY-OUT subsystem, the KAFKA subsystem and the ESB-API subsystem and the The ESB-CONSUMER subsystem is communicatively connected, the ESB-API subsystem is communicatively connected to the ESB-PROXY-IN subsystem, and the ESB-CONSUMER subsystem is respectively connected to the ESB-PROXY-OUT subsystem and the ESB -FILE-PROXY-OUT subsystem communication connection, where,

The ESB-API subsystem is used to publish the dubbo service agreement, and the ESB-PROXY-IN subsystem is used to publish the http service agreement and the https service agreement;

The ESB-API subsystem is further configured to send the message encapsulated by the enterprise service bus system to the KAFKA subsystem by a message producer in the ESB-API subsystem;

The KAFKA subsystem is configured to configure a corresponding message queue according to the to-be-processed message when receiving the to-be-processed message sent by the ESB-API subsystem, and to process the to-be-processed message;

The ESB-CONSUMER subsystem is used to pull the message to be processed from the message queue of the KAFKA subsystem every preset time period, and create a corresponding thread according to the message to be processed;

The ESB-CONSUMER subsystem is used to parse the message to be processed, and send the parsed content to the server corresponding to the data request;

The ESB-CONSUMER subsystem is also used to obtain the request result sent by the server of the data request and temporarily store the request result in the message queue of the KAFKA subsystem;

The ESB-API subsystem is further configured to obtain the request result from the message queue of the KAFKA subsystem, and send the request result to the requester of the data request.

A second aspect of the present invention provides an enterprise service bus data processing method using the enterprise service bus system, the method comprising:

Publishing service for the requester to call, the ESB-API subsystem publishing service protocol is the dubbo protocol, and the ESB-PROXY-IN subsystem publishing service protocol is the http protocol and the https protocol;

Detect whether a data request from the requester is received;

After detecting the data request received by the requester, identify whether the requester of the data request is legal;

When identifying the requesting party of the data request is legitimate, identifying the serving party of the data request according to the data request;

According to the requesting server, the data request is encapsulated into a corresponding message, and the encapsulated message is sent to the KAFKA subsystem by the message producer in the ESB-API subsystem;

When the KAFKA subsystem receives the to-be-processed message sent by the ESB-API subsystem, it configures a corresponding message queue according to the to-be-processed message, and processes the to-be-processed message;

Every preset time period, the ESB-CONSUMER subsystem pulls the message to be processed from the message queue of the KAFKA subsystem, and creates a corresponding thread according to the message to be processed;

The ESB-CONSUMER subsystem parses the message to be processed, and sends the parsed content to the server corresponding to the data request;

The ESB-CONSUMER subsystem obtains the request result sent by the server of the data request and temporarily stores the request result in the message queue of the KAFKA subsystem;

The ESB-API subsystem obtains the request result from the message queue of the KAFKA subsystem, and sends the request result to the requester of the data request.

Preferably, the data request of the requesting party is a data request that the requesting party encapsulates the data request according to the service protocol published by the enterprise service system, including:

When the requester is the enterprise system, encapsulate the data request according to the dubbo protocol;

When the requester is a third-party system, the data request is encapsulated according to the http protocol or the https protocol.

Preferably, the identifying the service party of the data request according to the data request includes:

Identify the service party of the data request according to the data request service party identifier carried in the data request;

When the identification of the data requesting service party is identified as the first identification, it is determined that the data requesting service party is an enterprise system;

When the identification of the data requesting service party is identified as the second identification, it is determined that the data requesting service party is a third-party system.

Preferably, the service party according to the request encapsulates the data request into a corresponding message, and sends the encapsulated message to the KAFKA sub-system by a message producer in the ESB-API subsystem The system includes:

When the receiving system of the data request is the ESB-API subsystem, the data request is encapsulated into a message, and the encapsulated message is sent by the message producer in the ESB-API subsystem to the KAFKA subsystem;

When the receiving system of the data request is the ESB-PROXY-IN subsystem, the data request is encapsulated into a message and forwarded to the ESB-API subsystem through the ESB-PROXY-IN subsystem , the message producer in the ESB-API subsystem sends the message to the KAFKA subsystem.

Preferably, the creating a corresponding thread according to the to-be-processed message includes: creating a thread pool of a corresponding size according to the concurrent request number corresponding to the server, where the concurrent request number is equal to the size of the thread pool.

Preferably, the sending the parsed content to the service party corresponding to the data request includes: when the service party of the data request is the third-party system, using the ESB-PROXY-OUT subsystem or the third-party system The ESB-FILE-PROXY-OUT subsystem sends the parsed content to the service party of the data request; when the service party of the data request is the enterprise system, it directly sends the parsed content to the service party of the data request. Describe the enterprise system.

Preferably, the sending the request result to the requester of the data request includes: when the requester of the data request is the enterprise system, the ESB-API subsystem directly sends the request result to the enterprise system; when the requester of the data request is the third-party system, the ESB-API subsystem passes the ESB-PROXY-OUT subsystem or the ESB-FILE-PROXY-OUT sub-system The system sends the request result to the third-party system.

Preferably, the method further includes:

Determine whether the current request status satisfies the fuse condition;

When the current request status satisfies the fuse condition, configure the first number of message queues, and return preset information to the requester of the currently received data request;

When the current request state does not meet the circuit breaker condition, configure a second number of message queues.

Preferably, the fusing conditions include a combination of one or more of the following:

The elapsed time of the current data request exceeds the preset elapsed time; or

The average time-consuming time of all data requests within the preset time period is greater than or equal to the interface timeout time; or

The number of messages in the ESB-API subsystem message queue exceeds a preset number; or

The number of data requests of the requester who received the same data request exceeds the preset number.

A third aspect of the present invention provides a terminal, the terminal includes a processor and a memory, and the processor is configured to implement the enterprise service bus data processing method when executing a computer program stored in the memory.

A fourth aspect of the present invention provides a computer-readable storage medium, where a computer program is stored thereon, and when the computer program is executed by a processor, the enterprise service bus data processing method is implemented.

To sum up, the heavy-mode enterprise service bus system, data processing method, terminal, and storage medium described in the embodiments of the present invention adopt the synchronous processing mechanism of the open source framework dubbo, combined with the asynchronous processing mechanism of the open source processing stream platform KAFKA, The synchronous processing mechanism is converted into an asynchronous processing mechanism, so that ESB not only has the characteristics of dubbo's load balancing, but also has the buffering function of KAFKA's message queue, which can achieve dynamic expansion and message persistence: messages are persisted to the local disk, And support data backup to prevent data loss. Secondly, set a unified message format, by converting the data requests sent by the requesters of different data requests into a unified message and request protocol, it can shield the underlying technical differences of the heterogeneous systems, and also facilitate the requester of the data request. development and management.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.

FIG. 1 is an operating environment diagram of a heavy-mode enterprise service bus system provided by Embodiment 1 of the present invention.

FIG. 2 is a flowchart of a data processing method for a heavy-mode enterprise service bus provided by Embodiment 2 of the present invention.

FIG. 3 is a schematic diagram of a terminal according to Embodiment 3 of the present invention.

The following specific embodiments will further illustrate the present invention in conjunction with the above drawings.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments may be combined with each other under the condition of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

Example 1

FIG. 1 is an operating environment diagram of a heavy-mode enterprise service bus system provided by Embodiment 1 of the present invention.

In this embodiment, the enterprise system 200 performs communication connection and data interaction with the third-party system 300 through a heavy-mode enterprise service bus system 100 (Enterprise Service Bus, ESB, hereinafter referred to as ESB 100 ). The third-party system 300 performs communication connection and data exchange with the enterprise system 200 through the ESB 100 .

The enterprise system 200 may include multiple enterprise internal applications or enterprise internal file services, for example, enterprise application 1, enterprise application 2 and enterprise application 3, and different enterprise internal applications or enterprise internal file services may be configured on different servers.

The third-party system 300 may include multiple third-party applications or third-party file services, for example, third-party application 1, third-party application 2, third-party application 3, and third-party file services, different third-party applications or third-party files. Services can be configured on different servers.

The ESB 100 may include multiple subsystems, for example: ESB-API subsystem (hereinafter referred to as ESB-API) 101, KAFKA subsystem (hereinafter referred to as KAFKA) 102, ESB-CONSUMER subsystem (hereinafter referred to as ESB-CONSUMER) 103. ESB-PROXY-IN subsystem (hereinafter referred to as ESB-PROXY-IN) 104, ESB-PROXY-OUT subsystem (hereinafter referred to as ESB-PROXY-OUT) 105, ESB-FILE-PROXY-OUT subsystem ( Hereinafter referred to simply as ESB-FILE-PROXY-OUT) 106 . Among them, KAFKA102 communicates with ESB-API101 and ESB-CONSUMER103 respectively, ESB-API101 communicates with ESB-PROXY-IN104, and ESB-CONSUMER103 communicates with ESB-PROXY-OUT105 and ESB-FILE-PROXY-OUT106 respectively. ESB-PROXY-IN104 can be called access proxy server. ESB-PROXY-OUT105 and ESB-FILE-PROXY-OUT106 can be called outgoing proxy servers.

In other embodiments, the ESB 100 may further include a ZOOKEEPER subsystem 107 and a NAS subsystem 108 .

In this embodiment, each subsystem (eg, ESB-API10, ESB-PROXY-IN104, ESB-PROXY-OUT105, ESB-FILE-PROXY-OUT106) in the ESB 100 is used for publishing services. The publishing service is a publishing service protocol, and the service protocol may include: a dubbo protocol, an http protocol, and an https protocol.

In this embodiment, the enterprise system 200 , the ESB 100 , and the third-party system 300 may be a requester or a service party. The ESB 100 will establish unique identifiers and request concurrency numbers for different enterprise systems 200 and different third-party systems 300 , for example, create an identifier "B10047" for a certain bank, and the request concurrency number is 100.

In this embodiment, the ESB-API101 publishes the service protocol as the dubbo protocol, the service name format is "esb-api.service.XXXXXX_YYYYYY", and the "XXXXXX" in the interface name is the service party (the enterprise system 200 or The identifier of the third-party system 300), "YYYYYY" in the interface name is the service name provided by the service party, and the ESB-API 101 publishes the service for the enterprise system 200 and the ESB-PROXY-IN 104 to call.

In this embodiment, the ESB-PROXY-IN104 publishing service protocol is http protocol and https protocol, the service name format is "esb.YYYYYY", and "YYYYYY" in the interface name is the service party (the enterprise The service name provided by the system 200 or the third-party system 300), the ESB-PROXY-IN104 publishes the service for the third-party system 300 to call. When the ESB-PROXY-IN104 publishes a service, a service correspondence table will be maintained to correspond the service published by the ESB-PROXY-IN104 with the service published by the ESB-API101, for example, the ESB-PROXY-IN101 The published esb.customer.queryReferrerList service corresponds to the esb-api.service.Fin004.ff.customer.queryReferrerList service published by the ESB-API101.

In this embodiment, the ESB-PROXY-OUT105 and ESB-FILE-PROXY-OUT106 publish service protocols are dubbo protocols, the service name format is "esb-proxy.service.XXXXXX_YYYYYY", and "XXXXXX" in the interface name is the identification of the service party (the enterprise system 200 or the third-party system 300), "YYYYYY" in the interface name is the service name provided by the service party, the ESB-PROXY-OUT105, ESB-FILE - PROXY-OUT 106 publishes services for the ESB-API 101 to call. The services published by the ESB-PROXY-OUT105 and ESB-FILE-PROXY-OUT106 appear in pairs with the services published by the ESB-API101, for example, esb-proxy.service.B10043.QueryGradingInterest published by the ESB-PROXY-OUT105

The RateService service is paired with the esb-api.service.B10043.QueryGrad-ingInterestRateService service published by the ESB-API101.

It should be noted that the ESB100 as a requester is mainly relative to each subsystem within the ESB100, because there is also a calling relationship between the subsystems within the ESB100.

For example: ESB-PROXY-IN104 of ESB100 calls other subsystems of ESB100, and ESB-PROXY-IN104 of ESB100 also calls ESB-API 101 of ESB100. At this time, ESB-PROXY-IN104 is the requester relative to ESB-API101 .

The ESB 100 is also used to detect whether a data request from a requester is received.

The requester's data request is the data request after the requester encapsulates the data request according to the service published by the ESB100.

In this embodiment, the encapsulation of the data request by the requester according to the published service may include:

1) When the requester is the enterprise system 200, the data request is encapsulated according to the dubbo protocol issued by each subsystem of the ESB 100.

2) When the requesting party is the third-party system 300, the data request is encapsulated according to the http protocol or https protocol issued by each subsystem of the ESB 100.

The requester encapsulates the data request according to the services published by the various subsystems of the ESB100. The requester only needs to encapsulate the data request according to the unified request message format of the ESB100, and does not need to pay attention to the request message formats of different service parties. different question.

For example, the format of the unified request message provided by this embodiment is as follows:

The ESB 100 is further configured to identify whether the requester of the data request is legal after detecting that the data request of the requester is received.

In this embodiment, when the ESB 100 receives the data request, it identifies whether the requester of the data request is legal according to the data request. The legality refers to whether the requesting party is a legal enterprise system 200 or a third-party system 300 recognized by ESB 100, and the enterprise system 200 and the third-party system 300 refer to the systems that ESB 100 is docked with and has established a unique identifier for. .

In this embodiment, the data request may carry a data request source identifier, and the data request source identifier is used to indicate the identity of the requester of the data request. The source identifier is the unique identifier established by ESB100 for the system.

The ESB 100 is further configured to interrupt the request when identifying that the requester of the data request is illegal.

The ESB 100 is further configured to, when identifying the requesting party of the data request, identify the serving party of the data request according to the data request.

In this embodiment, when the ESB determines that the requesting party of the data request is a legitimate requesting party, it identifies the serving party of the data request according to the data request, and the serving party may be the enterprise system 200 or the third-party system 300 .

In this embodiment, the data request may carry a data request service party identifier, and the data request service party identifier is used to indicate the identity of the data request service party. The identification of the data requesting service party includes: a first identification and a second identification, wherein the first identification is used to indicate that the data requesting service party is an intra-enterprise application or an intra-enterprise file service, and the second identification is used to indicate that The data request service party is a third-party application or a third-party file service.

Identifying the service side of the data request according to the data request may include: identifying the service side of the data request according to the data request service side identifier carried in the data request; when identifying the data request service side identifier as the first When identifying the data requesting service party, it is determined that the data requesting service party is an enterprise system; when the data requesting service party is identified as the second identification, it is determined that the data requesting service party is a third-party system.

For example, the identifier of the data requesting service party in this embodiment may be the "XXXXXX" part of the service name published by the ESB-API101, ESB-PROXY-OUT105, and ESB-FILE-PROXY-OUT106.

The ESB 100 is further configured to encapsulate the data request into a corresponding message according to the service side of the request, and send the encapsulated message to the KAFKA 102 by the message producer in the ESB-API 101 .

In this embodiment, the ESB-API 101 is a function predefined by an application programming interface (Application Programming Interface, API), and is a service entry of the ESB 100 and a message producer.

In this embodiment, encapsulating the data request into a corresponding message according to the requesting server, and sending the encapsulated message to the KAFKA 102 by the message producer in the ESB-API 101 may include:

1) When the receiving system of the data request is the ESB-API 101, the data request is encapsulated into a message, and the encapsulated message is sent to the KAFKA 102 by the message producer in the ESB-API 101.

2) When the receiving system of the data request is ESB-PROXY-IN104, encapsulate the data request as a message, and after forwarding it to ESB-API101 through the first proxy server, the message producer in ESB-AP101I sends the message to KAFKA102.

After the message is encapsulated, it is directly sent by the message producer in ESB-API101 to the message queue of KAFKA102, and the message carries an identification number to indicate the location information of the stored message queue. For example, when the identification number is A, it indicates that the message is stored in message queue 1, and when the identification number is B, it indicates that the message is stored in message queue 2. The message queue may be a message queue of KAFKA 102 or a message queue in memory.

The ESB-PROXY-IN 104 may also include a load balancing function (for example, F5 shown in the figure) to perform load balancing control on the data request of the third-party system 300 . Prevents paralysis due to stress on enterprise systems (eg, banking systems) when high concurrency occurs.

When the KAFKA 102 receives a message to be processed sent by the ESB-API 101, configure a corresponding message queue according to the message to be processed, and process the message to be processed.

In this embodiment, KAFKA102 is a high-throughput distributed publish-subscribe messaging system that provides a message queue for ESB100 and can process all action flow data in consumer-scale websites. KAFKA102 has high throughput, supports load balancing, Features of asynchronous sending and message persistence. When KAFKA 102 detects a message sent by ESB-API 101, it acquires the message to be processed, configures a corresponding message queue, and processes the message to be processed.

The processing of the messages to be processed may include: distributed storage of the messages to be processed into multiple copies; when the number of the messages to be processed exceeds a preset number threshold, each of the messages to be processed is stored. Messages are sliced or diced; messages to be processed are compressed. This ensures that pending messages are not lost and re-consumed.

In this embodiment, ESB100 uses KAFKA102 as a message queue, which has the functions of high throughput and message persistence, which can well improve the processing capability of ESB100, making the processing capability of ESB100 controllable. When the service performance of ESB100 is low , you can use the message queue to buffer the information to avoid crashes.

The ESB-CONSUMER103 is configured to pull the to-be-processed message from the message queue of the KAFKA 102 every preset time period, and create a corresponding thread according to the to-be-processed message.

ESB-CONSUMER103 can monitor whether there are messages to be processed in KAFKA102 every preset time period (for example, 200 milliseconds). When ESB-CONSUMER103 detects that there are messages to be processed in KAFKA102, it pulls the messages from KAFKA102, and according to the message content Route to the corresponding proxy server.

ESB100 will create corresponding thread pools based on message queues, one message queue corresponds to one thread pool, each thread pool consumes the pending messages in the corresponding message queue, and different thread pools do not interfere with each other. For example, thread pool 1 is only responsible for consuming pending messages in message queue 1, and thread pool 2 is only responsible for consuming pending messages in message queue 2.

In this embodiment, the creating a corresponding thread according to the to-be-processed message may include: creating a thread pool of a corresponding size according to the concurrent number of requests corresponding to the server. The number of concurrent requests is equal to the size of the thread pool.

Thread pools of different sizes are created according to the number of concurrent requests pre-configured by the server, so as to implement different concurrency controls according to different data request volumes. For example, the data request volume of an enterprise system is generally small, and the server (Third-party system 300) The pre-configured number of concurrent requests is correspondingly low, so the thread pool created by ESB-CONSUMER103 is also small, for example, 20-50; for another example, the data request volume of the third-party system is generally large , the number of concurrent requests preconfigured by the server (enterprise system 200 ) is correspondingly high, so the thread pool created by ESB-CONSUMER103 is relatively large, for example, 100-150.

ESB-CONSUMER103 parses the to-be-processed message, and sends the parsed content to the server corresponding to the data request.

In this embodiment, the thread of the ESB-CONSUMER 103 may be pre-configured with a configuration table corresponding to the message to be processed. The configuration table includes: the system service name required to complete the data request, the system service physical route, the system service follow-up process condition code, the condition code corresponding to the follow-up service and other information. Parsing the to-be-processed message is to retrieve the configuration table and interpret the content to determine the specific service content, and to determine the system service name, system service physical route, subsequent process condition codes, and subsequent services corresponding to the condition codes.

In this embodiment, the service party refers to a service provider that responds to the data request of the requester and provides corresponding request results according to the data request. The service party may include: the enterprise system 200 or the third-party system 300 . Of course, if the enterprise system 200 includes a plurality of enterprise subsystems, for example, enterprise subsystem 1, enterprise subsystem 2. The service party may also be one enterprise subsystem among multiple enterprise subsystems. Similarly, if the third-party system 300 includes multiple third-party subsystems, for example, third-party subsystem 1 and third-party subsystem 2 . The serving party may also be a third-party subsystem among multiple third-party subsystems. That is, the requester and the service party can be both the enterprise system 200 and the third-party system 300 at the same time. For example, when the enterprise subsystem 1 requests to call the enterprise subsystem 2, at this time, the enterprise subsystem 1 is the requester and the enterprise Subsystem 2 is the server.

For example, the service provider provided in this embodiment responds to the data request, that is, the format of the response message is as follows:

In this embodiment, sending the parsed content to the server corresponding to the data request may include:

1) When the service party of the data request is the third-party system 300, the parsed content is sent to the service party (third-party system) of the data request through the outgoing proxy server.

When the outgoing proxy server receives the parsed content sent by the thread of the ESB-CONSUMER 103 , it forwards the parsed content to the third-party system 300 . The ESB-PROXY-OUT105 is used to receive the message of the ESB-CONSUMER103, and according to the message, request the third-party system to respond to the data request and return the request result according to the message according to the request mode preset by the third-party system. The ESB-FILE-PROXY-OUT106 is used to receive the message of the ESB-CONSUMER103, and request the third-party system 300 according to the message according to the request mode preset by the third-party system 300, upload or download the corresponding file in the data request .

2) When the server of the data request is the enterprise system 200, the parsed content is directly sent to the enterprise system 200.

ESB-CONSUMER103 obtains the request result sent by the server of the data request and temporarily stores the request result in the message queue of KAFKA102.

In this embodiment, the request result sent by the service party that obtains the data request may include:

1) When the service party of the data request is the third-party system 300, obtain the request result from the third-party system 300 through the outgoing proxy server, and temporarily store the request result in the message of KAFKA102 in the queue.

2) When the server of the data request is the enterprise system 200, directly obtain the request result from the enterprise system, and temporarily store the request result in the message queue of the KAFKA 102.

The ESB-API 101 obtains the request result from the message queue of the KAFKA 102, and sends the request result to the requester of the data request.

In this embodiment, sending the request result to the requester of the data request may include:

1) When the requester of the data request is the enterprise system 200, the ESB-API 101 directly sends the request result to the enterprise system 200.

2) When the requester of the data request is the third-party system 300, the ESB-API 101 sends the request result to the third-party system 300 through the access proxy server.

It should be noted that both the enterprise system 200 and the third-party system 300 can be both the requester of the data request and the server of the data request that responds to the data request and provides the request result. That is, when the enterprise system 200, as the requester of the data request, sends the data request to the server of the data request (the third-party system 300), it can also act as the server of the data request to respond to the data request and be the requester of the data request (the third-party system 300). The third party system 300) provides the request result. As the requester of the data request, the third-party system can also respond to the data request as the servicer of the data request and be the requester of the data request (enterprise system 200) while sending the data request to the servicer (enterprise system 200) of the data request. ) provides the request result.

Further, when ESB100 is also used to receive a data request, determine whether the current request state satisfies the fuse condition; when the current request state satisfies the fuse condition, configure the first number of message queues, and send the request to the currently received data request. to return to the default information. When the current request state does not meet the circuit breaker condition, configure a second number of message queues.

ESB100 can count the time-consuming time of each data request. The time-consuming time of data request refers to the time from ESB100 receiving the data request sent by the requester of the data request to returning the request corresponding to the data request to the requester of the data request. result time.

The current request state may include one or more of the following combinations: the time-consuming time of the current data request; the average value of the time-consuming time of all data requests within a preset time period; the messages in the ESB-API message queue Quantity; the number of data requests from the requester that received the same data request.

The fusing condition may include one or more of the following combinations: the time-consuming time of the current data request exceeds the preset time-consuming time; the average time-consuming time of all data requests within the preset time period is greater than or equal to the interface Exceeded time; the number of messages in the ESB-API101 message queue exceeds the preset number; the number of data requests from the requester receiving the same data request exceeds the preset number.

The preset time-consuming time is a predefined time value. The preset information may be "the system is busy with business, please try again later". The first quantity value is equal to the size value of the thread pool, and the second quantity value is equal to the size value of the thread pool multiplied by the ratio of the interface timeout value and the average time-consuming time value.

In this embodiment, by presetting the fusing condition, when it is detected that the fusing condition is satisfied, the instantaneous high concurrency can exceed the processing capability of the ESB100, for example, when the number of messages in the ESB-API101 message queue exceeds the preset number, KAFKA102 and ESB-CONSUMER103 are too late to process data requests, or when a large number of third-party systems 300 are abnormal or the request times out, they can automatically fuse the current data request, increase the waiting time of messages in the message queue, and buffer messages. , reducing the pressure on the ESB100, and at the same time returning preset information to the requester of the current data request in time, achieving a timely response and improving the experience of the requester of the data request.

Further, the ESB 100 can also be used to configure a visual management tool to store messages and respond to data requests. The visual management tools may include: mongo dB, mongo booster, and admin Mongo.

Further, the ESB 100 can also be used to provide distributed application coordination services, and to provide a registry for KAFKA and dubbo. The distributed application coordination service may be a ZooKeeper component.

To sum up, the heavy-mode enterprise service bus system described in the embodiment of the present invention adopts the synchronous processing mechanism of the open source framework dubbo, and combines the asynchronous processing mechanism of the open source processing stream platform KAFKA to convert the synchronous processing mechanism into an asynchronous processing mechanism , so that ESB not only has the characteristics of dubbo's load balancing, but also has the buffering function of KAFKA's message queue, which can achieve dynamic expansion and message persistence: messages are persisted to local disks, and data backup is supported to prevent data loss. Secondly, set a unified message format, by converting the data requests sent by the requesters of different data requests into a unified message and request protocol, it can shield the underlying technical differences of the heterogeneous systems, and also facilitate the requester of the data request. development and management. In addition, the corresponding threads are created according to the pending messages in the message queue, and the size of the thread pool is configured, which effectively prevents the pressure caused by the data request server when high concurrency occurs, which leads to the paralysis of the data request server system. Moreover, producers and consumers are deployed independently, and a producer can configure multiple consumers to consume messages to be processed. Finally, set the fuse condition, when the instantaneous high concurrency exceeds the ESB processing capacity or when a large number of exceptions in the third-party system or the request times out, the current data request can be automatically blown and controlled, and the waiting time of the message in the message queue can be increased. The function of buffering messages reduces the pressure on the ESB, and at the same time, it can return preset information to the requester of the current data request in time, so as to achieve a timely response and improve the experience of the requester of the data request.

Embodiment 2

FIG. 2 is a flowchart of a data processing method for an enterprise service bus provided by Embodiment 2 of the present invention. According to different requirements, the execution order in this flowchart can be changed, and some steps can be omitted.

The data processing method of the enterprise service bus according to the embodiment of the present invention may be applied to one or more terminals. The data processing method of the enterprise service bus can also be applied to a hardware environment composed of a terminal and a server connected to the terminal through a network. The network includes, but is not limited to: a wide area network, a metropolitan area network or a local area network. The data processing method of the enterprise service bus according to the embodiment of the present invention may be executed by a server or a terminal; and may also be executed jointly by a server and a terminal.

For the terminal that needs to perform the data processing method of the enterprise service bus, the data processing function of the enterprise service bus provided by the method of the present invention may be directly integrated on the terminal, or a client for implementing the method of the present invention may be installed. For another example, the method provided by the present invention can also be run on a server and other equipment in the form of a software development kit (Software Development Kit, SDK), and the interface of the enterprise service bus architecture function is provided in the form of an SDK. The interface can realize the data processing of the enterprise service bus.

S11. Publish the service for the requester to call.

In this embodiment, the service can be published by each subsystem in the ESB, and each subsystem in the ESB includes: ESB-API, ESB-PROXY-IN, ESB-PROXY-OUT, ESB-FILE-PROXY-OUT, and publishes the service In order to publish the service protocol, the service protocol may include: dubbo protocol, http protocol, https protocol.

In this embodiment, the requesting party may be an enterprise system, an ESB, or a third-party system. The enterprise system may include multiple intra-enterprise applications or intra-enterprise file services, and different intra-enterprise applications or intra-enterprise file services may be configured on different servers. The ESB can be each subsystem in the ESB in this solution. The third-party system may include multiple third-party applications or third-party file services, and different third-party applications or third-party file services may be configured on different servers. The ESB will establish unique identifiers and request concurrent numbers for different enterprise systems and different third-party systems. For example, an identifier "B10047" is established for a certain bank, and the number of concurrent requests is 100.

In this embodiment, the ESB-API publishing service protocol is the dubbo protocol, the service name format is "esb-api.service.XXXXXX_YYYYYY", and "XXXXXX" in the interface name is the service party (the enterprise system or the The identifier of the third-party system), the "YYYYYY" in the interface name is the service name provided by the service party, and the ESB-API publishes the service for the enterprise system and the ESB-PROXY-IN to call.

In this embodiment, the ESB-PROXY-IN publishing service protocol is http protocol and https protocol, the service name format is "esb.YYYYYY", and "YYYYYY" in the interface name is the service party (the enterprise The service name provided by the system or the third-party system), the ESB-PROXY-IN publishes the service for the third-party system to call. When the ESB-PROXY-IN publishes a service, a service correspondence table is maintained, and the service published by the ESB-PROXY-IN corresponds to the service published by the ESB-API, for example, the ESB-PROXY-IN The published esb.customer.queryReferrerList service corresponds to the esb-api.service.Fin004.ff.customer.queryReferrerList service published by the ESB-API.

In this embodiment, the ESB-PROXY-OUT, ESB-FILE-PROXY-OUT publishing service protocol is the dubbo protocol, the service name format is "esb-proxy.service.XXXXXX_YYYYYY", and "XXXXXX" in the interface name is the identification of the service party (the enterprise system or the third-party system), "YYYYYY" in the interface name is the service name provided by the service party, the ESB-PROXY-OUT, ESB-FILE-PROXY -OUT publishes services for the ESB-API to call. The services published by the ESB-PROXY-OUT and ESB-FILE-PROXY-OUT appear in pairs with the services published by the ESB-API, for example, esb-proxy.service.B10043.QueryGradingInterestRateService published by the ESB-PROXY-OUT The service is paired with the esb-api.service.B10043.QueryGradingInterestRateService service published by the ESB-API.

It should be noted that the ESB as a requester is mainly relative to each subsystem within the ESB, because there is also a calling relationship between the subsystems within the ESB.

For example, the ESB-PROXY-IN subsystem of the ESB calls other subsystems of the ESB, and the ESB-PROXY-IN subsystem of the ESB also calls the ESB-API subsystem of the ESB. At this time, the ESB-PROXY-IN subsystem is relatively The ESB-API subsystem is the requester.

S12. Detect whether a data request from the requester is received.

The requester's data request is a data request after the requester encapsulates the data request according to the service published by the ESB.

In this embodiment, the encapsulation of the data request by the requester according to the published service may include:

1) When the requester is an enterprise system, the data request is encapsulated according to the dubbo protocol issued by each subsystem of the ESB.

2) When the requester is a third-party system, the data request is encapsulated according to the http protocol or https protocol issued by each subsystem of the ESB.

The requester encapsulates the data request according to the services published by the various subsystems of the ESB. The requester only needs to encapsulate the data request according to the unified request message format of the ESB, and does not need to pay attention to the request message formats of different service parties. different question.

For example, the format of the unified request message provided by this embodiment is as follows:

When it is detected that the data request of the requester is received, step S13 is performed; otherwise, it is possible to continue to detect whether the data request of the requester is received.

S13. Identify whether the requester of the data request is legal.

In this embodiment, when the ESB receives the data request, it identifies whether the requester of the data request is legal according to the data request. The legality refers to whether the requesting party is a legal enterprise system or a third-party system recognized by the ESB, and the enterprise system and the third-party system refer to the systems that the ESB connects to and establishes a unique identifier for it. If the requester is illegal, the request is interrupted, and if the requester is legal, the subsequent process S14 is performed.

In this embodiment, the data request may carry a data request source identifier, and the data request source identifier is used to indicate the identity of the requester of the data request. The source identifier is the unique identifier established by the ESB for the system.

S14. Identify the service party of the data request according to the data request.

In this embodiment, when the ESB determines that the requesting party of the data request is a legitimate requesting party, it identifies the serving party of the data request according to the data request, and the serving party may be an enterprise system or a third-party system. The enterprise system may include multiple intra-enterprise applications or intra-enterprise file services, and different intra-enterprise applications or intra-enterprise file services may be configured on different servers. The third-party system may include multiple third-party applications or third-party file services, and different third-party applications or third-party file services may be configured on different servers.

In this embodiment, the data request may carry a data request service party identifier, and the data request service party identifier is used to indicate the identity of the data request service party. The identification of the data requesting service party includes: a first identification and a second identification, wherein the first identification is used to indicate that the data requesting service party is an intra-enterprise application or an intra-enterprise file service, and the second identification is used to indicate that The data request service party is a third-party application or a third-party file service.

Identifying the service side of the data request according to the data request may include: identifying the service side of the data request according to the data request service side identifier carried in the data request; when identifying the data request service side identifier as the first When identifying the data requesting service party, it is determined that the data requesting service party is an enterprise system; when the data requesting service party is identified as the second identification, it is determined that the data requesting service party is a third-party system.

For example, the identifier of the data requesting service party in this embodiment may be the "XXXXXX" part of the service name published by the ESB-API, ESB-PROXY-OUT, and ESB-FILE-PROXY-OUT.

S15 , according to the requesting server, encapsulate the data request into a corresponding message, and send the encapsulated message to KAFKA by the message producer in the ESB-API.

In this embodiment, an application programming interface (Application Programming Interface, API) is a predefined function, and is a service entry of the ESB and a message producer.

In this embodiment, encapsulating the data request into a corresponding message according to the requesting server, and sending the encapsulated message to KAFKA by the message producer in the ESB-API may include:

1) When the receiving system of the data request is the ESB-API, the data request is encapsulated into a message, and the encapsulated message is sent to KAFKA by the message producer in the ESB-API.

2) When the receiving system of the data request is the first proxy server, the data request is encapsulated as a message, and after being forwarded to the ESB-API through the first proxy server, the message producer in the ESB-API sends the message to KAFKA.

After the message is encapsulated, it is directly sent by the message producer in the ESB-API to the message queue of KAFKA, and the message carries an identification number to indicate the location information of the stored message queue. For example, when the identification number is A, it indicates that the message is stored in message queue 1, and when the identification number is B, it indicates that the message is stored in message queue 2. The message queue may be a KAFKA message queue or an in-memory message queue.

The first proxy server may be an access proxy server (ESB-PROXY-IN).

The first proxy server may further include a load balancing function (for example, F5 shown in the figure) to perform load balancing control on the data request of the third-party system. Prevents paralysis due to stress on enterprise systems (eg, banking systems) when high concurrency occurs.

S16. When KAFKA receives the message to be processed sent by the ESB-API, it configures a corresponding message queue according to the message to be processed, and processes the message to be processed.

In this embodiment, KAFKA is a high-throughput distributed publish-subscribe messaging system that provides a message queue for ESB and can process all action flow data in consumer-scale websites. KAFKA has high throughput, supports load balancing, Features of asynchronous sending and message persistence. When KAFKA detects a message sent by ESB-API, it obtains the message to be processed, configures a corresponding message queue, and processes the message to be processed.

The processing of the messages to be processed may include: distributed storage of the messages to be processed into multiple copies; when the number of the messages to be processed exceeds a preset number threshold, each of the messages to be processed is stored. Messages are sliced or diced; messages to be processed are compressed. This ensures that pending messages are not lost and re-consumed.

In this embodiment, the ESB uses KAFKA as the message queue, which has the functions of high throughput and message persistence, which can well improve the processing capability of the ESB, making the processing capability of the ESB controllable. When the service performance of the ESB is low , you can use the message queue to buffer the information to avoid crashes.

S17. The ESB-CONSUMER pulls the to-be-processed message from the KAFKA message queue every preset time period, and creates a corresponding thread according to the to-be-processed message.

ESB-CONSUMER can monitor whether there are messages to be processed in KAFKA every preset time period (for example, 200 milliseconds). Route to the corresponding proxy server.

The ESB will create a corresponding thread pool according to the message queue. One message queue corresponds to a thread pool. Each thread pool consumes the messages to be processed in the corresponding message queue. Different thread pools do not interfere with each other. For example, thread pool 1 is only responsible for consuming pending messages in message queue 1, and thread pool 2 is only responsible for consuming pending messages in message queue 2.

In this embodiment, the creating a corresponding thread according to the to-be-processed message may include: creating a thread pool of a corresponding size according to the concurrent number of requests corresponding to the server. The number of concurrent requests is equal to the size of the thread pool.

Thread pools of different sizes are created according to the number of concurrent requests pre-configured by the server, so as to implement different concurrency controls according to different data request volumes. For example, the data request volume of an enterprise system is generally small, and the server (Third-party system) The pre-configured number of concurrent requests is correspondingly low, so the thread pool created by ESB-CONSUMER is also small, for example, 20-50; The number of concurrent requests pre-configured by the server (enterprise system) is also relatively high, so the thread pool created by ESB-CONSUMER is relatively large, for example, 100-150.

S18. The ESB-CONSUMER parses the to-be-processed message, and sends the parsed content to the server corresponding to the data request.

In this embodiment, the thread of the ESB-CONSUMER may be pre-configured with a configuration table corresponding to the message to be processed. The configuration table includes: the system service name required to complete the data request, the system service physical route, the system service follow-up process condition code, the condition code corresponding to the follow-up service and other information. Parsing the to-be-processed message is to retrieve the configuration table and interpret the content to determine the specific service content, and to determine the system service name, system service physical route, subsequent process condition codes, and subsequent services corresponding to the condition codes.

In this embodiment, the service party refers to a service provider that responds to a data request of the requester and provides a corresponding request result according to the data request, and the service party may include an enterprise system or a third-party system. Of course, if the enterprise system includes multiple enterprise subsystems, for example, enterprise subsystem 1, enterprise subsystem 2. The service party may also be one enterprise subsystem among multiple enterprise subsystems. Similarly, if the third-party system includes multiple third-party subsystems, for example, third-party subsystem 1 and third-party subsystem 2 . The serving party may also be a third-party subsystem among multiple third-party subsystems. That is, the requester and the server can be both enterprise systems and third-party systems. For example, when enterprise subsystem 1 requests to call enterprise subsystem 2, at this time, enterprise subsystem 1 is the requester, and enterprise subsystem 1 is the requester. 2 is the server.

For example, the service provider provided in this embodiment responds to the data request, that is, the format of the response message is as follows:

In this embodiment, sending the parsed content to the server corresponding to the data request may include:

1) When the service party of the data request is a third-party system, the parsed content is sent to the service party (third-party system) of the data request through the second proxy server.

The second proxy server may be an outgoing proxy server (service adapter ESB-PROXY-OUT or service file adapter ESB-FILE-PROXY-OUT), and the second proxy server receives the parsing sent by the thread of ESB-CONSUMER. When the content is parsed, the parsed content is forwarded to the third-party system. The ESB-PROXY-OUT is used to receive the message of the ESB-CONSUMER, and request the third-party system to respond to the data request and return the request result according to the message according to the request mode preset by the third-party system. The ESB-FILE-PROXY-OUT is used to receive the message of the ESB-CONSUMER, and according to the message, request the third-party system according to the request mode preset by the third-party system, and upload or download the corresponding file in the data request.

2) When the server of the data request is an enterprise system, directly send the parsed content to the enterprise system.

S19. The ESB-CONSUMER acquires the request result sent by the server of the data request and temporarily stores the request result in the message queue of KAFKA.

In this embodiment, the request result sent by the service party that obtains the data request may include:

1) When the service party of the data request is a third-party system, obtain the request result from the third-party system through the second proxy server, and temporarily store the request result in the message queue of KAFKA .

2) When the server of the data request is an enterprise system, the request result is directly obtained from the enterprise system, and the request result is temporarily stored in the message queue of KAFKA.

S20. The ESB-API obtains the request result from the KAFKA message queue, and sends the request result to the requester of the data request.

In this embodiment, sending the request result to the requester of the data request may include:

1) When the requester of the data request is an enterprise system, the ESB-API directly sends the request result to the enterprise system.

2) When the requester of the data request is a third-party system, the ESB-API sends the request result to the third-party system through the first proxy server.

It should be noted that, whether it is an enterprise system or a third-party system, both can be the requester of the data request, and can also be the server of the data request that responds to the data request and provides the request result. That is, when the enterprise system as the requester of the data request sends the data request to the servicer (third-party system) of the data request, it can also respond to the data request as the servicer of the data request and be the requester of the data request (the third-party system). system) to provide the request result. As the requester of the data request, the third-party system can also respond to the data request as the server of the data request and provide the requester (enterprise system) request result.

Further, when a data request is received, the method may further include: judging whether the current request state satisfies the fuse condition; when the current request state satisfies the fuse condition, configuring a first number of message queues, and sending a message to the currently received data The requester of the request returns preset information. When the current request state does not meet the circuit breaker condition, configure a second number of message queues.

The ESB can count the time-consuming time of each data request. The time-consuming time of the data request refers to the time from the data request sent by the requester receiving the data request to the request corresponding to the data request being returned to the requester of the data request by the ESB. result time.

The current request state may include one or more of the following combinations: the time-consuming time of the current data request; the average value of the time-consuming time of all data requests within a preset time period; the messages in the ESB-API message queue Quantity; the number of data requests from the requester that received the same data request.

The fusing condition may include one or more of the following combinations: the time-consuming time of the current data request exceeds the preset time-consuming time; the average time-consuming time of all data requests within the preset time period is greater than or equal to the interface Exceeded time; the number of messages in the ESB-API message queue exceeds the preset number; the number of data requests from the requester that receives the same data request exceeds the preset number.

The preset time-consuming time is a predefined time value. The preset information may be "the system is busy with business, please try again later". The first quantity value is equal to the size value of the thread pool, and the second quantity value is equal to the size value of the thread pool multiplied by the ratio of the interface timeout value and the average time-consuming time value.

In this embodiment, by presetting the fusing condition, when it is detected that the fusing condition is satisfied, the instantaneous high concurrency can exceed the processing capability of the ESB, for example, when the number of messages in the ESB-API message queue exceeds the preset number, KAFKA and ESB-CONSUMER are too late to process data requests, or when a large number of third-party systems are abnormal or the request times out, they can automatically fuse the current data request, increase the waiting time of the message in the message queue, and play the role of buffering the message. It reduces the pressure on the ESB, and at the same time, it can return preset information to the requester of the current data request in time, so as to achieve a timely response and improve the experience of the requester of the data request.

Further, the method may further include: configuring a visual management tool to store messages and respond to data requests. The visual management tools may include: mongo dB, mongo booster, and admin Mongo.

Further, the method may further include: providing a distributed application coordination service for providing a registry for KAFKA and dubbo. The distributed application coordination service may be a ZooKeeper component.

To sum up, the heavy-mode enterprise service bus data processing method described in the embodiment of the present invention adopts the synchronous processing mechanism of the open source framework dubbo, and combines the asynchronous processing mechanism of the open source processing stream platform KAFKA to convert the synchronous processing mechanism into an asynchronous processing mechanism. The processing mechanism makes ESB not only has the characteristics of dubbo's load balancing, but also has the buffering function of KAFKA's message queue, which can achieve dynamic expansion and message persistence: messages are persisted to local disks, and data backup is supported to prevent data loss. . Secondly, set a unified message format, by converting the data requests sent by the requesters of different data requests into a unified message and request protocol, it can shield the underlying technical differences of the heterogeneous systems, and also facilitate the requester of the data request. development and management. In addition, the corresponding threads are created according to the pending messages in the message queue, and the size of the thread pool is configured, which effectively prevents the pressure caused by the data request server when high concurrency occurs, which leads to the paralysis of the data request server system. Moreover, producers and consumers are deployed independently, and a producer can configure multiple consumers to consume messages to be processed. Finally, set the fuse condition, when the instantaneous high concurrency exceeds the ESB processing capacity or when a large number of exceptions in the third-party system or the request times out, the current data request can be automatically blown and controlled, and the waiting time of the message in the message queue can be increased. The function of buffering messages reduces the pressure on the ESB, and at the same time, it can return preset information to the requester of the current data request in time, so as to achieve a timely response and improve the experience of the requester of the data request.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this. improvements, but these all belong to the protection scope of the present invention.

Embodiment 3

FIG. 3 is a schematic diagram of a terminal according to Embodiment 5 of the present invention.

The terminal 3 includes: a memory 31 , at least one processor 32 , a computer program 33 stored in the memory 31 and executable on the at least one processor 32 , and at least one communication bus 34 .

When the at least one processor 32 executes the computer program 33, the steps in the foregoing embodiments of the enterprise service bus data processing method are implemented.

Exemplarily, the computer program 33 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 31 and executed by the at least one processor 32, to complete the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 33 in the terminal 3 .

The terminal 3 may be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. Those skilled in the art can understand that the schematic diagram 3 is only an example of the terminal 3, and does not constitute a limitation on the terminal 3, and may include more or less components than the one shown, or combine some components, or different components For example, the terminal 3 may further include an input and output device, a network access device, a bus, and the like.

The at least one processor 32 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC) ), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. The processor 32 can be a microprocessor or the processor 32 can also be any conventional processor, etc. The processor 32 is the control center of the terminal 3, and uses various interfaces and lines to connect various parts of the entire terminal 3 part.

The memory 31 can be used to store the computer program 33 and/or modules/units, and the processor 32 executes or executes the computer programs and/or modules/units stored in the memory 31 and calls the computer programs and/or modules/units stored in the memory 31. 31 to realize various functions of the terminal 3 . The memory 31 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may Data created according to the use of the terminal 3 (such as audio data, phone book, etc.) and the like are stored. In addition, the memory 31 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card , a flash memory card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

If the modules/units integrated in the terminal 3 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the readable medium of the computer can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction area. For example, in certain jurisdictions, according to legislation and patent practice, computer readable medium Excluding electric carrier signals and telecommunications signals.

In the several embodiments provided by the present invention, it should be understood that the disclosed terminal and method may be implemented in other manners. For example, the terminal embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division manners in actual implementation.

In addition, each functional unit in each embodiment of the present invention may be integrated in the same processing unit, or each unit may exist physically alone, or two or more units may be integrated in the same unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the invention is to be defined by the appended claims rather than the foregoing description, which are therefore intended to fall within the scope of the claims. All changes within the meaning and range of the equivalents of , are included in the present invention. Any reference signs in the claims shall not be construed as limiting the involved claim. Furthermore, it is clear that the word "comprising" does not exclude other units or, and the singular does not exclude the plural. Several units or means recited in the system claims can also be realized by one unit or means by means of software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present invention.

Claims (10)

1. an enterprise service bus system, the enterprise service bus system communication connects a plurality of enterprise systems and a plurality of third-party systems, it is characterized in that, the enterprise service bus system comprises: ESB-API subsystem, KAFKA subsystem, ESB-CONSUMER subsystem, ESB-PROXY-IN subsystem, ESB-PROXY-OUT subsystem, ESB-FILE-PROXY-OUT subsystem, the KAFKA subsystem is respectively connected with the ESB-API subsystem and the ESB -CONSUMER subsystem is communicatively connected, the ESB-API subsystem is communicatively connected to the ESB-PROXY-IN subsystem, and the ESB-CONSUMER subsystem is respectively connected to the ESB-PROXY-OUT subsystem and the ESB- FILE-PROXY-OUT subsystem communication connection, where, The ESB-API subsystem is used to publish the dubbo service agreement, and the ESB-PROXY-IN subsystem is used to publish the http service agreement and the https service agreement; The ESB-API subsystem is further configured to send the message encapsulated by the enterprise service bus system to the KAFKA subsystem by the message producer; The KAFKA subsystem is configured to configure a corresponding message queue according to the to-be-processed message when receiving the to-be-processed message sent by the ESB-API subsystem, and to process the to-be-processed message, including: The messages to be processed are distributed and stored as multiple copies; when the number of messages to be processed exceeds a preset number threshold, each message in the messages to be processed is sliced or diced; the messages to be processed compress; The ESB-CONSUMER subsystem is used to pull the message to be processed from the message queue of the KAFKA subsystem every preset time period, and create a corresponding thread according to the message to be processed, including: according to the service Create a thread pool of corresponding size according to the number of concurrent requests corresponding to the party, and the number of concurrent requests is equal to the size of the thread pool; The ESB-CONSUMER subsystem is used to parse the message to be processed, and send the parsed content to the server corresponding to the data request; The ESB-CONSUMER subsystem is also used to obtain the request result sent by the server of the data request and temporarily store the request result in the message queue of the KAFKA subsystem; The ESB-API subsystem is further configured to obtain the request result from the message queue of the KAFKA subsystem, and send the request result to the requester of the data request. 2. a kind of utilizing enterprise service bus system as claimed in claim 1 to carry out enterprise service bus data processing method, it is characterized in that, described method comprises: Publishing service for the requester to call, the ESB-API subsystem publishing service protocol is the dubbo protocol, and the ESB-PROXY-IN subsystem publishing service protocol is the http protocol and the https protocol; Detect whether a data request from the requester is received; After detecting the data request received by the requester, identify whether the requester of the data request is legal; When identifying the requesting party of the data request is legitimate, identifying the serving party of the data request according to the data request; According to the requesting server, the data request is encapsulated into a corresponding message, and the encapsulated message is sent to the KAFKA subsystem by the message producer in the ESB-API subsystem; When the KAFKA subsystem receives the to-be-processed message sent by the ESB-API subsystem, it configures a corresponding message queue according to the to-be-processed message, and processes the to-be-processed message; Every preset time period, the ESB-CONSUMER subsystem pulls the message to be processed from the message queue of the KAFKA subsystem, and creates a corresponding thread according to the message to be processed; The ESB-CONSUMER subsystem parses the message to be processed, and sends the parsed content to the server corresponding to the data request; The ESB-CONSUMER subsystem obtains the request result sent by the server of the data request and temporarily stores the request result in the message queue of the KAFKA subsystem; The ESB-API subsystem obtains the request result from the message queue of the KAFKA subsystem, and sends the request result to the requester of the data request. 3. The method according to claim 2, wherein the data request of the requesting party is a data request after the requesting party encapsulates the data request according to the service agreement issued by the enterprise service system, comprising: When the requester is the enterprise system, encapsulate the data request according to the dubbo protocol; When the requester is a third-party system, the data request is encapsulated according to the http protocol or the https protocol. 4. The method of claim 2, wherein the identifying the service party of the data request according to the data request comprises: Identify the service party of the data request according to the data request service party identifier carried in the data request; When the identification of the data requesting service party is identified as the first identification, it is determined that the data requesting service party is an enterprise system; When the identification of the data requesting service party is identified as the second identification, it is determined that the data requesting service party is a third-party system. 5. The method according to claim 2, wherein the service party according to the request encapsulates the data request into a corresponding message, and sends the encapsulated message from the ESB-API subsystem The message producers that send messages to the KAFKA subsystem include: When the receiving system of the data request is the ESB-API subsystem, the data request is encapsulated into a message, and the encapsulated message is sent by the message producer in the ESB-API subsystem to the KAFKA subsystem; When the receiving system of the data request is the ESB-PROXY-IN subsystem, the data request is encapsulated into a message and forwarded to the ESB-API subsystem through the ESB-PROXY-IN subsystem , the message producer in the ESB-API subsystem sends the message to the KAFKA subsystem. 6. The method of claim 2, wherein The creating a corresponding thread according to the to-be-processed message includes: creating a thread pool of a corresponding size according to the concurrent number of requests corresponding to the server, where the concurrent number of requests is equal to the size of the thread pool; The sending the parsed content to the service party corresponding to the data request includes: when the service party of the data request is the third-party system, using the ESB-PROXY-OUT subsystem or the ESB- The FILE-PROXY-OUT subsystem sends the parsed content to the server of the data request; when the server of the data request is the enterprise system, it directly sends the parsed content to the enterprise system ; The sending the request result to the requester of the data request includes: when the requester of the data request is the enterprise system, the ESB-API subsystem directly sends the request result to the data request. Enterprise system; when the requester of the data request is the third-party system, the ESB-API subsystem transfers the data through the ESB-PROXY-OUT subsystem or the ESB-FILE-PROXY-OUT subsystem The request result is sent to the third-party system. 7. The method according to any one of claims 2 to 6, wherein the method further comprises: Determine whether the current request status satisfies the fuse condition; When the current request status satisfies the fuse condition, configure the first number of message queues, and return preset information to the requester of the currently received data request; When the current request state does not meet the circuit breaker condition, configure a second number of message queues. 8. The method of claim 7, wherein the fusing condition comprises a combination of one or more of the following: The elapsed time of the current data request exceeds the preset elapsed time; or The average time-consuming time of all data requests within the preset time period is greater than or equal to the interface timeout time; or The number of messages in the ESB-API subsystem message queue exceeds a preset number; or The number of data requests of the requester who received the same data request exceeds the preset number. 9. A terminal, characterized in that the terminal comprises a processor and a memory, and the processor is configured to implement the enterprise service according to any one of claims 2 to 8 when executing a computer program stored in the memory Bus data processing method. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the computer program as claimed in any one of claims 2 to 8 is implemented. Enterprise Service Bus data processing methods.

Images