CN107784068A - Acquisition methods, device, storage medium, processor and the service end of data variation - Google Patents

Abstract

The invention discloses a data change acquisition method, device, storage medium, processor and server. The method includes: receiving a request message from a terminal, wherein the request message is used to acquire data changes in one or more databases on the server from a first time point to a second time point; querying data changes according to the request message , get the query result; return the query result to the terminal. The invention solves the technical problems in the related art that the method of manually inquiring about the data changes in the database is time-consuming, laborious, inefficient, and poor in accuracy.

Description

Data change acquisition method, device, storage medium, processor and server

technical field

The present invention relates to the field of computers, in particular to a data change acquisition method, device, storage medium, processor and server.

Background technique

At present, in the process of software development, development and testing engineers usually need to accurately know the data changes stored in the database at a specific time node. In order to achieve this goal, development and test engineers often need to initiate connections through various database client tools to access various databases, and then write and execute Structured Query Language (Structured Query Language, referred to as SQL) statements to query the database to generate specific data changes. However, the above operation method has the following disadvantages:

(1) The efficiency is not high. For example, if multiple data tables on multiple database instances change, engineers need to establish connections with different database instances, and then write different SQL statements to access the data tables to query data changes, which is time-consuming, laborious and inefficient .

(2) It is easy to miss. For example, when multiple data tables change, the engineer may forget to check a certain data table, or the program has a bug that modifies the data in the data table that should not be modified, but the engineer does not find out in time.

(3) It is difficult to know the sequence of data changes. When it is necessary to verify the sequence of data changes in multiple data tables, due to the very fast speed of the computer, it is difficult to check in time by manual methods.

(4) When there is stock data in the database, it is usually difficult to distinguish the exact range of data changes by manual methods. For example, if a data table contains tens of thousands of rows of data, which pieces of data have been inserted, which fields of which pieces of data have been modified, and which pieces of data have been deleted, it is difficult to accurately confirm only manually .

For the above problems, no effective solution has been proposed yet.

Contents of the invention

At least some embodiments of the present invention provide a data change acquisition method, device, storage medium, processor, and server to at least solve the time-consuming, labor-intensive, low-efficiency, and Technical issues with poor accuracy.

According to one embodiment of the present invention, a method for acquiring data changes is provided, including:

Receive a request message from the terminal, wherein the request message is used to obtain the data changes of one or more databases on the server from the first time point to the second time point; query the data changes according to the request message, and obtain the query result ; Return the query result to the terminal.

Optionally, before receiving the request message from the terminal, it also includes: receiving configuration information from the terminal, wherein the configuration information includes: identification information of one or more database instances generated on the server; according to the configuration information Determine that one or more database instances are connected.

Optionally, after receiving the request message from the terminal, the method further includes: verifying the parameter information carried in the request message, where the parameter information includes: a first time point, a second time point, and data change indication information; If the verification of the request message is successful, the request message is split into multiple query requests according to the database instance corresponding to the data change indication information, wherein each query query request corresponds to a different database instance.

Optionally, querying data changes according to the request message, and obtaining the query results include: obtaining data access logs from corresponding database instances through each query request; extracting data changes corresponding to each database instance from the obtained data access logs information; converting the extracted data change information from a first preset format to a second preset format to obtain a query result.

Optionally, after querying data changes according to the request message and obtaining query results, the method further includes: converting the query results from the second preset format to a third preset format.

Optionally, the first time point is the start time point when the current user starts the current code testing process on the terminal, and the second time point is the time point when the terminal initiates the request message, wherein the timing of the request message includes one of the following: the terminal The time point at which the query result is displayed and the time point at which the terminal periodically displays the query result are determined according to the control command input by the current user.

According to one embodiment of the present invention, a device for acquiring data changes is also provided, including:

The first receiving module is configured to receive a request message from the terminal, wherein the request message is used to obtain data changes of one or more databases on the server from the first time point to the second time point; the obtaining module, It is used to query data changes according to the request message to obtain the query result; the feedback module is used to return the query result to the terminal.

Optionally, the above device further includes: a second receiving module, configured to receive configuration information from the terminal, wherein the configuration information includes: identification information of one or more database instances generated on the server; Based on the configuration information to determine that one or more database instances are connected.

Optionally, the above device further includes: a checking module, configured to check the parameter information carried in the request message, where the parameter information includes: the first time point, the second time point, and data change indication information; A module for splitting the request message into multiple query requests according to the database instance corresponding to the data change indication information when the verification of the request message is successful, wherein each query query request corresponds to a different database instance.

Optionally, the acquiring module includes: an acquiring unit, configured to acquire data access logs from corresponding database instances through each query request; an extracting unit, configured to extract data corresponding to each database instance from the acquired data access logs Data change information; a processing unit configured to convert the extracted data change information from a first preset format to a second preset format to obtain query results.

Optionally, the above device further includes: a processing module, configured to convert the query result from the second preset format to a third preset format.

Optionally, the first time point is the start time point when the current user starts the current code testing process on the terminal, and the second time point is the time point when the terminal initiates the request message, wherein the timing of the request message includes one of the following: the terminal The time point at which the query result is displayed and the time point at which the terminal periodically displays the query result are determined according to the control command input by the current user.

According to one embodiment of the present invention, a storage medium is further provided, and the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above method for obtaining data changes.

According to one embodiment of the present invention, a processor is also provided, and the processor is used to run a program, wherein the above method for acquiring data changes is executed when the program is running.

According to one embodiment of the present invention, there is also provided a server, including: one or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory, and Configured to be executed by one or more processors, one or more programs are used to execute the data change acquisition method in any one of claims 1 to 6.

In at least some embodiments of the present invention, a method of receiving a request message from a terminal, where the request message is used to obtain data changes in one or more databases on the server from the first time point to the second time point , query data changes according to the request message, and get the query results; return the query results to the terminal, to achieve the purpose of automatically displaying the data changes of the specified database instance within a specific period of time, so that development engineers can smoothly carry out debugging work during the software development process And test engineers can carry out testing work smoothly, improve work efficiency, and then solve the technical problems of time-consuming, laborious, low efficiency, and poor accuracy in related technologies by manually querying data changes in the database.

Description of drawings

The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:

FIG. 1 shows a schematic structural diagram of a system for obtaining a data change method;

Fig. 2 is a flowchart of a method for acquiring data changes according to an embodiment of the present invention;

Fig. 3 is a structural block diagram of a device for acquiring data changes according to an embodiment of the present invention;

Fig. 4 is a structural block diagram of a device for acquiring data changes according to one preferred embodiment of the invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

According to one embodiment of the present invention, an embodiment of a data change acquisition method is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions , and, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

This method embodiment can be executed in a system composed of a terminal and a server. FIG. 1 shows a schematic structural diagram of a system for obtaining a data change method. As shown in FIG. 1 , the terminal can be connected to a server via a data network or electronically, such as a security server, a resource server, or a game server. In an optional embodiment, the aforementioned terminal may be any mobile computing device. The data network connection may be a local area network connection, a wide area network connection, an Internet connection, or another type of data network connection. Terminals can be implemented to connect to web services performed by a server (eg: a resource server) or a group of distributed servers. Web services are web-based user services, such as data queries.

In a specific implementation process, the server may include: a first server and a second server. The terminal first sends an access request to the first server. After the first server completes the verification and splitting of the access request, it sends it to the second server to obtain the data changes on the database within the specified time range, and then converts its format into SQL format, where the second server installs There is a binlog2sql tool for parsing the binary log (binlog). The binlog2sql tool can convert the core log (binlog, which includes all accesses and parameters to the database) of the database software system of the database into an SQL format. Then, the second server returns the query result to the terminal via the first server. Finally, the terminal formats the query results so that they can be presented to engineers in a complete and orderly manner, which greatly improves the work efficiency of debugging and testing.

It should be noted that the above-mentioned distributed architecture of the first server and the second server may also be replaced by combining the functions required to be implemented by the first server and the second server into one server.

Under the above operating environment, the present application provides a data change acquisition method as shown in FIG. 2 . Fig. 2 is a flowchart of a method for acquiring data changes according to one embodiment of the present invention. As shown in Fig. 2, the method includes the following steps:

Step S22, receiving a request message from the terminal, wherein the request message is used to obtain data changes in one or more databases on the server from the first time point to the second time point;

Step S25, query the data change according to the request message, and obtain the query result;

Step S27, returning the query result to the terminal.

Through the above steps, it is possible to adopt the method of receiving a request message from the terminal, the request message is used to obtain the data changes of one or more databases on the server from the first time point to the second time point, according to the request message Query data changes to obtain query results; return the query results to the terminal to achieve the purpose of automatically displaying the data changes of the specified database instance within a specific period of time, so that development engineers can smoothly carry out debugging work and test engineers in the software development process. The test work was carried out smoothly, and the work efficiency was improved, thereby solving the technical problems of manual query of data changes in the database, which was time-consuming, labor-intensive, low-efficiency, and poor-accuracy in related technologies.

The above database instance can be understood as a group of background processes/threads and a shared memory area. Multiple database instances can be run on the server, and different database instances correspond to different ports. The data stored in each database instance is relatively independent. One database instance can correspond to multiple databases, and one database can correspond to multiple data tables. Each database instance can mount and open one database during its lifetime.

Optionally, in step S22, before receiving the request message from the terminal, the following execution steps may also be included:

Step S20, receiving configuration information from the terminal, wherein the configuration information includes: identification information of one or more database instances generated on the server;

Step S21, according to the configuration information, it is determined that one or more database instances are in a connected state.

The terminal can provide the user with an interactive interface, so that the user can configure the relevant information of each database instance, which may include but not limited to: name, Internet Protocol (IP) address, port, user name and password.

The first server receives and stores the configuration information from the terminal, and then tests the connectivity between the database instances according to the configuration information. If one or more database instances cannot be connected, a failure prompt message will be returned to the terminal to prompt the user to check and modify configuration information.

In the preferred implementation process, the above-mentioned first time point is the start time point when the current user starts the current code testing process on the terminal, and the second time point is the time point when the terminal initiates the request message, wherein the initiation timing of the request message includes the following One: The terminal determines the time point for displaying the query results according to the control command currently input by the user, and the time point for periodically displaying the query results.

A "Start" button can be set on the interactive interface, and when the user clicks the "Start" button, the terminal will automatically record the time point of starting the test (equivalent to the above-mentioned first time point). The terminal needs to verify its connectivity with the first server. If the connection between the terminal and the first server cannot be achieved, a failure prompt message needs to be displayed on the interactive interface to prompt the user to check the configuration information.

In addition, the end time of the database change (equivalent to the second time point above) can be obtained in the following two ways:

Method 1. Configure the "Display Data Change" button in the above-mentioned interactive interface. After the user clicks this button, the above-mentioned request message will be triggered, and the time when this button is clicked can be recorded as the end time of the database change.

Method 2: When the above request message is triggered every preset time interval (for example: 15 seconds), whenever the end time point of the preset time length is reached, the end time point is recorded as the end time of the database change.

Optionally, in step S22, after receiving the request message from the terminal, the following execution steps may also be included:

Step S23, verifying the parameter information carried in the request message, wherein the parameter information includes: a first time point, a second time point, and data change indication information;

Step S24, if the verification of the request message is successful, split the request message into multiple query requests according to the database instance corresponding to the data change indication information, wherein each query query request corresponds to a different database instance.

The terminal sends a request message to the first server, wherein the request message is used to obtain data changes of one or more database instances, and the parameter information carried in the request message may include but not limited to: the current user, the start time of the database change (ie The time point at which the above-mentioned test starts) and the end time of the database change, and the data change indication information (including: a list of database instances that need to be changed, specified and excluded database instances and data tables). For example, if the user wants to view the data table C of the database B of the database instance A, then, the obtained overall database change data needs to be screened, so that irrelevant data will not be returned from the first server to the terminal.

The first server receives and verifies the request message. That is, the first server needs to confirm that the request message carries complete and accurate parameter information, and if the parameter information is incomplete or inaccurate, this type of request message will be identified as an illegal message. For example: the start time of the database change and/or the end time of the database change are not carried in the request message; or, the content corresponding to the start time of the database change should have filled in the time type information, but actually corresponds to the text type information.

Further, the database changes requested by the above request message may involve multiple database instances, and the request message is sent from the terminal to the first server at one time. Therefore, the first server needs to split the request message into multiple query requests for accessing different database instances according to the principle that each database instance corresponds to a query request to access the binlog2sql-api in the second server. Among them, binlog2sql-api It is based on the existing functions of binlog2sql to provide an interface that can be accessed by the interactive interface of the terminal (for example: web client).

Optionally, in step S25, querying data changes according to the request message, and obtaining query results may include the following execution steps:

Step S251, obtaining data access logs from corresponding database instances through each query request;

Step S252, extracting data change information corresponding to each database instance from the obtained data access log;

Step S253, converting the extracted data change information from the first preset format to the second preset format to obtain query results.

Every time binlog2sql-api receives a query request, it will verify the request and forward the request to binlog2sql after verification. The verification here is mainly to ensure that each query request contains necessary parameter information.

Further, binlog2sql obtains database change information from the corresponding database instance according to each query request, so as to obtain the data access log binlog of the database instance. binlog2sql formats the binlog to extract the database change information within the specified time period (that is, the time period between the start time of the database change and the end time of the database change) from the complicated data provided by the binlog, and then The obtained database change information is converted from text format to SQL format. Finally, binlog2sql returns the query result to binlog2sql-api. The binlog2sql-api returns the query result to the first server.

Optionally, in step S25, after querying data changes according to the request message and obtaining the query result, the following execution steps may also be included:

Step S26, converting the query result from the second preset format to the third preset format.

After verifying the query result, the first server formats the query result and returns it to the terminal. The formatting here is mainly to transform part of the information in the database change information in SQL format into information that is easy for users to read, for example: to convert the time information in the database change information into a daily time format that can be recognized by users.

The terminal formats the query results and presents them to the user through an interactive interface, thereby achieving the following technical effects:

(1) All database changes are automatically displayed on the same page, without the need for users to write SQL scripts and access database instances one by one.

(2) All database changes are displayed completely, thereby avoiding omission of data tables or data items.

(3) The changes of the database are displayed in chronological order, and the user can clearly understand the sequence relationship of the database changes.

(4) All data changes can be clearly listed and clearly distinguished from historical data, which is convenient for users to accurately check whether this data change meets expectations. For example: displaying historical data in a first color and displaying latest data in a second color, or displaying historical data in a first area and displaying latest data in a second area.

The formatting process performed here is mainly used to determine the display mode of the database changes, so as to make the display content as beautiful as possible and save space.

According to one embodiment of the present invention, an embodiment of a device for acquiring data changes is also provided. FIG. 3 is a structural block diagram of a device for acquiring data changes according to an embodiment of the invention. As shown in FIG. 3 , the device Including: a first receiving module 10, configured to receive a request message from the terminal, wherein the request message is used to obtain data changes of one or more databases on the server from the first time point to the second time point; The acquiring module 20 is configured to query data changes according to the request message to obtain query results; the feedback module 30 is configured to return the query results to the terminal.

Optionally, FIG. 4 is a structural block diagram of an apparatus for acquiring data changes according to one preferred embodiment of the invention. As shown in FIG. 4, the above-mentioned apparatus may further include: a second receiving module 40, configured to receive data from Configuration information, wherein the configuration information includes: identification information of one or more database instances generated on the server; a determination module 50 configured to determine that one or more database instances are in a connected state according to the configuration information.

Optionally, as shown in FIG. 4, the above-mentioned device may further include: a checking module 60, configured to check the parameter information carried in the request message, where the parameter information includes: a first time point, a second time point . Data change indication information; splitting module 70, used for splitting the request message into a plurality of query requests according to the database instance corresponding to the data change indication information when the verification of the request message is successful, wherein each request query The requests correspond to different database instances respectively.

Optionally, the acquisition module 20 includes: an acquisition unit (not shown in the figure), configured to acquire data access logs from corresponding database instances through each query request; an extraction unit (not shown in the figure), configured to extract Extract data change information corresponding to each database instance from the obtained data access log; a processing unit (not shown in the figure) is used to convert the extracted data change information from a first preset format to a second preset format format to get the query result.

Optionally, as shown in FIG. 4 , the above apparatus may further include: a processing module 80 configured to convert the query result from the second preset format to a third preset format.

Optionally, the first time point is the start time point when the current user starts the current code testing process on the terminal, and the second time point is the time point when the terminal initiates the request message, wherein the timing of the request message includes one of the following: the terminal The time point at which the query result is displayed and the time point at which the terminal periodically displays the query result are determined according to the control command input by the current user.

The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

In the above-mentioned embodiments of the present invention, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed technical content can be realized in other ways. Wherein, the device embodiments described above are only illustrative. For example, the division of the units may be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes. .

The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.

Claims (15)

1. A kind of 1. acquisition methods of data variation, it is characterised in that including：

   The request message for coming from terminal is received, wherein, the request message is used to obtain one or more data in service end Storehouse is from first time point to the data variation occurred between the second time point；

   The data variation is inquired about according to the request message, obtains Query Result；

   The Query Result is back to the terminal.
2. 2. according to the method for claim 1, it is characterised in that receive come from the terminal the request message it Before, in addition to：

   The configuration information for coming from the terminal is received, wherein, the configuration information includes：One generated in the service end The identification information of individual or multiple database instances；

   Determine that one or more of database instances are in connection status according to the configuration information.
3. 3. according to the method for claim 1, it is characterised in that receive come from the terminal the request message it Afterwards, in addition to：

   The parameter information carried in the request message is verified, wherein, the parameter information includes：The very first time Point, second time point, data variation configured information；

   In the case of to request message verification successfully, according to database instance corresponding to the data variation configured information The request message is split as multiple queries request, wherein, it is real that each requesting query request corresponds to different databases respectively Example.
4. 4. according to the method for claim 3, it is characterised in that the data variation is inquired about according to the request message, obtained Include to the Query Result：

   Data access daily record is obtained to corresponding database instance by each inquiry request respectively；

   Data variation information corresponding with each database instance is extracted from the data access daily record got；

   The data variation information extracted is converted into the second preset format from the first preset format, obtains the Query Result.
5. 5. according to the method for claim 4, it is characterised in that the data variation is being inquired about according to the request message, After obtaining the Query Result, in addition to：

   The Query Result is converted into the 3rd preset format from second preset format.
6. 6. according to the method for claim 1, it is characterised in that the first time point is in the terminal by active user Point between at the beginning of startup current code test process, when second time point is that the terminal initiates the request message Between point, wherein, the initiation opportunity of the request message includes one below：The control that the terminal inputs according to the active user System order determines to show the time point that the time point of the Query Result, the terminal periodic show the Query Result.
7. A kind of 7. acquisition device of data variation, it is characterised in that including：

   First receiving module, the request message of terminal is come from for receiving, wherein, the request message is used to obtain service end Upper one or more databases are from first time point to the data variation occurred between the second time point；

   Acquisition module, for inquiring about the data variation according to the request message, obtain Query Result；

   Feedback module, for the Query Result to be back into the terminal.
8. 8. device according to claim 7, it is characterised in that described device also includes：

   Second receiving module, the configuration information of the terminal is come from for receiving, wherein, the configuration information includes：Institute State the identification information of the one or more database instances generated in service end；

   Determining module, for determining that one or more of database instances are in connection status according to the configuration information.
9. 9. device according to claim 7, it is characterised in that described device also includes：

   Correction verification module, for being verified to the parameter information carried in the request message, wherein, the parameter information bag Include：The first time point, second time point, data variation configured information；

   Split module, for the request message verification successfully in the case of, according to the data variation configured information pair The request message is split as multiple queries request by the database instance answered, wherein, each requesting query request corresponds to respectively Different database instances.
10. 10. device according to claim 9, it is characterised in that the acquisition module includes：

    Acquiring unit, for obtaining data access daily record to corresponding database instance respectively by each inquiry request；

    Extraction unit, for extracting data variation letter corresponding with each database instance from the data access daily record got Breath；

    Processing unit, for the data variation extracted information to be converted into the second preset format from the first preset format, obtain The Query Result.
11. 11. device according to claim 10, it is characterised in that described device also includes：

    Processing module, for the Query Result to be converted into the 3rd preset format from second preset format.
12. 12. device according to claim 7, it is characterised in that the first time point is at the end by active user Point between at the beginning of end startup current code test process, second time point are that the terminal initiates the request message Time point, wherein, the initiation opportunity of the request message includes one below：The terminal inputs according to the active user Control command determines to show the time point that the time point of the Query Result, the terminal periodic show the Query Result.
13. A kind of 13. storage medium, it is characterised in that the storage medium includes the program of storage, wherein, run in described program When control the storage medium where data variation in equipment perform claim requirement 1 to 6 described in any one acquisition methods.
14. A kind of 14. processor, it is characterised in that the processor is used for operation program, wherein, right of execution when described program is run Profit requires the acquisition methods of the data variation described in any one in 1 to 6.
15. A kind of 15. service end, it is characterised in that including：One or more processors, memory, display device and one or Multiple programs, wherein, one or more of programs are stored in the memory, and be configured as by one or Multiple computing devices, one or more of programs are used for the data variation described in any one in perform claim requirement 1 to 6 Acquisition methods.