WO2019072212A1 - Log processing method and device and server - Google Patents

Abstract

Embodiments of the present invention provide a log processing method and device and a server. The method comprises: applying for a log segment, wherein the log segment comprises M log spaces arranged in an order; concurrently writing N logs to the M log spaces, wherein N ≤ M and the N logs are generated by an external device; sliding a sliding window from a start log space towards an end log space of the log segment, wherein the sliding window covers K log spaces and K < M; and sending to the external device first confirmation information for each log space in a write-completed state in the sliding window, such that each log space in the write-completed state in the sliding window is in the write-completed state, and the first confirmation information is no longer sent to the external device for each log space in the write-completed state in the sliding window. With the above method, the efficiency of log reading can be ensured and loss caused by concurrent writing is reduced.

Description

Log processing method and device, server Technical field

The present application relates to the field of information technology, and in particular, to a log processing method and apparatus, and a server.

Background technique

In the log system of the prior art, the log system applies for a log segment on the disk, and the log segment includes a plurality of log spaces arranged in sequence. The external device sends a log write request carrying the log to the log system, and the log system receives the log. The log is stored in the log space on the log side.

In the prior art, the external device sends a plurality of log write requests to the log system, and the log system writes the logs carried by the multiple log write requests to the log space of the log segment concurrently, for any of the log segments. In the case of a log space, if the log space is written to the log space, the log system does not send an acknowledgment (ACK) to the external device in the other log space in front of the log space, the log system cannot Sending an ACK to an external device, the log system must wait until the other log space located in front of the log space sends an acknowledgment message ACK to the external device, in order to send an acknowledgment message ACK to the external device for the log space. In the prior art, the above setting ensures that when the log segment is read later, the log of the first write failure is read, and the other log space of the log segment is stopped, thereby ensuring high efficiency. Read the log segment.

However, in the prior art, since the log system must wait until the other log space located in front of the specific log space sends an acknowledgment message ACK to the external device, the ACK message ACK can be sent to the external device for the specific log space, which results in The loss of write concurrency affects the performance of the write log. If the dependency between the ACKs is removed, then the log segment may have a successful write log and a write failed log stored in the log segment, so When reading the log segment, if you want to get a valid log, you must traverse the entire log segment, which makes the log read inefficient.

Summary of the invention

To solve the problem of the prior art, the embodiment of the invention provides a log processing method and device, and a server, which can ensure the efficiency of log reading and reduce the loss of write concurrency.

In a first aspect, an embodiment of the present invention provides a log processing method, including:

Application log segment, the log segment includes M log spaces arranged in order;

Write N logs concurrently to M log spaces, where N≤M, and N logs are generated by an external device;

Use the sliding window to slide from the starting log space of the log segment to the log space at the end of the log segment, wherein the sliding window covers K log spaces, K<M;

Sending the first confirmation information to the external device for each log space in the write completion state in the sliding window, so that each log space in the write completion state in the sliding window is in a write success state, and is written outside the sliding window. Each log space in the completion status does not send the first confirmation message to the external device;

The first confirmation information is used to notify the external device that the log writing is successful.

In summary, in the embodiment of the present invention, since the sliding window is used, and the first confirmation information is sent to the external device for the log space in the writing completion state in the sliding window, the log is in the writing completion state for the sliding window. The space does not send the first confirmation message to the external device, and since the sliding window covers a larger range with respect to a single log space, the loss of write concurrency can be reduced. Further, since the log space before the sliding window is in the write success state, the efficiency of the log reading can be ensured.

In an implementation manner of the embodiment of the present invention, the sliding window is used to slide from the starting log space of the log segment to the end log space, which specifically includes:

When the starting log space of the sliding window is in the write success state, the sliding window is swiped to the end log space of the log segment until the new starting log space of the sliding window is in a state other than the non-writing success state.

The sliding window is swiped in the above manner to ensure that the log space before the sliding window is in a successful writing state.

In another implementation manner of the embodiment of the present invention, K is a variable, and K=Kn, where n is the number of slidings of the sliding window, and Kn is that the sliding window covers the Kn log space after the nth sliding, wherein K0 =M/10, and:

When N>=4,

When N<4,

By dynamically adjusting the sliding window size under different load pressures, the concurrent write performance of the log system can be effectively improved, and the efficiency of log reading can be improved.

In another implementation manner of the embodiment of the present invention, the method further includes:

When the initial log space of the sliding window is in the write failure state, invalid data is written on the start log space of the sliding window, and after the invalid data is written, the second confirmation message is sent to the starting log space of the sliding window to The external device waits for the log space in the sliding window except the starting log space of the sliding window to complete the log writing, wherein the second confirmation information is used to notify the external device that the log writing fails.

In another implementation manner of the embodiment of the present invention, the method further includes:

Each log space of the log segment is read one by one from the log space of the log segment to the log space at the end of the log segment;

When invalid data is first read, K-1 log spaces located after the log space with invalid data are read one by one.

Through the above reading mode, it is ensured that the logs of the K-1 log spaces in the sliding window of the last sliding can be read, thereby ensuring that the data read to the log is complete.

In a second aspect, an embodiment of the present invention provides a log processing apparatus, including:

The log segment application module is used to apply for a log segment, and the log segment includes M log spaces arranged in order;

a log writing module, configured to concurrently write N logs to the M log spaces, where N≤M, and N logs are generated by an external device;

a sliding window module, configured to slide from a starting log space of a log segment to a log space at a end of the log segment by using a sliding window, wherein the sliding window covers K log spaces, K<M;

The confirmation information sending module is configured to separately send the first confirmation information to the external device for each log space in the write completion state in the sliding window, so that each log space in the write completion state in the sliding window is in a write success state. , the first confirmation information is not sent to the external device for each log space in the write completion state outside the sliding window;

The first confirmation information is used to notify the external device that the log writing is successful.

The implementation of the second aspect or the second aspect is the system implementation corresponding to the first aspect or the implementation manner of any one of the first aspect, and the description in any one of the first aspect or the first aspect is applicable to the second aspect. Or any implementation of the second aspect, and details are not described herein again.

In an implementation manner of the embodiment of the present invention, the sliding window module is specifically configured to:

When the starting log space of the sliding window is in the write success state, the sliding window is swiped to the end log space of the log segment until the new starting log space of the sliding window is in a state other than the non-writing success state.

In another implementation manner of the embodiment of the present invention, K is a variable, and K=Kn, where n is the number of slidings of the sliding window, and Kn is that the sliding window covers the Kn log space after the nth sliding, wherein K0 =M/10, and:

When N>=4,

When N<4,

In another implementation manner of the embodiment of the present invention, the confirmation information sending module is further configured to:

When the initial log space of the sliding window is in the write failure state, invalid data is written on the start log space of the sliding window, and after the invalid data is written, the second confirmation message is sent to the starting log space of the sliding window to The external device waits for the log space in the sliding window except the starting log space of the sliding window to complete the log writing, wherein the second confirmation information is used to notify the external device that the log writing fails.

In another implementation manner of the embodiment of the present invention, the log processing apparatus further includes a log reading module, and a log reading module, configured to:

Each log space of the log segment is read one by one from the log space of the log segment to the log space at the end of the log segment;

When invalid data is first read, K-1 log spaces located after the log space with invalid data are read one by one.

In a third aspect, an embodiment of the present invention provides a server, including a memory and a processor, where the memory stores program instructions, and the processor executes the program instructions to cause the server to perform the functions of the log processing apparatus.

In a fourth aspect, the present application provides a computer readable storage medium having instructions stored therein that, when run on a computer, cause the computer to perform any of the first aspect or the first aspect described above The method described in the implementation.

In a fifth aspect, the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method described in the first aspect or any of the possible implementations of the first aspect.

DRAWINGS

1 is a schematic structural diagram of a log processing system according to an embodiment of the present invention;

2 is a flowchart of a log processing method according to an embodiment of the present invention;

3 is a schematic diagram of a log segment in a first state according to an embodiment of the present invention;

4 is a schematic diagram of concurrently writing a log in a log segment according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a log segment in a second state according to an embodiment of the present invention; FIG.

6 is a schematic diagram of a log segment in a third state according to an embodiment of the present invention;

7 is a schematic diagram of a log segment in a fourth state according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a log segment in a fifth state according to an embodiment of the present invention; FIG.

9 is a schematic diagram of a log segment in a sixth state according to an embodiment of the present invention;

FIG. 10 is another flowchart of a log processing method according to an embodiment of the present invention; FIG.

11 is a schematic structural diagram of an apparatus of a log processing apparatus according to an embodiment of the present invention;

FIG. 12 is a schematic structural diagram of an apparatus of a server according to an embodiment of the present invention.

Detailed ways

First, please refer to FIG. 1. FIG. 1 is a schematic structural diagram of a log processing system according to an embodiment of the present invention. As shown in FIG. 1, the log processing system includes a server 10, a network 20, and external devices 201, 202, and 203, and the server 10 passes through a network. 20 is connected to external devices 201, 202, 203.

The server 10 specifically includes a log processing device 100, an operating system 102, and hardware 101. The hardware 101 specifically includes a magnetic disk 1011, a network card 1012, a CPU 1013, and a memory 1014. The log processing device 100 runs on the operating system 102, and controls the network card 1012 to receive the log-writing request written by the external device 201, 202, 203 from the network 20 through the operating system 102, and can send a confirmation via the network 20. The information is to the external devices 201, 202, 203.

Moreover, the log processing apparatus 100 can apply for the disk 1011 space to the disk 1011 as a log segment through the operating system 102, wherein the log segment includes a plurality of sequentially arranged log spaces, and the log space is used to store the log.

Optionally, a log space stores a log carried by a log write request.

Referring to FIG. 2, FIG. 2 is a flowchart of a log processing method according to an embodiment of the present invention. As shown in FIG. 2, a log processing method according to an embodiment of the present invention includes the following steps:

Step S101: The log processing device 100 applies for the log segment 300, and sets the sliding window 200 to the starting position of the log segment 300. The log segment 300 includes M log spaces arranged in order, and the sliding window 200 covers K log spaces. K<M.

For example, referring to FIG. 3, FIG. 3 is a schematic diagram of a log segment 300 in a first state according to an embodiment of the present invention. As shown in FIG. 3, the log processing device 100 applies to the log segment 300, and M=16. The sliding window 200 covers 7 log spaces, K=7.

In the embodiment of the present invention, the starting log space of the sliding window 200 coincides with the starting log space of the log segment 300, so that the sliding window 200 is set at the starting position of the log segment 300.

In this step, since the log processing apparatus 100 does not perform log writing in the log segment 300, each log space of the log segment 300 is in an unwritten state.

Step S102: The log processing apparatus 100 concurrently writes N logs to the M log spaces, where N≤M, and N logs are generated by the external device 201.

Optionally, in the embodiment of the present invention, each log space is correspondingly written with one log, and N logs are synchronously written into the first N log spaces of the M log spaces.

For example, referring to FIG. 4, FIG. 4 is a schematic diagram of concurrently writing logs in the log segment 300 according to an embodiment of the present invention. As shown in FIG. 4, N=13, that is, the log processing device 100 is simultaneously in the log space 1 to 13 Write logs 1 to 13 respectively.

Step S103: The log processing apparatus 100 separately transmits the first confirmation information to each external space 201 for each log space in the write completion state in the sliding window 200, and does not transmit for each log space in the write completion state outside the sliding window 200. The first confirmation information is sent to the external device 201, wherein the first confirmation information is used to notify the external device 201 that the log writing is successful.

Reference may be made in conjunction with FIGS. 5 and 6, wherein FIG. 5 is a schematic diagram of a log segment 300 in a second state, and FIG. 6 is a log segment 300 in a third state, in accordance with an embodiment of the present invention. schematic diagram.

As shown in FIG. 5, the log spaces 1, 2, 3, 5, 8, 10, 11, and 13 are all in the write completion state, and the log spaces 4, 6, 7, 9, 12, 14, 15, and 16 are all in the Write status.

The reason that the log space 4, 6, 7, 9, 12 is in the unwritten state is that the log is being written, or the log writing fails, and the log spaces 14, 15 and 16 are in the unwritten state because Log writes were not made for these log spaces.

In this step, the log processing apparatus 100 transmits the first confirmation information to the external device 201 for each of the log spaces 1, 2, 3, and 5 in the write completion state in the sliding window 200, and is written outside the sliding window 200. Each of the log spaces 8, 10, 11, and 13 of the completion status does not transmit the first confirmation information to the external device 201.

In the embodiment of the present invention, after the log processing apparatus 100 sends the first confirmation information to the external device 201 for the log space in the write completion state, it indicates that the log writing in the log space is successful, and the log space is at this time. Write success status.

For details, referring to FIG. 6, after the log processing apparatus 100 transmits the first confirmation information to the external device 201 for each of the log spaces 1, 2, 3, and 5 in the write completion state in the sliding window 200, the log space is 1, 2, 2, 3 and 5 are in the write success state, and the log spaces 8, 10, 11, and 13 outside the sliding window 200 are still in the write completion state.

Step S104: The log processing device 100 determines whether the initial log space of the sliding window 200 is in the write success state. If yes, step S105 is performed, and if no, step S106 is performed.

As shown in FIG. 6, the initial log space 1 of the sliding window 200 is in a write success state, and the log processing apparatus 100 executes step S105.

Step S105: The log processing apparatus 100 slides the sliding window 200 to the end log space of the log segment 300 until the new starting log space of the sliding window 200 is in another state of the non-writing success state, or up to the end of the sliding window 200. The log space coincides with the end position space of the log segment 300.

For example, referring to FIG. 7, FIG. 7 is a schematic diagram of a log segment 300 in a fourth state according to an embodiment of the present invention. As shown in FIG. 7, the sliding window 200 slides to the end log space 16 of the log end until sliding. The new start log space 4 of the window 200 is in another state of the non-write success state, and the sliding window 200 stops sliding, wherein the new start log space 4 of the sliding window 200 is in the unwritten state.

In other examples, other states of the non-write success state may also be write completion states.

After the sliding window 200 stops sliding, the log processing apparatus 100 jumps to step S103 to perform similar processing for the log space within the new sliding window 200.

Specifically, the log processing apparatus 100 transmits the first confirmation information to the external device 201 for each of the log spaces 8 and 10 in the write completion state in the new sliding window 200, and is in the write completion state for the outside of the sliding window 200. Each of the log spaces 11 and 13 does not transmit the first confirmation information to the external device 201.

8 is a schematic diagram of a log segment 300 in a fifth state, as shown in FIG. 8, after the above processing, the log spaces 5, 8 in the new sliding window 200 and 10 are in a write success state.

At this time, the log processing apparatus 100 may jump to step S104 to determine whether the initial log space 4 of the new sliding window 200 is in a write success state, and as shown in FIG. 8, the log space 4 is in an unwritten state. If the result of the determination is NO, the log processing apparatus 100 jumps to step S106.

Step S106: The log processing apparatus 100 writes invalid data on the initial log space of the sliding window 200, transmits the second confirmation information to the external device 201 for the initial log space of the sliding window 200, and waits for the sliding window 200 to include the sliding window. Log writes are completed in other log spaces outside the starting log space of 200. The second confirmation information is used to notify the external device 201 that the log write failed.

Reference may be made in conjunction with FIG. 9, which is a schematic diagram of a log segment 300 in a sixth state, as shown in FIG. 9, the log processing device 100 may be in a new starting log space 4 of the sliding window 200. The invalid data is written on, so that the log space 4 is in the write failure state, and waits for other log spaces 6, 7, and 9 other than the start log space 4 of the sliding window 200 in the sliding window 200 to complete the log writing.

Optionally, for the log space that is in the unwritten state in the sliding window 200, the log processing device 100 may wait for a preset time after writing the log to the log, and if the preset time is exceeded, the invalid data is directly written. .

For example, for the log space 4, the log processing device 100 counts the log space 4 after writing the log space. If the preset time is exceeded, the invalid data is directly written in the log space 4 in the sliding window 200, so that the log space 4 is written. The failure state is entered, and similar processing can be performed for the log spaces 6, 7, and 9.

In the embodiment of the present invention, the log processing apparatus 100 may further write invalid data in the log space after the sliding window 200, so that the log space after the sliding window 200 is in a write failure state.

Here, the log processing apparatus 100 completes the log write operation of the current log segment 300, and the log space 1, 2, 3, 5, 8, 10 of the current log segment 300 records a valid log, and other log spaces record invalid. data.

In the embodiment of the present invention, since the sliding window 200 is used, and the first confirmation information is transmitted to the external device 201 for the log space in the writing completion state in the sliding window 200, the log is in the writing completion state for the sliding window 200. The space does not send the first confirmation information to the external device 201, and the log space before the sliding window 200 is ensured to be in a write success state, so as to avoid the occurrence of a successful write log and a write failure log stored in the log segment 300. It can guarantee the efficiency of log reading.

Please refer to FIG. 10, which is another flowchart of a log processing method according to an embodiment of the present invention. Figure 10 shows a specific manner of reading the data segments generated by Figure 2.

As shown in FIG. 10, the log processing method according to the implementation of the present invention further includes:

Step S201: The log processing apparatus 100 reads the initial log space of the log segment 300.

Step S202: The log processing apparatus 100 determines whether the invalid data is first read in the log segment 300, and if so, executes step S203, and if no, executes step S204.

Step S203: The log processing apparatus 100 reads K-1 log spaces located after the log space having invalid data one by one.

Step S204: The log processing apparatus 100 reads the next log space of the current log space, and jumps to step S202.

In summary, the log processing apparatus 100 reads each log space of the log segment 300 one by one from the initial log space of the log segment 300 to the end log space of the log segment 300. When the invalid data is read for the first time, the read one by one is located. K-1 log space after the log space of invalid data, thus completing the log end reading.

It can be understood in conjunction with FIG. 9. In step S201, the log processing apparatus 100 first reads the initial log space 1 of the log end, and determines whether the data read in the initial log space 1 is invalid data. According to FIG. 9, the log is known. The data of the space 1 is a valid log, so the process goes to step S204, the log processing apparatus 100 reads the next log space 2 of the log space 1, and performs a similar loop, thereby respectively reading the log space 1 in which the valid log is stored. Up to 3, until the log space 4 is read, since the log space 4 is written as invalid data, the log processing apparatus 100 jumps to step S203, and the log processing apparatus 100 reads the log space after the log space having invalid data one by one. - 1 log space, since K = 7 in the embodiment shown in Fig. 9, the log processing apparatus 100 reads the 6 log spaces 5 to 10 after the log space 4 having invalid data one by one, when in the log space When 4, 6, 7, and 9 read invalid data, the invalid data read is directly discarded. When valid logs are read in log spaces 5, 8, and 10, valid logs are recorded. Thus all valid logs are read in log spaces 1, 2, 3, 5, 8, and 10 of log segment 300.

In summary, after the embodiment of the present invention writes the log to the log segment 300 concurrently based on the sliding window 200, since the log space before the sliding window 200 is in the write success state, the success can be successfully read from the log segment 300. All the logs written can effectively ensure the reading efficiency.

It should be noted that, in this embodiment, the sliding window 200 is a fixed-length sliding window, that is, K is a constant, and the value of the K value can be set according to actual needs.

However, it should be understood that in other examples, the sliding window 200 employed may be a variable length sliding window.

Specifically, K is a variable, and K=Kn, where n is the number of slides of the sliding window 200, and Kn indicates that the sliding window 200 covers the Kn log spaces after the nth sliding, wherein K0=M/10, and:

When N>=4,

When N<4,

和

采用向上取整。 It is worth noting that in other examples, it is also possible

with

Use rounding up.

By dynamically adjusting the size of the sliding window 200 under different load pressures, the concurrent write performance of the log system can be effectively improved, and the efficiency of log reading can be improved.

In summary, in the embodiment of the present invention, since the sliding window 200 is used, and the first confirmation information is transmitted to the external device 201 for the log space in the writing completion state in the sliding window 200, the writing is performed outside the sliding window 200. The log space of the completion state does not transmit the first confirmation information to the external device 201, and since the sliding window covers a larger range with respect to a single log space, the loss of write concurrency can be reduced. Further, since the log space before the sliding window 200 is in the write success state, the efficiency of the log reading can be ensured.

Referring to FIG. 11 , FIG. 11 is a schematic structural diagram of a device of a log processing apparatus according to an embodiment of the present invention. As shown in FIG. 11 , the log processing apparatus 100 includes:

The log segment application module 1001 is configured to apply for the log segment 300, and the log segment 300 includes M log spaces arranged in order;

The log writing module 1002 is configured to concurrently write N logs to the M log spaces, where N≤M, and N logs are generated by the external device 201;

The sliding window module 1003 is configured to slide from the initial log space of the log segment 300 to the end log space of the log segment 300 using the sliding window 200, wherein the sliding window 200 covers K log spaces, K<M;

The confirmation information sending module 1004 is configured to separately send the first confirmation information to the external device 201 for each log space in the writing completion state in the sliding window 200, so that each log space in the writing completion state in the sliding window 200 is in the In the write success state, the first confirmation information is not sent to the external device 201 for each log space in the write completion state outside the sliding window 200;

The first confirmation information is used to notify the external device 201 that the log writing is successful.

Optionally, the sliding window module 1003 is specifically configured to:

When the initial log space of the sliding window 200 is in the write success state, the sliding window 200 is slid to the end log space of the log segment 300 until the new starting log space of the sliding window 200 is in a non-writing success state. status.

Alternatively, K is a variable, and K=Kn, where n is the number of slides of the sliding window 200, and Kn indicates that the sliding window 200 covers the Kn log spaces after the nth sliding, wherein K0=M/10, and:

When N>=4,

When N<4,

Optionally, the confirmation information sending module 1004 is further configured to:

When the initial log space of the sliding window 200 is in another state of the non-write success state, invalid data is written on the start log space of the sliding window 200, and after the invalid data is written, the start log for the sliding window 200 is written. The space sends the second confirmation information to the external device 201, and waits for other log spaces in the sliding window 200 except the starting log space of the sliding window 200 to complete the log writing, wherein the second confirmation information is used to notify the external device 201 to write the log. Failure failed.

Optionally, the log processing apparatus 100 further includes a log reading module, and a log reading module, configured to:

Each log space of the log segment 300 is read one by one from the initial log space of the log segment 300 to the end log space of the log segment 300;

When invalid data is first read, K-1 log spaces located after the log space with invalid data are read one by one.

Referring to FIG. 12, FIG. 12 is a schematic structural diagram of a device according to an embodiment of the present invention. As shown in FIG. 12, the server 10 includes a processor 1021, a memory 1022, and a bus 1023. The processor 1021 and the memory 1022 are respectively connected to the bus. 1023 is connected, the memory 1022 stores program instructions, and the processor executes the program instructions to cause the server 10 to implement the functions of the log processing apparatus 100 disclosed above.

In summary, in the embodiment of the present invention, since the sliding window 200 is used, and the first confirmation information is transmitted to the external device 201 for the log space in the writing completion state in the sliding window 200, the writing is performed outside the sliding window 200. The log space of the completion state does not transmit the first confirmation information to the external device 201, and since the sliding window covers a larger range with respect to a single log space, the loss of write concurrency can be reduced. Further, since the log space before the sliding window 200 is in the write success state, the efficiency of the log reading can be ensured.

It should be noted that the processor in the embodiment of the present application may be a central processing unit (English: central processing unit, abbreviated as CPU), a network processor (English: network processor, abbreviated as NP), or a combination of a CPU and an NP. The processor may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (abbreviated as PLD), or a combination thereof. The above PLD can be a complex programmable logic device (English: complex programmable logic device, abbreviation: CPLD), field-programmable gate array (English: field-programmable gate array, abbreviation: FPGA), general array logic (English: generic array Logic, abbreviation: GAL) or any combination thereof.

It should be noted that the memory in the embodiment of the present application may include a volatile memory (English: volatile memory), such as random-access memory (abbreviation: RAM); the memory may also include non-volatile memory. Non-volatile memory (English: non-volatile memory), such as read-only memory (English: read-only memory, abbreviation: ROM), flash memory (English: flash memory), hard disk (English: hard disk drive, abbreviation: HDD) Or a solid state drive (English: solid-state drive, abbreviated: SSD); the memory may also include a combination of the above types of memory.

Based on the same inventive concept, the present application provides a computer readable storage medium having instructions stored therein that, when run on a computer, cause the computer to perform any of the possible implementations of the above-described embodiments Said method.

Based on the same inventive concept, the present application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method described in any of the possible implementations of the above-described embodiments.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer usable memory channels (including but not limited to disk storage and optical storage, etc.) in which computer usable program code is embodied.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. The computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine instruction for generating instructions executed by a processor of a computer or other programmable data processing device Means for implementing the functions specified in one or more flows of the flowchart or in a block or blocks of the flowchart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

It will be apparent to those skilled in the art that various modifications and changes can be made in the present application without departing from the spirit and scope of the application. Thus, it is intended that the present invention cover the modifications and variations of the present invention.

Claims (11)

A log processing method, comprising: Applying a log segment, the log segment includes M log spaces arranged in order; Writing N logs concurrently to the M log spaces, where N≤M, and the N logs are generated by an external device; Using a sliding window to slide from the starting log space of the log segment to the end log space of the log segment, wherein the sliding window covers K log spaces, K<M; And sending, to the external device, the first confirmation information to each of the log spaces in the write completion state in the sliding window, so that each log space in the write completion state in the sliding window is in a write success state, Not sending the first confirmation information to the external device for each log space in the write completion state outside the sliding window; The first confirmation information is used to notify the external device that the log writing is successful. The method according to claim 1, wherein the sliding from the starting log space of the log segment to the end log space by using a sliding window comprises: When the initial log space of the sliding window is in a write success state, sliding the sliding window to the end log space of the log segment until the new starting log space of the sliding window is not written Other states of the success state. The method according to claim 1 or 2, wherein said K is a variable, and said K = Kn, wherein n is the number of slides of said sliding window, and Kn is said that said sliding window performs an nth sliding After covering Kn log spaces, where K0=M/10, and: When N>=4,

When N<4,

The method according to any one of claims 1 to 3, wherein the method further comprises: Writing invalid data on a start log space of the sliding window when the initial log space of the sliding window is in another state of a non-write success state, after the invalid data is written, for the sliding The start log space of the window sends a second confirmation message to the external device, and waits for other log spaces in the sliding window except the start log space of the sliding window to complete log writing, wherein the second The confirmation information is used to notify the external device that the log write failed. The method of claim 4, wherein the method further comprises: Reading each log space of the log segment one by one from the initial log space of the log segment to the end log space of the log segment; When the invalid data is read for the first time, K-1 log spaces located after the log space having the invalid data are read one by one. A log processing device, comprising: The log segment application module is configured to apply for a log segment, where the log segment includes M log spaces arranged in order; a log writing module, configured to concurrently write N logs to the M log spaces, where N≤M, and the N logs are generated by an external device; a sliding window module, configured to slide from a starting log space of the log segment to an end log space of the log segment by using a sliding window, wherein the sliding window covers K log spaces, K<M; a confirmation information sending module, configured to respectively send first confirmation information to the external device for each log space in the writing completion state in the sliding window, so that each log in the writing completion state is in the sliding window The space is in a write success state, and the first confirmation information is not sent to the external device for each log space in the write completion state outside the sliding window; The first confirmation information is used to notify the external device that the log writing is successful. The device according to claim 6, wherein the sliding window module is specifically configured to: When the initial log space of the sliding window is in a write success state, sliding the sliding window to the end log space of the log segment until the new starting log space of the sliding window is not written Other states of the success state. The apparatus according to claim 6 or 7, wherein said K is a variable, and said K = Kn, wherein n is the number of times of sliding of said sliding window, and Kn is said that said sliding window performs an nth sliding After covering Kn log spaces, where K0=M/10, and: When N>=4,

When N<4,

The device according to any one of claims 6 to 8, wherein the confirmation information sending module is further configured to: Writing invalid data on a start log space of the sliding window when the initial log space of the sliding window is in another state of a non-write success state, after the invalid data is written, for the sliding The start log space of the window sends a second confirmation message to the external device, and waits for other log spaces in the sliding window except the start log space of the sliding window to complete log writing, wherein the second The confirmation information is used to notify the external device that the log write failed. The device according to claim 9, further comprising a log reading module, wherein the log reading module is configured to: Reading each log space of the log segment one by one from the initial log space of the log segment to the end log space of the log segment; When the invalid data is read for the first time, K-1 log spaces located after the log space having the invalid data are read one by one. A server, comprising a memory and a processor, the memory storing program instructions, the processor executing the program instructions to cause the server to perform the steps of claims 1 to 5.

Images