RU2789621C1 - E-mail processing method - Google Patents

Abstract

FIELD: computers, telecommunications and information security.

SUBSTANCE: invention relates to the field of computers, telecommunications and information security. E-mail processing method includes receiving e-mail messages from clients and placing them in the mail server queue, determining parameters of the received message, performing the mail server occupation dynamic evaluation, in which the percentage of the mail server queue occupation P is calculated, and the following shall be taken into account: if the mail server queue occupation P percentage is equal or greater than P1, then the system shall stop receiving messages from all clients and automatic message on the reasons of receiving blockage shall be generated and sent to all clients, if the mail server queue occupation P percentage is less than P1 but equal or greater than P2, then the clients whose messages resulted in the server occupation P increase, the automatic notification shall be created and sent to the clients, and the message processing shall be continued, if the mail server queue occupation P percentage is less than P2, the disconnected clients check shall be performed, if disconnected clients are detected, they shall be reconnected, automatic notification shall be created and sent to the clients, and the message processing shall be continued, and if no disconnected clients are detected, the message processing still shall be continued, P2 shall be calculated based on the clients activity statistics received in previous messages.

EFFECT: invention provides possibility to increase the mail server stability by reducing the probability of the "Denial of Service" occurrence caused by directing the information flow from customers exceeding the server's processing capacity.

1 cl, 1 dwg, tbl

Description

FIELD OF TECHNOLOGY

The invention relates to the computer field, the field of telecommunications and information security. It is used on mail servers when processing information that has cryptographic protection in order to reduce the likelihood of a Denial of Service threat.

BACKGROUND OF THE INVENTION

In the prior art, there is a method for disaster recovery and management for an e-mail system (RU 2395116, 20.07.2010) with a plurality of user accounts, containing the steps at which: route e-mail messages to one or more servers that receive, process and transmitting email messages directed to the client email system; detecting a client email system failure; automatically create a plurality of e-mail mailboxes corresponding to said plurality of user accounts in response to detection of a failure in the client e-mail system, and route e-mail messages directed to the client e-mail system to said one or more e-mail mailboxes until the outage has been resolved, with the multiple mailboxes created being available to the respective user accounts, respectively, for the duration of the client e-mail system failure.

The disadvantage of this method is the complex structure of e-mail redistribution in the event of a denial of service. Also, this method does not provide the ability to analyze the previous actions of subscribers, which will allow to determine the potential occurrence of a denial of service of the server caused by its overflow.

In the prior art, a method is known for managing mail on a mail receiving server (US20100235435A1, 09/02/2014) containing a mail receiving unit configured to perform mail receiving processing, a first mailbox and a second mailbox for storing mail in accordance with each mailing address, and storage space for mail that cannot be stored in the first mailbox, the method comprising the steps of determining, at the time of receiving mail by the mail receiving unit, whether there is a free capacity capable of storing received mail in the first mailbox corresponding to the mailing address specified in destination for delivery of received mail; storing the mail as received mail with an error in the storage area when it is determined in the determination step that the first mailbox does not have free storage capacity; extracting header information and body information from the received error message stored in the storage space in the storage step; creating an information notification message for reporting information regarding the receipt of the error message to the mailing address described in the delivered destination of receiving the error message using header information and body information extracted by the extraction unit; and storing the information notification mail generated in the information notification mail management step in the second mailbox.

The disadvantage of this method is the complex structure of e-mail redistribution in the event of a denial of service. Also, this method does not provide the ability to analyze the previous actions of subscribers, which will allow to determine the potential occurrence of a denial of service of the server caused by its overflow.

None of these methods uses dynamic evaluation of the state of the server queue, based on the results of the analysis made using hidden Markov models and/or artificial intelligence. Also, the solutions known in the prior art are applicable in global public networks, and for local internal mail exchange they are redundant and complex.

DISCLOSURE OF THE INVENTION

The task to be solved by the claimed solution is to eliminate the shortcomings identified in the prior art.

The technical result of the claimed invention is to increase the stability of the mail server by reducing the likelihood of a "Denial of Service" threat caused by the direction of the information flow from clients exceeding the capabilities of the mail server to process it.

The claimed technical result is achieved due to the fact that the method of processing e-mail includes receiving electronic messages from clients and placing them in the mail server queue, determining the parameters of the received message, conducting a dynamic assessment of the occupancy of the mail server, in which the percentage of occupancy of the mail server queue P is calculated, wherein:

a) if the percentage of occupancy of the mail server queue P is equal to or greater than Р ₁ , then the reception of messages from all clients is stopped and an automatic message is generated to all clients about the reasons for blocking reception,

b) if the percentage of occupancy of the mail server queue P is less than P ₁ , but equal to or higher than the specified value P ₂ , then specific clients are disconnected, whose messages led to an increase in the percentage of occupancy of the mail server P, an automatic notification is generated and sent, and message processing continues,

c) if the percentage of occupancy of the mail server queue P is less than the specified value P ₂ , then a check is made for the presence of disconnected clients, if there are any, they are included in the reception, they are sent automatic messages about the resumption of reception, message processing continues,

at the same time, in the absence of disconnected clients, message processing continues, while

P ₂ is determined on the basis of statistical data on the activity of clients received on previous messages.

BRIEF DESCRIPTION OF THE DRAWINGS

The figure shows a flowchart of a method for processing e-mail.

IMPLEMENTATION OF THE INVENTION.

The claimed method is applicable to systems that represent a strictly defined number of mail servers, in strictly defined conditions of use, each mail server can have no more than 300 clients. The method is used in the conditions of using e-mail based on various protocols (POP3, SMTP, IMAP, X.400, etc.) and where the body of the letter and its attachments are not subject to analysis due to the cryptographic transformations that they underwent when sending. In the claimed method, only the local state of the server is considered, without taking into account the states of clients and other servers. Also, the condition is the presence of all network users as trusted, which is confirmed by an electronic signature.

The method for processing e-mail includes receiving electronic messages from clients and placing them in a mail server queue, determining the parameters of the received message, performing a dynamic evaluation of the mail server occupancy, in which the percentage of occupancy of the mail server queue P is calculated and compared with the parameters P ₁ and P ₂ . The percentage of fullness of the queue of the mail server P is calculated after each message entering the queue, that is, upon the fact of the event.

If the percentage of occupancy of the mail server queue P is equal to or greater than P ₁ , then receiving messages from all clients is stopped and an automatic message is generated to all clients about the reasons for blocking reception. If the occupancy percentage of the mail server queue P is less than P ₁ , but equal to or higher than the specified value P ₂ , then specific clients whose messages led to an increase in the occupancy percentage of the mail server P are disconnected, an automatic notification is generated and sent, and message processing continues. If the occupancy percentage of the mail server queue P is less than the specified value P ₂ , then a check is made for the presence of disconnected clients, if there are any, they are included in the reception, they are sent automatic messages about the resumption of reception, message processing continues.

If there are no disconnected clients, message processing continues.

P ₁ is fixed and determined by artificial intelligence based on statistical data for the year, while it can be corrected by a person.

Example:

The maximum size of a single message from client 1, 2 and 3 is 40 MB (41943040 bytes), therefore, when sending messages at once, they will create a load of 3x41943040 bytes = 125829120 bytes.

The maximum size of a single message from client 4, and 5 is 20 MB (20971520 bytes), therefore, when sending messages at a time, they will create a load of 2x 20971520 bytes = 41943040 bytes.

The maximum size of a single message from client 6, 7 and 8 is 90 MB (94371840 bytes), therefore, when sending messages at once, they will create a load of 3x 94371840 bytes = 283115520 bytes.

The maximum size of a single message from clients 9-18 is 95 MB (99614720 bytes), therefore, when sending messages at once, they will create a load of 10x 99614720 bytes = 996147200 bytes.

The maximum size of a single message from client 19-58 is 55 MB (57671680 bytes), therefore, if they send messages at once, they will create a load of 40x 57671680 bytes = 2306867200 bytes.

When sending one message at a time, from all subscribers there will be free space in the queue (4624616384 bytes (server queue size) - 3753902080 bytes (total volume of sent messages) = 870714304 or 81.2% of the queue fullness status. Based on this, we make the assumption that the probability of the server transitioning to a denial of service threat state is 0.812 (queue state in percent divided by 100).

P ₂ is determined on the basis of statistical data on the activity of clients received on previous messages (statistics on sent messages).

Clients send different mail messages with different initial values (different number of recipient addresses, since multicast messages are no exception and different amount of attachments)

After processing messages that have accumulated in the queue, blocked requests from clients for receiving messages are analyzed; if there are any, messages from these clients are processed first, and if there are none, messages are received. The analysis is performed in the following way - in accordance with entries in the system logs - clients are identified who, during the specified period, requested the server to receive mail. Messages containing the address of the sender or recipient receive the highest priority when processed in the server's queue.

Table 1 presents all the coefficients obtained from the statistical information provided by the server system logs.

V _total - the volume in bytes of all messages in the queue in the specified period of time;

λ is the intensity of the message flow, this is the ratio of V _total to the volume of the mail server queue (the value is fixed and is set when installing the software)

V _max - the maximum volume of one message recorded in the specified period of time, in bytes; Table 1 shows an example for an hourly period, and it is used by default, if necessary, corrected.

V _min - the minimum volume of one message recorded in the specified period of time, in bytes;

V _Wednesday - the average volume of one message recorded in the specified period of time, in bytes;

n messages of equal volume, this is the number of messages of the same volume recorded in the specified period of time;

N _total - the total number of messages recorded in the specified period of time;

N _errors - the number of messages, the recipient's address of which is not in the lists of the mail server, fixed in the specified period of time;

N _ext - the number of messages intended for the internal domain zone fixed in the specified period of time;

N _ext - the number of messages intended for the external domain zone fixed in the specified period of time;

Nin/Ntotal - reserved

Next/Ntotal - reserved

Nerror/Ntotal - the corresponding relationship fixed in the specified period of time; reserved - corresponding to the indicators in the table

Server Queue Fill % - the amount of information in the mail server queue, in percent.

An example for artificial intelligence (AI).

Parameters signaling AI about the occurrence of a denial of service threat:

The following change in the indicators of the system in the aggregate is characteristic only for the implementation of the threat of denial of service:

λ - tend to 0;

V _total - tend to ∞;

_Vmax - does not change;

n messages of equal volume - tend to ∞;

N _total - tend to ∞;

N _osh - tend to ∞ provided that the server cannot determine the recipient of messages, otherwise it does not change;

N _vn - does not change;

N _external - tend to ∞.

The probability of a denial of service threat is equal to 1, provided that the server queue is 100% occupied or _Vtotal volume is equal to or the volume of the server queue, i.e. if the server's queue is full, then any processing on it becomes impossible, and it goes into the state of a realized denial of service threat.

For example, the server queue size is 4624616384 bytes.

Then when the queue is full at 4224616384 bytes

And when the queue is full at 3724616384 bytes

Probability calculation for stable operation for AI in order to determine normal behavior for clients.

Based on the analysis, it was revealed that the following clients systematically send information of the maximum volume:

CLIENT 10 - up to 200 days per year;

CLIENT 12 – up to 15 days per year;

CLIENT 15 – up to 50 days per year;

CLIENT 16 – up to 10 days per year;

CUSTOMER 45 - up to 150 days per year.

Accordingly, the total probability of sending messages of the maximum volume, in accordance with the general theory of probability, is - P (A) \u003d m / n, where n is the number of days in a year, and m is the number of days in which the maximum volume of information was sent. This information is necessary for artificial intelligence to understand which activity is normal for which client and which is abnormal.

It takes 45 100 MB messages to bring the server into a denial of service state. If the size of the mail server queue is 4624616384 bytes, then when 45 messages of 100 MB in size are sent to it at the same time, 45x100 (translated into megabytes) x1024 (translated into kilobytes) x1024 (translated into bytes) = 4718592000 bytes will get into the mail server queue. Which exceeds the capabilities of the mail server queue for storing information.

If the mail server queue size is 4624616384 bytes, then when 90 messages of 50 MB in size are sent to it at the same time, 90x50 (translated into megabytes) x1024 (translated into kilobytes) x1024 (translated into bytes) = 4718592000 bytes will be sent to the mail server queue. Which exceeds the capabilities of the mail server queue for storing information.

If the size of the mail server queue is 4624616384 bytes, then when 180 messages of 25 MB in size are sent to it at the same time, 180x25 (translated into megabytes) x1024 (translated into kilobytes) x1024 (translated into bytes) = 4718592000 bytes will get into the mail server queue. Which exceeds the capabilities of the mail server queue for storing information.

Based on the statistics for the years of operation, the typical load from each client is calculated (in the system for which this method is being developed, it is not supposed to use a new mail server with a new set of clients, in an exceptional case, the statistics of the head server and its clients will be taken as the basis, as the busiest.

If critical parameters from the table are changed, the likelihood of a denial of service threat increases.

List of critical parameters and their states that affect the occurrence of a denial of service threat.

λ - tend to 0;

V _total - tend to ∞;

_Vmax - does not change;

n messages of equal volume - tend to ∞;

N _total - tend to ∞;

N _osh - tend to ∞ provided that the server cannot determine the recipient of messages, otherwise it does not change;

N _vn - does not change;

N _ext - tend to ∞.

As shown in the example (see Table 1), out of 192 messages, 30 (30 messages of equal maximum size were recorded) were 115606184 bytes in size (about 110 MB), which led to the server queue overflow (server queue status + incoming volume - amounted to 4624616384 bytes, which corresponds to the queue size of the mail server).

This could have arisen for the following reasons: one client sent 30 messages of the specified volume within an hour, or 2 to 15, or one sent 2 messages, the second one, and the third 27, etc.

We can not exclude the situation when one sent 4 to 30, and the other two 60 to 35 - which will lead to a similar situation.

That is, in case of non-typical behavior of clients, as well as in the event of a denial of service threat, it is necessary to enable protective mechanisms, as was shown in the algorithm.

An example of a simple Hidden Markov Model (HMM)

N - server queue size

Where:

P(client 1) - the probability of a denial of service threat from client No. 1,

Р(client n) - probability of realizing a denial of service threat from a client
No.n,

Vtotal (client 1) the amount of information from client No. 1 for the period under review,

Vtotal (client n) volume of information from client No. n for the period under review

For example, the server queue volume is 4624616384 and there are three clients whose directed volume is known only after the fact and is not predictable from the previous point in time.

No threat of denial of service.

There is a denial of service threat and it comes from client 2

There is a denial of service threat coming from clients 2 and 3

The claimed method makes it possible to prevent the situation when the server stops processing messages and the system administrator has to manually clear the queue, restart the server and restore information exchange.

Claims (9)

An e-mail processing method including, receiving emails from clients and placing them in the mail server queue, determining the parameters of the received message, conducting a dynamic assessment of the occupancy of the mail server, in which the percentage of occupancy of the mail server queue P is calculated, while: a) if the percentage of occupancy of the mail server queue P is equal to or greater than Р ₁ , then the reception of messages from all clients is stopped and an automatic message is generated to all clients about the reasons for blocking reception, b) if the percentage of occupancy of the mail server queue P is less than Р ₁ , but equal to or higher than the specified value Р ₂ , then specific clients are disconnected, whose messages led to an increase in the percentage of occupancy of the mail server Р, an automatic notification is generated and sent, and message processing continues, c) if the percentage of occupancy of the mail server queue P is less than the specified value P ₂ , then a check is made for the presence of disconnected clients, if there are any, they are included in the reception, they are sent automatic messages about the resumption of reception, message processing continues, at the same time, in the absence of disconnected clients, message processing continues, while P ₂ is determined on the basis of statistical data on the activity of clients received on previous messages.

Images