CN112037818A

CN112037818A - Abnormal condition determining method and forward matching formula generating method

Info

Publication number: CN112037818A
Application number: CN202010891416.2A
Authority: CN
Inventors: 颜廷旭; 张佳林; 沙泓州
Original assignee: Beijing Didi Infinity Technology and Development Co Ltd
Current assignee: Beijing Didi Infinity Technology and Development Co Ltd
Priority date: 2020-08-30
Filing date: 2020-08-30
Publication date: 2020-12-04

Abstract

The embodiment of the application provides an abnormal condition determining method and a forward matching formula generating method, and relates to the technical field of traffic safety. The abnormal condition determining method obtains the audio information corresponding to the service to be detected and obtains the text information corresponding to the audio information. And then determining the correlation degree information of the text information and the abnormal condition according to the forward matching formula and the text information, wherein the forward matching formula has a forward relation with the abnormal condition, and the correlation degree information represents the correlation degree value of the text information and the abnormal condition. And when the correlation degree information is greater than the correlation degree threshold value, determining that the service to be detected has an abnormal condition. Therefore, whether abnormal conditions such as traffic accidents occur to the vehicle corresponding to the service to be detected can be effectively detected, and the accuracy of detecting the abnormal conditions such as the traffic accidents is improved.

Description

Abnormal condition determining method and forward matching formula generating method

Technical Field

The application relates to the technical field of traffic safety, in particular to an abnormal condition determining method and a forward matching formula generating method.

Background

The help seeking after the traffic accident and the accident is always the problem which is solved in the field of traffic safety, the traffic accident has the characteristics of high occurrence frequency and serious loss to human bodies and property after the occurrence, particularly, in the aspect of personal safety, human injuries occur slightly, and the serious phenomenon of death can be directly caused.

At present, audio analysis is usually performed on a sound recording file directly to detect whether a car accident occurs in a vehicle corresponding to the sound recording file. Or, whether the vehicle has the phenomena of sudden braking or collision or the like is judged according to data returned by the vehicle-mounted equipment, and whether a traffic accident occurs is further determined. However, the above methods all have a problem of low detection accuracy.

Disclosure of Invention

In view of the above, the present application provides an abnormal situation determination method and a method for generating a forward matching equation to improve the above problem.

The embodiment of the application can be realized as follows:

in a first aspect, an embodiment of the present application provides an abnormal situation determining method, where the method includes:

acquiring audio information corresponding to a service to be detected;

acquiring text information corresponding to the audio information;

determining the correlation degree information of the text information and the abnormal situation according to a forward matching formula and the text information, wherein the forward matching formula and the abnormal situation have a forward relation, and the correlation degree information represents the correlation degree value of the text information and the abnormal situation;

and when the correlation degree information is greater than a correlation degree threshold value, determining that the service to be detected has an abnormal condition.

In an alternative embodiment, the forward matching formula comprises at least one forward string, each of the forward strings having a forward association with the abnormal condition; the step of determining the information of the degree of correlation between the text information and the abnormal situation according to the forward matching formula and the text information comprises the following steps:

matching each forward character string with the text information respectively;

when the text information has a text which is successfully matched with the forward character string, taking the forward character string as a target character string;

obtaining forward scores corresponding to all the target character strings; the positive score corresponding to the target character string is positively correlated with the abnormal condition;

and taking forward scores corresponding to all the target character strings as the correlation degree information.

In an optional embodiment, the step of obtaining forward scores corresponding to all the target character strings includes:

obtaining a secondary numerical value of each target character string matched with the corresponding text and weight information of a forward character string corresponding to each target character string;

obtaining a weighted sum of all the target character strings according to all the secondary numerical values and the weight information;

and taking the weighted sum as the forward score.

In an optional implementation manner, before the step of obtaining the audio information corresponding to the service to be detected, the method further includes:

acquiring historical audio information corresponding to at least one historical trip;

converting each historical audio information into historical text information;

performing text splitting on the historical text information to obtain a plurality of historical character strings;

obtaining the initial weight of each historical character string and the abnormal condition;

obtaining at least one forward character string from the historical character strings, wherein the initial weight of the forward character string is greater than or equal to a first preset threshold value;

and taking all the forward character strings as the forward matching expression, and obtaining the weight information of each forward character string according to the initial weight of all the forward character strings, wherein the initial weight is positively correlated with the weight information.

In an optional embodiment, when the forward matching formula includes at least one forward character string, the forward matching formula further includes logical association information between the respective forward character strings.

In an optional implementation manner, the historical text information includes positive text information and negative text information, the positive text information represents that an abnormal condition occurs, and the negative text information represents that no abnormal condition occurs;

the step of obtaining the initial weight of each history character string and the abnormal condition comprises the following steps:

and obtaining an initial weight of each historical character string and the abnormal situation based on a first frequency and a second frequency, wherein the first frequency is a frequency value of each historical character string appearing in the positive text information, and the second frequency is a frequency value of each historical character string appearing in the negative text information.

In an alternative embodiment, the initial weight, the first frequency and the second frequency satisfy the following formula:

the PMI (w, c ═ 1) is an initial weight of the w-th history string and the abnormal condition, p (w | c ═ 1) is a first frequency corresponding to the w-th history string, and p (w | c ═ 0) is a second frequency corresponding to the w-th history string.

In an alternative embodiment, the step of obtaining at least one forward string from the history strings comprises:

obtaining at least one intermediate character string from the historical character string, wherein the initial weight of the intermediate character string is greater than or equal to the first preset threshold;

and acquiring at least one forward character string from all the intermediate character strings, wherein each character included in each forward character string is different from each character included in other forward character strings.

In an optional implementation manner, the obtaining, according to initial weights of all the forward character strings, weight information of each forward character string, where the step of positively correlating the initial weights with the weight information includes:

obtaining the intermediate initial weight of each forward character string and the abnormal condition;

and taking the intermediate initial weight as corresponding weight information.

In an optional implementation manner, before the step of obtaining the text information corresponding to the audio information, the method further includes:

and deleting the voice navigation information in the audio information.

In a second aspect, an embodiment of the present application provides a forward matching formula generation method, where the method includes:

converting each historical audio information into historical text information;

In an optional implementation manner, the step of obtaining the weight information of each forward character string according to the initial weights of all the forward character strings includes:

obtaining the final weight of each forward character string and the abnormal condition;

and taking the final weight as corresponding weight information. In a third aspect, an embodiment of the present application provides an abnormal situation determination apparatus, where the apparatus includes:

the acquisition module is used for acquiring the audio information corresponding to the service to be detected;

the acquisition module is further used for acquiring text information corresponding to the audio information;

the determining module is used for determining the correlation degree information of the text information and the abnormal condition according to the forward matching formula and the text information; the forward matching expression has a forward relation with the abnormal condition; the correlation degree information represents a correlation degree value of the text information and the abnormal condition;

the determining module is further configured to determine that the service to be detected is abnormal when the correlation degree information is greater than a correlation degree threshold.

In an alternative embodiment, the forward matching formula comprises at least one forward string, each of the forward strings having a forward association with the abnormal condition;

the determining module is configured to match each forward character string with the text information respectively;

In an optional embodiment, the determining module is configured to obtain a secondary value of each target character string matching with a corresponding text, and weight information of a forward character string corresponding to each target character string;

and taking the weighted sum as the forward score.

In an optional embodiment, the abnormal situation determination apparatus further comprises:

and the deleting module is used for deleting the voice navigation information in the audio information.

In a fourth aspect, an embodiment of the present application provides a forward matching formula generating apparatus, where the forward matching formula generating apparatus includes:

the audio information acquisition module is used for acquiring historical audio information corresponding to at least one historical trip;

the conversion module is used for converting each historical audio information into historical text information;

the splitting module is used for performing text splitting on the historical text information to obtain a plurality of historical character strings;

the weight obtaining module is used for obtaining the initial weight of each historical character string and the abnormal condition;

the character string obtaining module is used for obtaining at least one forward character string from the historical character strings, wherein the initial weight of the forward character string is greater than or equal to a first preset threshold value;

and the forward matching formula acquisition module is used for taking all the forward character strings as the forward matching formulas and obtaining the weight information of each forward character string according to the initial weights of all the forward character strings, wherein the initial weights are positively correlated with the weight information.

the weight obtaining module is configured to obtain an initial weight of each historical character string and an abnormal condition based on a first frequency and a second frequency, where the first frequency is a value of the number of occurrences of each historical character string in the positive text information, and the second frequency is a value of the number of occurrences of each historical character string in the negative text information.

In an optional embodiment, the character string obtaining module is configured to obtain at least one intermediate character string from the history character string, where a forward score of the intermediate character string is greater than or equal to the first preset threshold;

In an optional embodiment, the forward matching formula obtaining module is configured to obtain a final weight of each forward character string and an abnormal condition;

and taking the final weight as corresponding weight information.

In a fifth aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the memory communicate with each other through the bus, and the processor executes the machine-readable instructions to perform the steps of the abnormal situation determination method according to any one of the foregoing embodiments; or performing the steps of the forward matching formula generating method described in any one of the preceding embodiments.

In a sixth aspect, an embodiment of the present application provides an abnormal situation determination system, including a service provider and a client that communicate with each other;

the client is used for acquiring audio information corresponding to the service to be detected;

the service provider is used for acquiring the audio information, acquiring text information corresponding to the audio information, and determining correlation degree information between the text information and an abnormal condition according to a forward matching formula and the text information, wherein the forward matching formula and the abnormal condition have a forward relation, the correlation degree information represents a correlation degree value between the text information and the abnormal condition, and when the correlation degree information is greater than a correlation degree threshold value, it is determined that the service to be detected is abnormal.

In a seventh aspect, an embodiment of the present application provides a readable storage medium, where a computer program is stored, and when the computer program is executed, the foregoing abnormal situation determination method is implemented; or implementing the forward matching formula generation method described in any of the foregoing embodiments.

In view of this, the embodiment of the present application provides an abnormal situation determination method, an abnormal situation determination device, an electronic device, and a readable storage medium, in which the method determines the degree of correlation between text information and an abnormal situation by acquiring audio information corresponding to a service to be detected, acquiring text information corresponding to the audio information, and using a forward matching formula having a forward relationship with the abnormal situation, so as to effectively detect whether an abnormal situation such as a traffic accident occurs in a vehicle corresponding to the service to be detected, and improve the accuracy of detecting the abnormal situation such as the traffic accident.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, several embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application.

Fig. 2 is a block diagram of a service provider according to an embodiment of the present disclosure.

Fig. 3 is a schematic flow chart of a forward matching formula generation method according to an embodiment of the present application.

Fig. 4 is a schematic flowchart of a sub-step of step S103 in fig. 3 according to an embodiment of the present application.

Fig. 5 is a schematic flowchart of a sub-step of step S104 in fig. 3 according to an embodiment of the present disclosure.

Fig. 6 is a flowchart of an abnormal situation determination method according to an embodiment of the present application.

Fig. 7 is a schematic flowchart of a sub-step of step S202 in fig. 6 according to an embodiment of the present disclosure.

Fig. 8 is a functional module schematic diagram of an abnormal situation determination apparatus according to an embodiment of the present application.

Icon: 1-a request terminal; 2-a server; 3-a vehicle; 22-a memory; 21-a processor; 24-abnormal situation determination means; 241-an obtaining module; 242-a determination module; 25-forward matching formula generating means; 26-bus.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.

It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.

The help seeking after the traffic accident and the accident is always the problem which is solved in the field of travel safety, the traffic accident has the characteristics of high occurrence frequency and serious loss to human bodies and property after the occurrence, particularly, in the aspect of personal safety, personal injury is slightly caused, and the serious phenomenon of death can be directly caused.

Currently, audio analysis is generally performed on a sound file directly, for example, voiceprint information or spectrum information in the sound file is extracted to detect whether a car accident occurs. Or, whether the vehicle has the phenomena of sudden braking or collision or the like is judged through the vehicle-mounted equipment, and whether a traffic accident occurs is further determined. However, the above methods all have a problem of poor detection accuracy due to a large amount of noise in the recording file.

The above process of determining whether an accident occurs through recording is described below by taking a network car booking scene as an example. As shown in fig. 1, fig. 1 is a schematic view of an application scenario provided in the embodiment of the present application. The network appointment scene comprises a request terminal 1, a server 2 and a vehicle 3. The request terminal 1, the server 2, and the vehicle 3 establish communication connection with each other via a network.

The request terminal 1 may be a terminal held by a user, and the vehicle receives the order of the network car booking service through the server, so as to provide the network car booking service for the user and send the user to a destination.

The number of the request terminals 1 may be plural, and the number of the vehicles 3 may also be plural. The request terminal 1 sends an order request to the server 2, wherein the order request carries the current location information of the request terminal 1 and the location information of the destination to which the user of the request terminal 1 intends to go.

After obtaining the order request, the server 2 selects a service vehicle from a plurality of vehicles within a preset distance range from the request terminal 1 based on the position information of the request terminal 1, and sends a service order containing the navigation information of the request terminal to the service vehicle, so that the service vehicle goes to the location of the request terminal through the navigation information. Further, after receiving the user of the request terminal, the service vehicle sends the user of the request terminal to a destination to which the service vehicle is going.

In the course of the service vehicle traveling to the point where the request terminal 1 is located through the navigation information, there is a possibility that an abnormal situation such as a traffic accident may occur, or in the course of the service vehicle sending the user corresponding to the request terminal 1 to the destination, there is a possibility that an abnormal situation such as a traffic accident may occur.

When abnormal conditions such as traffic accidents occur, rescues are timely provided for drivers and passengers in the service vehicles, so that life or property loss can be greatly recovered, and therefore, it is very important to judge whether abnormal conditions such as traffic accidents occur in the service vehicles.

The following provides one possible implementation, which exemplifies the current rescue method. The service vehicle collects sound in the whole running process and sends collected audio data to the server. And the server analyzes the abnormal accident based on the frequency spectrum information or the voiceprint information in the audio data, and finds out the collision sound, the scream sound or the crying sound. And further determining whether the service vehicle has an abnormal accident such as a traffic accident.

However, the above-mentioned analysis methods for directly performing audio analysis on a recording file to detect whether an abnormal condition occurs all analyze the characteristics of the sound itself, and all have the problem of poor detection accuracy.

In view of this, the present application provides an abnormal situation determination system. As a possible implementation manner, the embodiment of the present application takes the application of the system to the above-mentioned car booking scenario as an example, and details of the abnormal situation determination system are described.

The abnormal situation determination system comprises a client and a service provider which are communicated with each other. The client may be a terminal used by a driver of the vehicle or a terminal used by a passenger riding the vehicle in the online booking scenario, for example, the request terminal 1 in fig. 1, in which service software is installed, and after audio information is collected by the terminal, the collected audio information is sent to the service provider through the service software. The service provider may be the server 2 in fig. 1.

The client acquires audio information in the driving process of the vehicle and sends the acquired audio information to the service provider.

And the service provider is used for extracting the key information in the audio information, analyzing and processing the key information and the abnormal condition by utilizing the mapping relation between the key information and the abnormal condition to obtain the correlation degree between the key information and the abnormal condition, and further determining whether the abnormality occurs in the service process corresponding to the audio information.

In combination with the scheme provided by the above abnormal situation determination system, the service provider provided in the embodiment of the present application is taken as the server 2 in fig. 1 as an example, and the server 2 is further described below.

Referring to fig. 2, fig. 2 is a block diagram of a server according to an embodiment of the present disclosure, where the server may be the server 2 in fig. 1.

In an implementation manner, the abnormal situation determination method and/or the forward matching formula generation method provided by the embodiment of the present application are applied to the server 2 shown in fig. 2, and the server 2 executes the abnormal situation determination method and/or the forward matching formula generation method provided by the embodiment of the present application.

Optionally, the forward matching formula generation method may be executed in the server 2, and the server 2 further executes the abnormal situation determination method provided in the embodiment of the present application. In another scenario, the server 2 may execute the abnormal condition determining method provided in the embodiment of the present application, and the other electronic device executes the forward matching formula generating method, or the server 2 may execute the forward matching formula generating method provided in the embodiment of the present application, and the other electronic device executes the abnormal condition determining method.

As a possible real-time scenario, the server 2 includes a processor 21, a memory 22, a bus 26, an abnormal situation determination device 24 and a forward matching formula generation device 25, where the memory 22 stores machine readable instructions executable by the processor 21, when the server 2 runs, the processor 21 and the memory 22 communicate with each other through the bus 26, and the processor 21 executes the machine readable instructions and executes the steps of the abnormal situation determination method or the forward matching formula generation method.

The memory 22, the processor 21 and other elements are electrically connected to each other directly or indirectly to realize signal transmission or interaction.

For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The abnormal situation determination means 24 comprises at least one software function module which may be stored in the form of software or firmware (firmware) in the memory 22. The processor 21 is configured to execute executable modules stored in the memory 22, such as software functional modules or computer programs included in the abnormal situation determination device 24 and/or the forward matching formula generation device 25.

Alternatively, the abnormal situation determining device 24 may be a software functional module or a computer program corresponding to the abnormal situation determining method provided in the embodiment of the present application, and the forward matching formula generating device 25 may be a software functional module or a computer program corresponding to the forward matching formula generating method provided in the embodiment of the present application.

The Memory 22 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.

In some embodiments, processor 21 may process information and/or data related to a service request to perform one or more of the functions described herein. In some embodiments, processor 21 may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). Merely by way of example, a Processor may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application Specific Instruction Set Processor (ASIP), a Graphics Processing Unit (GPU), a Physical Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller Unit, a Reduced Instruction Set computer (Reduced Instruction Set computer, RISC), a microprocessor, or the like, or any combination thereof.

The method defined by the process disclosed in any of the embodiments of the present application may be applied to the processor 21, or may be implemented by the processor 21.

It will be appreciated that the configuration shown in figure 2 is merely illustrative. The server 2 may also have more or fewer components than shown in fig. 2, or a different configuration than shown in fig. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof. For example, the server 2 may also provide a human-machine interface. When the abnormal condition determining system preliminarily determines that the service to be detected is abnormal, the audio information corresponding to the abnormal service to be detected can be sent to the man-machine interaction interface, so that related safety personnel further perform professional analysis on the audio information, whether the abnormal condition occurs in the service to be detected is further determined, and the related safety personnel can take rescue measures, for example, corresponding rescue workers are arranged to go to the accident occurrence place, or help to make a rescue call.

Based on the implementation architecture of the abnormal situation determination system and the structure of the server 2, as a possible implementation manner, in order to establish a relationship between the key information and the abnormal situation, the present application introduces a concept of a "forward matching formula", which has an effect of facilitating extraction of the key information from the audio information. Specifically, the present application provides a forward matching formula generating method, which is used for generating a forward matching formula, so that the server 2 in fig. 1 may extract key information in the audio information by using the forward matching formula, and further analyze and process a mapping relationship between the key information and an abnormal condition to obtain a correlation between the key information and the abnormal condition, thereby determining whether an abnormality occurs in a service process corresponding to the audio information.

Referring to fig. 3, fig. 3 is a schematic flow chart of a forward matching formula generation method according to an embodiment of the present application. The steps of the forward matching formula generation method are explained in detail below.

Step S100, historical audio information corresponding to at least one historical trip is acquired.

For example, the historical trip may be understood as a service trip in which the service vehicle has completed and is aware of whether an abnormal situation has occurred; the audio data of the historical trip may be collected through a terminal used by a driver driving the service vehicle or a terminal used by a passenger riding the service vehicle and stored in a database, the server 2 shown in fig. 1, or other devices, and meanwhile, the historical trip may include a plurality of trips in which an abnormal situation has occurred and a plurality of trips in which an abnormal situation has not occurred.

Step S101, converting each historical audio information into historical text information.

For example, each historical audio information may be converted into historical text information by Automatic Speech Recognition (ASR).

Step S102, performing text splitting on the historical text information to obtain a plurality of historical character strings.

The history character string may be a sentence, a word, or a single word included in the history text information. For example, the historical textual information is "I have arrived soon, you have again, and so on. "after the text splitting is performed on the historical text information, the obtained multiple historical character strings may include: "you wait for me again", "immediately" and "me".

Step S103, obtaining each history character string and the initial weight of the abnormal situation.

The abnormal condition can be understood as a condition that endangers passengers, drivers or other people during driving of the vehicle, for example, a traffic accident occurs during driving of the vehicle. The higher the initial weight, the greater the correlation of the history string with the occurrence of an abnormal situation.

And step S104, acquiring at least one forward character string from the historical character strings, wherein the initial weight of the forward character string is greater than or equal to a first preset threshold value.

Alternatively, since the plurality of history strings may include history strings unrelated to the abnormal situation such as the occurrence of the traffic accident, for example, "eat", "turn right", etc., the initial weight of the abnormal situation such as the occurrence of the traffic accident is low. Therefore, the historical character strings with the initial weight lower than the first preset threshold value can be filtered through the set first preset threshold value, and the forward character strings are obtained. Wherein, the first preset threshold value can be 0.1-1. It is understood that the first preset threshold may be set according to the requirements of practical applications, and is not limited herein.

And step S105, taking all the forward character strings as forward matching formulas, and obtaining the weight information of each forward character string according to the initial weights of all the forward character strings, wherein the initial weights are positively correlated with the weight information.

For example, in one possible implementation, the initial weight of each forward string may be directly used as the corresponding weight information, so as to obtain the weight information of each forward string.

For another example, the weight information corresponding to each forward character string may be determined according to the initial weight. Optionally, normalization processing may be performed on the initial weights corresponding to all the forward character strings to obtain corresponding weight information.

As a possible implementation scenario, the forward strings included in the forward matching formula and the weight information corresponding to each forward string may be shown in table 1:

TABLE 1

Forward character string	Traffic accident	Dozen 110	Injury by injury	Dial 120
					Weight information	0.4	0.2	0.1	0.3

It is understood that the examples in the above table are merely to illustrate the forward character string and the corresponding weight information more clearly, and in practical applications, the forward character string is not limited to the examples shown in the above table, but may also include other words with high relevance to the abnormal situation. The process of generating the forward matching equation may be executed based on a service provider included in the abnormal situation determination system, or may be executed based on another external electronic device.

According to the method and the device, the historical audio information is converted into the historical text information, the historical text information is split to obtain the plurality of historical character strings, the forward character string with the initial weight larger than or equal to the first preset threshold value is obtained from the historical character strings, a forward matching formula with forward correlation to the abnormal condition is mined from the historical audio information, the key information in the audio information of the service to be detected can be extracted through the forward matching formula in subsequent processing, whether the service to be detected is abnormal or not is determined, and the accuracy of detecting the abnormal conditions such as traffic accidents is improved.

As a possible implementation mode, the historical text information can be subjected to text splitting through the n-gram language model, and a plurality of historical character strings are obtained. For example, the n-gram Language Model may be a Chinese Language Model (CLM). Alternatively, the historical text information may be split into a historical character string composed of n words, where n may be a natural number such as 3, 4, 5 …, and the like.

For example, taking n as 2, splitting the text of "i love you" can result in: "i love", and "love you" total 2 history strings.

For another example, taking n as 3, splitting the text of "car accident" to obtain: the 'car accident' and 'car accident' are 2 non-repetitive historical character strings.

The inventor researches and discovers that Mutual Information (MI) as a correlation index can be generally used for calculating the correlation between two variables, and Point Mutual Information (PMI) is derived from the MI for evaluating the correlation between specific values of the two variables. In the forward matching mining of the embodiment of the present application, the main objective is to mine some keywords related to forward text information (travel service in which a traffic accident occurs), so in a possible implementation manner, historical text information can be mined by using mutual point information to form the forward matching.

Further, when the frequency of a certain historical character string appearing in the historical text information is more, the correlation between the historical character string and the abnormal situation is more reflected, so that the initial weight between the historical character string and the abnormal situation can be calculated by combining the mutual point information algorithm on the basis of the frequency of the historical character string appearing in different historical text information, and the forward character string can be obtained through initial weight screening. As a possible implementation manner, on the basis of fig. 3, fig. 4 is a schematic flowchart of a sub-step of step S103 in fig. 3 provided in an embodiment of the present application. Referring to fig. 4, one possible implementation of step S103 is:

and 103-1, acquiring the initial weight of each historical character string and the abnormal condition based on the first frequency and the second frequency. The first frequency is the value of the number of times each history character string appears in the positive text information, and the second frequency is the value of the number of times each history character string appears in the negative text information.

Wherein, when the history journey comprises the journey which is determined to have abnormal condition, and simultaneously comprises the journey which has not occurred abnormal condition. The historical text information includes positive text information and negative text information, the positive text information represents that an abnormal condition occurs, and the negative text information represents that the abnormal condition does not occur.

Further, in the embodiment of the present application, the initial weight, the first frequency and the second frequency satisfy the following formula (1):

The formula satisfied by the initial weight, the first frequency and the second frequency can be derived from the following formula (2):

in formula (2), w represents the w-th history character string, c ═ 1 represents positive text information, c ═ 0 represents negative text information, p (w | c ═ 1) is the frequency of occurrence of the w-th history character string in the positive text information, and p (w | c ═ 0) is the frequency of occurrence of the w-th history character string in the negative text information.

In the formula (2), the first step is a PMI definitional formula, the second step is full probability expansion of a denominator, and the third step uses a priori simplifying assumption, so that the probability that c is 1 is approximately considered to be 0 and the probability that c is 0 is approximately considered to be 1 because the historical journey occupation ratio of the traffic accident is extremely low relative to all historical journeys, thereby obtaining the final formula.

By calculating the PMI value, that is, the above-described initial weight, for each of the history strings by the above-described formula, the correlation of each of the history strings with the occurrence of an abnormal situation can be quantitatively evaluated.

When the historical text information is subjected to text splitting through the n-gram language model, since n can take different values, the obtained character strings may include meaningful repeated character strings when the same sentence is split according to different composition numbers n. For example: if "person is injured", a forward character string having the content "person is injured" may be obtained in the above steps S300 to S600 when n is 4. When n takes 3, forward character strings with contents of "injured" and "injured" may be obtained, and when n takes 2, forward character strings with contents of "injured" and "injured" may be obtained.

Based on the above situation, since the obtained forward character string includes a large number of character strings with repeated meanings, in order to reduce the calculation amount in actual application, the forward character string can be further screened through the following steps, and the calculation amount is reduced on the premise of ensuring the detection accuracy.

Referring to fig. 5 in combination, fig. 5 is a schematic view of a sub-step flow of step S104 in fig. 3 according to an embodiment of the present disclosure. An optional implementation manner of step S104 is:

and step S104-1, acquiring at least one intermediate character string from the historical character strings, wherein the initial weight of the intermediate character string is greater than or equal to a first preset threshold value.

Alternatively, the first preset threshold may take 0.1-1. It is understood that the first preset threshold may be set according to the requirements of practical applications, and is not limited herein.

Step S104-2, at least one forward character string is obtained from all the intermediate character strings, wherein each character included in each forward character string is different from each character included in other forward character strings.

For example, the forward string may include: "injury", "car accident", "rear-end collision", etc.

Optionally, in order to reduce the amount of calculation for calculating the correlation between the key information and the occurrence of the abnormal condition by using the forward matching equation, the forward character string may be further filtered by the following steps, and one possible implementation manner of step S104-2 in fig. 5 is:

and comparing the characters included in each intermediate character string with the characters included in other intermediate character strings one by one, and when a plurality of intermediate character strings including the same characters exist, taking the intermediate character string with the largest weight information as a forward character string.

For example, the middle string includes: "people are injured", "car accidents". Comparing the characters included in each intermediate character string with the characters included in other intermediate character strings one by one, wherein the plurality of intermediate character strings and the corresponding weight information are respectively as follows: "human injured-0.2", "injured-0.3", and "injured-0.4" all include the same character "injured", and at the same time, the weight information of the middle character string "injured" is the largest. Therefore, the middle string "injured" may be selected as the forward string.

Meanwhile, the plurality of intermediate character strings and the corresponding weight information are respectively: the car accident-0.3 and the car accident 0.1 comprise the same character of car accident, and the weight information of the middle character string of car accident is the largest. Therefore, the middle character string "car accident" can be arbitrarily selected as the forward character string.

Thus, based on the above embodiment, a display device including: the positive matching formulas of two positive character strings of 'injury' and 'car accident'. The forward matching formulas with repeated meanings are avoided, so that the calculation amount of calculating the related scores of abnormal conditions by using the forward matching formulas is reduced.

Further, the longer the character length included in the forward character string is, the more specific the linguistic meaning it expresses is, and the shorter the character length included in the forward character string is, the stronger the generalization ability thereof is. For example, if the forward character string a is "car accident", the frequency of occurrence of the forward character string a is counted only when the text information includes the complete text of "car accident". If the text message does not include the text message of 'car accident' but includes the text of 'car accident', the forward character string a will not match the text message.

When the forward character string a is a "car accident", the forward character string a can be matched with the text information regardless of whether the text information shows a "car accident" or a "car accident". Therefore, mining is further carried out based on the matching frequency without omission, and obviously, the shorter the character length contained in the forward character string is, the stronger the generalization capability is.

Therefore, in another possible implementation manner, in order to improve the calculation efficiency and accuracy in application, when the weight information between a plurality of intermediate character strings is not greatly different, the intermediate character string with a shorter character length may be adopted as the forward character string. Specifically, another optional implementation manner of step S104-2 in fig. 5 is:

comparing the characters included in each intermediate character string with the characters included in other intermediate character strings one by one, and selecting the intermediate character string with the shortest character length as a forward character string when a plurality of intermediate character strings including the same character exist and the difference value of the weight information corresponding to each intermediate character string is smaller than a preset threshold value.

For example, the intermediate character strings and the corresponding weight information respectively include: "human injury is 0.2", "traffic accident is 0.3". Comparing the characters included in each intermediate character string with the characters included in other intermediate character strings one by one, wherein the plurality of intermediate character strings: the "injured person", "injured", and "injured" include the same character "injured", and the corresponding weight information is the same. Meanwhile, the middle character string "injured" includes the shortest character length, and therefore, "injured" can be selected as the forward character string.

Meanwhile, a plurality of intermediate character strings: the car accidents and the car accidents comprise the same characters, and the corresponding weight information is the same. Meanwhile, the middle character string "car accident" includes the shortest character length, and thus the "car accident" can be selected as the forward character string.

Thus, based on the above embodiment, a display device including: the positive matching formulas of two positive character strings of 'injury' and 'car accident'. Compared with the 'people are injured' and 'car accidents', the 'injuries' and 'car accidents' with shorter characters have better generalization capability on data to be detected, so that the detection accuracy in the actual application process can be improved.

Further, when the forward matching formula includes at least one forward character string, the forward matching formula also includes logic association information between the respective forward character strings. For example, two forward strings may be associated together through a logical relationship, for example, a forward matching formula formed by a plurality of forward strings may be: dial (120| 110). Wherein, the logic association information of "typing" and "dialing" is "OR". The logical association information of "dial" and "120 | 110" is "and".

As a possible implementation scenario, when the forward matching formula further includes logical association information between forward strings, the list of forward matching formulas may be as shown in table 2:

TABLE 2

Forward character string	Traffic accident	(typing \| Dial) (120\|110)	Injury by injury
				Weight information	0.4	0.5	0.1

Comparing table 1 and table 2, it can be seen that the forward character strings "110" and "120" having similar relationships in table 1 are further associated together through a logical relationship to obtain the forward character string "(typing | dial) & (120| 110)" in table 2, so that the generalization capability of the forward matching equation can be further improved, and the accuracy of detecting abnormal conditions can be improved.

Further, when the forward matching formula further includes a logical association relationship between forward strings, since the forward matching formula is obtained by at least one forward string through the logical association relationship, the calculation frequency in the historical text information changes, and therefore, in order to improve the accuracy of the weight information thereof, the weight information of the forward strings can be further acquired again.

As an alternative implementation, the final weight of each forward character string and the abnormal condition may be obtained, and the final weight is used as the corresponding weight information.

For example, the first frequency, i.e., the value of the number of times each positive character string appears in the positive text information, may be obtained anew, and the second frequency, i.e., the value of the number of times each positive character string appears in the negative text information, may be obtained anew, in conjunction with step S103-1 in fig. 4 above. And (4) obtaining the final weight of each forward character string and the abnormal situation again by utilizing the first frequency, the second frequency and the formula, and taking the final weight as corresponding weight information. So as to improve the accuracy of the weight information corresponding to each forward character string.

The embodiment of the application also provides an abnormal condition determining method, and the method can analyze the audio information acquired in the service process based on the forward matching formula generated by the method and determine the correlation degree information of the text information and the abnormal condition, so that the problem of low accuracy in detecting abnormal conditions such as traffic accidents and the like is solved at present.

The embodiment of the application also provides an abnormal situation determination method, which can be executed by the service vehicle or the server in fig. 1. The abnormal situation determining method can obtain a forward matching formula associated with the abnormal situation in the forward direction through the method in fig. 3, extract key information in the audio information through the forward matching formula, and further analyze and process the mapping relation between the key information and the abnormal situation to obtain the correlation between the key information and the abnormal situation, so as to determine whether the abnormal situation occurs in the service process corresponding to the audio information.

Referring to fig. 6 in combination, fig. 6 is a flowchart of an abnormal situation determination method according to an embodiment of the present application. The flow diagram shown in fig. 6 is also explained in detail in the network appointment scenario below.

And step S200, acquiring audio information corresponding to the service to be detected.

The service to be detected may be a service trip currently performed by the service vehicle, and the audio information may be a recording collected by a passenger's terminal or a driver's terminal in the service vehicle.

Step S201, acquiring text information corresponding to the audio information.

Step S202, determining the correlation degree information of the text information and the abnormal situation according to the forward matching formula and the text information, wherein the forward matching formula and the abnormal situation have a forward relation, and the correlation degree information represents the correlation degree value of the text information and the abnormal situation.

The forward matching expression has a forward relationship with an abnormal condition, and the forward matching expression can be understood as a word or a sentence which often appears in the audio information when the abnormal condition occurs, for example: car accidents, rear-end collisions, injuries, etc.

The forward matching formula may be mined, generated and migrated to the current service vehicle or the server in other electronic devices by the method shown in fig. 3, or may be mined, generated and stored in advance in the current service vehicle or the server.

Step S203, when the correlation degree information is larger than the correlation degree threshold value, determining that the service to be detected has abnormal conditions.

Wherein, the threshold value of the correlation degree can be 0.1-1. It is understood that the threshold value of the correlation degree can be set according to the requirements of practical applications, and is not limited herein.

According to the embodiment of the application, the audio information corresponding to the service to be detected is acquired, the text information corresponding to the audio information is acquired, and the correlation degree information between the text information and the abnormal condition is determined by using the forward matching formula having the forward relation with the abnormal condition, so that whether the vehicle corresponding to the service to be detected has the abnormal conditions such as traffic accidents or not can be effectively detected, and the accuracy of detecting the abnormal conditions such as traffic accidents is improved.

Optionally, the method steps are explained in detail below with a server in the network appointment scenario shown in fig. 1 as an execution subject.

Optionally, one possible implementation manner of step S201 is: each audio message is converted to a text message by ASR.

Each audio information is converted into text information, effective information contained in the audio information can be rapidly mined based on a forward matching formula, and interferences such as noise and the like are eliminated. For example, the text corresponding to the navigation voice in the text information can be directly deleted.

Further, since the text information may include a plurality of texts matching the forward character string, and based on all the texts having a matching relationship with the forward character string, the association between the text information and the abnormal condition may be reflected, as an optional implementation manner, step S202 may be implemented in the following manner to determine the information of the degree of correlation between the text information and the abnormal condition. Referring to fig. 7 in combination, fig. 7 is a schematic view of a sub-step flow of step S202 in fig. 6 according to an embodiment of the present disclosure.

And step S202-1, matching each forward character string with the text information respectively.

And step S202-2, when the text information has the text which is successfully matched with the forward character string, taking the forward character string as the target character string.

And S202-3, obtaining forward scores corresponding to all target character strings, wherein the forward scores corresponding to the target character strings are positively correlated with the abnormal conditions.

In step S202-4, the forward scores corresponding to all the target character strings are used as the correlation degree information.

Wherein the forward matching formula comprises at least one forward string, and each forward string has a forward association with the abnormal condition. For example, when the abnormal condition is a traffic accident, the forward strings included in the forward matching equation with the forward association may be: the car accident, the injury, the people hit, the 110 hit, the 120 hit, the rear-end collision and the like.

Because different forward character strings correspond to different weight information and have different correlations with abnormal situations, and the frequency of the forward character strings appearing in the text information can reflect the correlations with the abnormal situations, the forward scores corresponding to all the target character strings can be calculated through the frequency and the weight information. One possible implementation of step S202-3 is:

and obtaining the secondary numerical value of each target character string matched with the corresponding text and the weight information of the forward character string corresponding to each target character string. And obtaining the weighted sum of all the target character strings according to all the sub-numerical values and the weight information. The weighted sum is taken as the forward score.

For example, as one possible implementation scenario, the text message may be "car accident, someone injured, and hit 120". Assume that the forward character strings include "car accident", "injury", "collided with person", and "rear-end collision", respectively. The weight score corresponding to each forward character string is 0.3, 0.2 and 0.3 respectively. As shown in table 3:

TABLE 3

Forward character string	Traffic accident	Injury by injury	Person who crashes	Rear-end collision
					Weight score	0.3	0.2	0.2	0.3

And each forward character string is matched with the text information, so that the successfully matched target character strings of 'car accidents' and 'injuries' can be obtained. As can be seen from table 3, the weight information corresponding to the target character string "car accident" is 0.3, and the weight information corresponding to the target character string "injury" is 0.2.

The matching times of the text corresponding to each target character string are all 1. Then the weighted sum of all the target strings is calculated as: 1 × 0.3+1 × 0.2 ═ 0.5. The forward score is equal to a weighted sum of 0.5.

Therefore, the audio information corresponding to the service to be detected is converted into the text information, and the correlation degree information between the text information and the abnormal condition is further determined based on the forward matching formula and the text information. On one hand, whether abnormal conditions such as traffic accidents occur or not can be effectively judged, and the accuracy of detecting the abnormal conditions such as the traffic accidents can be improved by searching the text information through the forward matching formula. On the other hand, the scene situation of the vehicle corresponding to the service to be detected when abnormal situations such as traffic accidents occur can be determined based on the meaning expressed by the forward matching formula.

Based on the same inventive concept, the embodiment of the application also provides an abnormal situation determining device corresponding to the abnormal situation determining method.

Referring to fig. 8, fig. 8 is a schematic functional block diagram of an abnormal situation determination apparatus 24 according to an embodiment of the present application, where the apparatus includes:

the obtaining module 241 is configured to obtain audio information corresponding to a service to be detected.

The obtaining module 241 is further configured to obtain text information corresponding to the audio information.

A determining module 242, configured to determine, according to the forward matching equation and the text information, degree information of correlation between the text information and the abnormal condition; the positive matching expression has positive relation with the abnormal condition; the correlation degree information represents a correlation degree value between the text information and the abnormal condition.

The determining module 242 is further configured to determine that the service to be detected is abnormal when the correlation degree information is greater than the correlation degree threshold.

In an alternative embodiment, the forward matching formula includes at least one forward string, and each of the forward strings has a forward association with the abnormal condition.

The determining module is configured to match each forward character string with the text information respectively.

And when the text information has the text which is successfully matched with the forward character string, taking the forward character string as a target character string.

Obtaining forward scores corresponding to all the target character strings; and the positive score corresponding to the target character string is positively correlated with the abnormal condition.

In an optional embodiment, the determining module is configured to obtain a secondary value of each target character string matching with a corresponding text, and weight information of a forward character string corresponding to each target character string.

And obtaining the weighted sum of all the target character strings according to all the sub-numerical values and the weight information.

And taking the weighted sum as the forward score.

Because the principle of solving the problem of the device in the embodiment of the present application is similar to that of the method for determining the abnormal situation in the embodiment of the present application, the implementation of the device may refer to the implementation of the method, and repeated details are not described again.

Based on the same inventive concept, an embodiment of the present application further provides a forward matching formula generating device corresponding to the forward matching formula generating method, where the forward matching formula generating device includes:

and the audio information acquisition module is used for acquiring historical audio information corresponding to at least one historical trip.

And the conversion module is used for converting each historical audio information into historical text information.

And the splitting module is used for performing text splitting on the historical text information to obtain a plurality of historical character strings.

And the weight obtaining module is used for obtaining the initial weight of each historical character string and the abnormal condition.

The character string obtaining module is used for obtaining at least one forward character string from the historical character strings, wherein the initial weight of the forward character string is greater than or equal to a first preset threshold value.

In an optional implementation manner, the historical text information includes positive text information and negative text information, the positive text information represents that an abnormal condition occurs, and the negative text information represents that no abnormal condition occurs.

In an optional embodiment, the character string obtaining module is configured to obtain at least one intermediate character string from the history character string, where a forward score of the intermediate character string is greater than or equal to the first preset threshold.

In an optional embodiment, the forward matching formula obtaining module is configured to obtain a final weight of each forward character string and an abnormal condition. And taking the final weight as corresponding weight information.

Because the principle of the apparatus in the embodiment of the present application for solving the problem is similar to that of the method for generating the forward matching formula in the embodiment of the present application, the method may be used for implementing the apparatus, and repeated details are not described herein.

The embodiment of the present application also provides a readable storage medium, in which a computer program is stored, and when the computer program is executed, the abnormal situation determination method or the forward matching formula generation method described above is implemented.

In summary, the embodiment of the present application provides an abnormal situation determination method and a forward matching formula generation method, where the abnormal situation determination method obtains audio information corresponding to a service to be detected and obtains text information corresponding to the audio information. And then determining the correlation degree information of the text information and the abnormal condition according to the forward matching formula and the text information, wherein the forward matching formula has a forward relation with the abnormal condition, and the correlation degree information represents the correlation degree value of the text information and the abnormal condition. And when the correlation degree information is greater than the correlation degree threshold value, determining that the service to be detected has an abnormal condition. Therefore, on one hand, whether abnormal conditions such as traffic accidents occur or not can be effectively judged, and the accuracy of detecting the abnormal conditions such as the traffic accidents is improved. On the other hand, the scene situation of the vehicle corresponding to the service to be detected when abnormal situations such as traffic accidents occur can be determined based on the meaning expressed by the forward matching formula.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An abnormal situation determination method, characterized in that the method comprises:

acquiring audio information corresponding to a service to be detected;

acquiring text information corresponding to the audio information;

2. The abnormal situation determination method according to claim 1, wherein the forward matching formula comprises at least one forward character string, each of the forward character strings having a forward association with the abnormal situation; the step of determining the information of the degree of correlation between the text information and the abnormal situation according to the forward matching formula and the text information comprises the following steps:

matching each forward character string with the text information respectively;

3. The abnormal situation determination method according to claim 2, wherein the step of obtaining forward scores corresponding to all the target character strings comprises:

and taking the weighted sum as the forward score.

4. The abnormal situation determination method according to claim 1, wherein before the step of obtaining the audio information corresponding to the service to be detected, the method further comprises:

converting each historical audio information into historical text information;

5. The abnormal situation determination method according to claim 4, wherein when the forward matching formula includes at least one forward character string, the forward matching formula further includes logical association information between the respective forward character strings.

6. The abnormal situation determination method according to claim 4, wherein the historical text information includes positive text information and negative text information, the positive text information indicates that an abnormal situation occurs, and the negative text information indicates that no abnormal situation occurs;

7. The abnormal situation determination method according to claim 6, wherein the initial weight, the first frequency and the second frequency satisfy the following formula:

8. The abnormal situation determination method according to claim 4, wherein the step of obtaining at least one of the forward strings from the history strings comprises:

obtaining at least one intermediate character string from the historical character string, wherein the forward score of the intermediate character string is greater than or equal to the first preset threshold;

9. The abnormal situation determination method according to claim 8, wherein the step of obtaining the weight information of each of the forward strings based on the initial weights of all of the forward strings comprises:

and taking the final weight as corresponding weight information.

10. The abnormal situation determination method according to claim 1, wherein before the step of obtaining the text information corresponding to the audio information, the method further comprises:

and deleting the voice navigation information in the audio information.

11. A method for generating a forward matching expression, the method comprising:

converting each historical audio information into historical text information;

12. The method of claim 11, wherein when the forward matching formula comprises at least one forward character string, the forward matching formula further comprises logical association information between the forward character strings.

13. The method for generating positive matching expressions according to claim 11, wherein the historical text information includes positive text information and negative text information, the positive text information indicates that an abnormal condition occurs, and the negative text information indicates that an abnormal condition does not occur;

14. The method according to claim 13, wherein the initial weight, the first frequency and the second frequency satisfy the following equation:

15. The forward matching formula generating method of claim 11, wherein the step of obtaining at least one forward character string from the history character strings comprises:

16. The forward matching formula generating method according to claim 15, wherein the step of obtaining weight information of each of the forward character strings according to initial weights of all the forward character strings comprises:

and taking the final weight as corresponding weight information.

17. An abnormal situation determination apparatus, characterized in that the apparatus comprises:

18. The abnormal situation determination apparatus of claim 17, wherein the forward matching formula comprises at least one forward string, each of the forward strings having a forward association with the abnormal situation;

19. The abnormal situation determination device of claim 18, wherein the determination module is configured to obtain a secondary value of each target character string matching with a corresponding text and weight information of a forward character string corresponding to each target character string;

and taking the weighted sum as the forward score.

20. The abnormal situation determination device according to claim 17, further comprising:

21. A forward matching formula generating apparatus, comprising:

22. The apparatus of claim 21, wherein when the forward matching formula comprises at least one forward character string, the forward matching formula further comprises logical association information between the forward character strings.

23. The apparatus according to claim 21, wherein the historical text information includes positive text information and negative text information, the positive text information indicates that an abnormal condition occurs, and the negative text information indicates that an abnormal condition does not occur;

24. The apparatus according to claim 23, wherein the initial weight, the first frequency and the second frequency satisfy the following equation:

25. The apparatus according to claim 21, wherein the character string obtaining module is configured to obtain at least one intermediate character string from the history character string, wherein a forward score of the intermediate character string is greater than or equal to the first preset threshold;

26. The apparatus according to claim 25, wherein the forward matching formula obtaining module is configured to obtain a final weight of each forward character string and an abnormal condition;

and taking the final weight as corresponding weight information.

27. An electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the abnormal situation determination method according to any one of claims 1 to 10; or to perform the steps of the forward matching formula generation method of any of claims 11-16.

28. An abnormal situation determination system is characterized by comprising a service provider and a client which are communicated with each other;

29. A readable storage medium storing a computer program which, when executed, implements the abnormal situation determination method of any one of 1 to 10; or implementing the forward matching formula generation method of any one of claims 11 to 16.