WO2021051585A1 - Method for constructing natural language processing system, electronic apparatus, and computer device - Google Patents

Abstract

Disclosed are a method for constructing a natural language processing system, an electronic apparatus, a computer device, and a storage medium. The method comprises: extracting several word and phrase features from received sentence text (S110); converting the word and phrase features to a D-dimensional text vector, and transmitting the text vector to a neural network (S120); acquiring an output sentence obtained after the neural network has received the text vector, and calculating a target weight parameter of the neural network according to an error term between the output sentence and the text vector and by means of a backpropagation algorithm (S130); and adjusting a weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network satisfies a preset condition, and obtaining a natural language processing system (S140). A neural network is automatically trained, and a good processing effect is achieved, thereby improving the accuracy and convenience of constructing a natural language processing system.

Description

Construction method, electronic device and computer equipment of natural language processing system

This application requires that it be submitted to the Chinese Patent Office on September 17, 2019. The patent name is "Natural language processing system construction method, electronic device and computer equipment", and the application number is 201910876792.1. The priority of the Chinese patent application for the invention patent, which The entire content is incorporated into this application by reference.

Technical field

This application relates to the field of artificial intelligence technology, and in particular to a method for constructing a natural language processing system, electronic devices, computer equipment, and storage media.

Background technique

Natural language processing, which uses computers to process human language, embodies the highest tasks and realm of artificial intelligence. Common application methods include information extraction, machine translation, and intelligent question and answer systems. In the natural language processing system of the intelligent question answering system, accurate and concise natural language is used to answer the questions raised by human users in natural language.

The inventor found that the current commonly used method is to use a pre-trained neural network-based classifier to extract the structured features of natural language sentences, and then retrieve or infer the corresponding answers from the pre-established knowledge base based on the structured features. In the training of the above-mentioned neural network-based classifier and the establishment of the knowledge base, it is necessary to provide a large amount of training data labeled with structured features for the neural network-based classifier to perform deep learning. This manual labeling is time-consuming and expensive. The scenarios that are not in the knowledge base have poor application effects, lack of versatility, and severely limited application scenarios.

Summary of the invention

In view of this, this application proposes a construction method, electronic device, computer equipment, and storage medium for a natural language processing system, which can automatically train neural networks and achieve better processing effects, and improve the accuracy and accuracy of building a natural language processing system. Convenience.

First of all, in order to achieve the above objective, this application proposes a method for constructing a natural language processing system, which includes the steps of: extracting several word features from the received sentence text; converting the word features into a D-dimensional text vector , And pass the text vector to the neural network; obtain the output sentence of the neural network after receiving the text vector, and calculate the goal of the neural network through the backpropagation algorithm according to the error term between the output sentence and the text vector Weight parameter; and adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets a preset condition, and a natural language processing system is obtained.

In addition, in order to achieve the above object, the present application also provides an electronic device, which includes: an extraction module, adapted to extract several word features from the received sentence text; a conversion module, adapted to convert the word features into D Dimensional text vector, and pass the text vector into the neural network; the calculation module is adapted to obtain the output sentence after the neural network receives the text vector, and pass the inverse direction according to the error term between the output sentence and the text vector The propagation algorithm calculates the target weight parameter of the neural network; and an adjustment module, adapted to adjust the weight parameter of each node of the neural network according to the target weight parameter, until the output sentence of the neural network meets the preset condition, and the natural language processing is obtained system.

To achieve the above objective, the present application also provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, and the processor executes the computer-readable instructions When realizing the steps of the above method.

To achieve the foregoing objective, the present application also provides a non-volatile computer-readable storage medium on which computer-readable instructions are stored, and the computer-readable instructions implement the steps of the foregoing method when executed by a processor.

The construction method, electronic device, computer equipment, and storage medium of the natural language processing system proposed in this application convert the word features extracted from the received sentence text into a D-dimensional text vector, and then transfer the text vector to the nerves. Network, obtain the output sentence of the neural network after receiving the text vector, and then calculate the target weight parameter of the neural network through the backpropagation algorithm according to the error term of the output sentence and the text vector, and then calculate the target weight parameter of the neural network according to the target weight parameter The weight parameters of each node of the neural network are adjusted, and the neural network is continuously and automatically trained until the output sentence of the neural network meets the preset conditions, and a natural language processing system with better processing effect is obtained, which improves the construction of the natural language processing system. Accuracy and convenience.

Description of the drawings

FIG. 1 is a schematic flowchart of a method for constructing a natural language processing system according to an exemplary embodiment of the present application;

Fig. 2 is a schematic structural diagram of a neural network shown in an exemplary embodiment of the present application;

3 is a schematic flowchart of a method for constructing a natural language processing system according to an exemplary embodiment of the present application;

4 is a schematic flowchart of a method for constructing a natural language processing system according to an exemplary embodiment of the present application;

FIG. 5 is a schematic flowchart of a method for constructing a natural language processing system according to an exemplary embodiment of the present application;

Fig. 6 is a schematic structural diagram of a neural network shown in an exemplary embodiment of the present application;

FIG. 7 is a schematic diagram of program modules of an electronic device shown in an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram of the hardware architecture of an electronic device shown in an exemplary embodiment of the present application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

detailed description

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be noted that the descriptions related to "first", "second", etc. in this application are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features . Therefore, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on what can be achieved by a person of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist. , Is not within the scope of protection required by this application.

Refer to Fig. 1, which is a schematic flowchart of a method for constructing a natural language processing system according to an embodiment of the present application. The method includes the following steps:

Step S110, extract several word features from the received sentence text;

Step S120: Convert the word feature into a D-dimensional text vector, and pass the text vector into the neural network;

Step S130: Obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network through the back propagation algorithm according to the error term between the output sentence and the text vector; and

Step S140: Adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets a preset condition, and a natural language processing system is obtained.

Natural language processing, which uses computers to process human language, embodies the highest tasks and realm of artificial intelligence. Common application methods include information extraction, machine translation, and intelligent question and answer systems. In the natural language processing system of the intelligent question answering system, accurate and concise natural language is used to answer the questions raised by human users in natural language. Taking the intelligent question answering system as an example, the system can understand the meaning of what the user says, and then make the corresponding answer.

Several word features are extracted from the received sentence text. The received sentence text can be the sentence text obtained by directly converting the words spoken by the user in the scenes of machine translation, intelligent question answering system, robot customer service, etc. The grammar is divided into several words, and several of the word features that can express the key information of the sentence text are extracted. For example, when the user says in the robot customer service scenario: "How is the progress of the insurance claim that I submitted last week?", it can be grammatically divided into "I", "Last week", "Submit", "Of" , "Insurance", "Claim", "Progress", "How", "Lau", extract the word features that can express the key information of the sentence text: "I", "的", "Insurance" , "Claim", "Progress". The word features need to be transformed into a form that can be recognized by the neural network. The dimension D of the word vector is set according to the dimension of the neural network, and all words are randomly initialized to a D-dimensional text vector, and then all D-dimensional text vectors for the context The encoding obtains a vector of a hidden layer, which becomes a form that can be processed by the neural network.

At present, natural language processing is usually performed through neural networks, which are actually multiple neurons connected according to certain rules. Refer to Figure 2. Figure 2 shows a fully connected (FC) neural network. Through observation, it can be found that the rules of the FC neural network include:

1). Neurons are laid out in layers. The leftmost layer is called the input layer, which is responsible for receiving input data; the rightmost layer is called the output layer, and users can get the data output by the neural network from the output layer. The layers between the input layer and the output layer are called hidden layers because they are invisible to the outside.

2) There is no connection between neurons in the same layer.

3) Each neuron in the Nth layer is connected to all neurons in the N-1th layer (this is the meaning of fully connected), and the output of the neurons in the N-1th layer is the input of the Nth layer of neurons.

4) Each connection has a weight.

The above rules define the structure of a fully connected neural network. Among them, the hidden layer shown in the neural network in Figure 2 is only one layer. In fact, the number of hidden layers can have multiple layers, which is not limited to what is shown in the figure. . In fact, there are many other types of neural networks, such as Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), all of which have different connection rules.

The neural network outputs information according to the received input information. In order to ensure the accuracy of the output information, the neural network needs to be trained, usually by training the neural network through training samples. In an embodiment of this application, the constructed neural network is a natural language processing system. Therefore, sentence text can be selected as a training sample. The sentence text can be a sentence in the form of a text obtained directly, or it can be obtained through audio, etc. For the sentence text converted from recognition, this application does not limit the source form of the sentence text.

After the sentence text is obtained, the sentence text is parsed to extract the word features, and the neural network "understands" the meaning of the sentence text according to the word features.

As shown in FIG. 3, in an embodiment of the present application, the step of extracting several word features from the received sentence text may include the following steps:

In step S301, the sentence text is divided into several words according to the grammar, and the part of speech of the words in the sentence text is marked according to the part of speech tag set; and

Step S302: Decompose the sentence text into word features according to the part of speech.

At present, the Chinese part-of-speech tag set of the Institute of Computing Technology (99 in total, 22 first-class, 66 second-class, 11 third-class) mainly refers to the following part-of-speech tag sets: Peking University "People's Daily" corpus part-of-speech tag set, Peking University 2002 new version of part-of-speech Marker set (draft), Tsinghua University Chinese Tree Bank Part-of-Speech Marker Set, Ministry of Education Pragmatics Part-of-Speech Marker Set (National Recommendation Standard Draft 2002 Edition), and Chinese Penn Tree Bank Part-of-Speech Marker Set of the University of Pennsylvania (Chinese Penn Tree Bank). For example, when the user says in the robot customer service scenario: "How is the progress of the insurance claim that I submitted last week?", it can be grammatically divided into "I", "Last week", "Submit", "Of" , "Insurance", "Claim", "Progress", "How", "Lau", mark the part of speech of the words in the sentence text according to the part of speech tag set.

Part of speech tagging is to determine the most appropriate part of speech for each word according to the context of the sentence. There are 26 basic parts of speech tags in the current tag set (noun n, time word t, location word s, location word f, numeral m, quantifier q, distinguishing word b, pronoun r, verb v, adjective a, state word z , Adverb d, preposition p, conjunction c, auxiliary u, modal particle y, interjection e, onomatopoeia o, idiom i, idiomatic expression l, abbreviation j, preceded by component h, followed by component k, morpheme g, non In addition to morphemes x and punctuation marks w), from the perspective of corpus application, proper nouns (person's name nr, place name ns, organization name nt, other proper nouns nz) have been added; some marks have also been added from the perspective of linguistics. More than 40 markers were used, see Table 1 for details.

Table 1 Marking comparison table of memory codes

Code name Code name Code name Ag Morpheme l Idioms s Location word a adjective Mg Number morphemes Tg Temporal morphemes ad Adverb m numeral t Time word an Nouns Ng Nominal Morpheme Ug Auxiliary morpheme Bg Distinguishing morphemes n noun u particle b Distinguishing words nr Person's name Vg Verbal morpheme c conjunction ns Place name v verb Dg Adverb morpheme nt Institutional groups vd Adverb d adverb nx Foreign characters vn Noun verb e interjection nz Other proper names w Punctuation f Position of the word o Onomatopoeia x Non-morpheme g Morpheme p preposition Yg Mood morpheme h Front component Qg Morpheme y Modal i idiom q quantifier z State word

j Abbreviation Rg Morpheme To To k Followed by ingredients r pronoun To To

The following uses an example to briefly introduce how to tag the sentence text "I am a little cat":

The code for part-of-speech tagging is as follows:

"Import jieba.posseg as pseg

words=pseg.cut("I am a little cat")

for word, flag in words:

print('%s%s'%(word,flag))"

The output is as follows:

I r

Is v

One m

Little meow n

According to the results of the above-mentioned part-of-speech tagging, the sentence text can be decomposed into word features. For example, the sentence text "I am a little cat" can be decomposed into 4 word features, namely "I r" and "是v" ", "a m", "little meow n".

One of the difficulties in part-of-speech tagging is that for words with multiple genders, such as work, performance and other words, they can be used as both verbs and nouns. Such words are also called mixed words, which appear in common words. The probability is very high. In view of this situation, we usually use probabilistic methods to solve it. For example, HMM (Hidden Markov Model, Hidden Markov Model) is a commonly used method to deal with the labeling of such words, specifically, the use of HMM graph models There is a probability of emission, that is, the probability from a part of speech to each word, and the transition probability between part of speech and part of speech to find p(t|w) (where p(t|w) means that the word belongs to The probability of a certain part of speech), the specific formula calculation can use the Bayesian method of calculating the probability. Of course, there are also part-of-speech tagging methods based on conversion ideas and classification-based ideas. This application does not limit the way of part-of-speech tagging.

For the neural network, the input information is actually a D-dimensional vector, and the dimension of the input vector = the number of nodes in the input layer.

As shown in FIG. 4, in an embodiment of the present application, the step of converting the word feature into a D-dimensional text vector may include the following steps:

Step S401, matching D-1 entries related to the word feature according to the similarity measurement method;

Step S402, setting D entry weights according to the correlation between the character feature and the entry; and

Step S403: Generate a D-dimensional text vector according to the weights of the D entries.

The word feature is converted into a D-dimensional text vector through the word2vec algorithm and/or the doc2vec algorithm; the text vector is passed into the deep neural network, and the text vector is processed through the hidden layer of the deep neural network To process. Since the dimension of the input vector of the neural network = the number of nodes in the input layer, it is necessary to convert a word feature into a list of vectors. The word feature can be broken into (01010101110) values through the hash algorithm, and the word2vec algorithm can be used The word2vec algorithm also considers the upper and lower semantics, while the word2vec algorithm also considers the order of the upper and lower sentences, which is better used in paragraphs.

The way to compose sentence vectors according to word vectors is as follows:

1). If a word feature is converted into a list of vectors, it is generally obtained by simple addition;

2) If a word feature is converted into a vector, the weight of the word feature can be used to recombine the vector.

Of course, it is also possible to match D-1 entries related to the word feature according to the similarity measurement method (for example, cosine distance); set D entries according to the degree of correlation between the word feature and the entry Weight; generating a D-dimensional text vector according to the weights of the D entries.

As shown in FIG. 5, in an embodiment of the present application, the step of calculating the target weight parameter of the neural network through a backpropagation algorithm according to the error term between the output sentence and the text vector may include the following steps:

Step S501, starting from the output layer of the neural network, calculate the error terms of each hidden layer in reverse order; and

Step S502: Calculate the weight parameter of each node of the hidden layer according to the error term.

A neural network is actually a function from an input vector to an output vector, that is, to calculate the output of the neural network based on the input, first the value of each element of the input vector needs to be assigned to the corresponding neuron of the input layer of the neural network, and then according to the formula in turn Calculate the value of each neuron in each layer forward, until the value of all neurons in the output layer of the last layer is calculated. Finally, string together the values of each neuron in the output layer to get the output vector. According to the difference between the output result and the expected result, how to adjust the weight parameter of the neuron is inversely deduced, so that the output result of the neural network can achieve the expected effect.

For ease of description, the following takes a fully connected (FC) neural network as an example for description. As shown in Figure 6, each unit of the neural network is numbered. The input layer has three nodes, numbered 1, 2, and 3 sequentially; the 4 nodes of the hidden layer are numbered 4, 5, 6, and 7 in order; the two nodes of the output layer are numbered 8, 9. Figure 6 shows a fully connected neural network, so each node is connected to all nodes in the upper layer. For example, there are connections between node 4 of the hidden layer and the three nodes 1, 2, and 3 of the input layer, and the weight parameters of the connections are w41, w42, and w43, respectively.

Nodes 1, 2, and 3 are the nodes of the input layer, and the output value is the input vector itself. According to Figure 6, the output values of nodes 1, 2, and 3 are x1, x2, and x3, respectively. The output value of node 1, 2, 3 is the input value of node 4. According to the output value of node 1, 2, 3, the weight parameter between node 4 and the input layer 1, 2, 3, node 4 can be calculated The output value a4. Similarly, it can be calculated that the output values of nodes 5, 6, and 7 are a5, a6, and a7, respectively, and the output values of nodes 8, 9 of the output layer are y1, and y2, respectively.

The output value of the output layer node is the actual output value of the neural network. The target weight parameter of the hidden layer can be calculated according to the error between the actual output value and the expected output value corresponding to the input value to reduce the actual output value and the expected output value The error between.

To calculate the error term of a node, you need to first calculate the error term of each node of the next layer connected to it. This requires that the calculation sequence of the error term must start from the output layer, and then calculate the error term of each hidden layer in reverse order until the hidden layer connected to the input layer. This is also the meaning of the name of the backpropagation algorithm. After the error terms of all nodes are calculated, the weight parameters of each node of the neural network can be updated. Repeat the above process until the output sentence of the neural network meets the preset conditions. The preset condition includes: the proportion of sentences that exist in nature in the output sentences of the neural network reaches a preset threshold.

The neural network "analyzes" the received sentence text to obtain an output sentence that can be "understood" by the machine. In an embodiment of the present application, it is possible to predict whether the output sentence exists in nature. Existing sentences are "true sentences", and sentences that do not exist in nature are marked as "fake sentences" for overall prediction. The data set publicly available on the Internet can be used as a true value reference to predict whether the output sentence is a "true sentence". The step until the output sentence of the neural network satisfies the preset condition further includes: judging whether the output sentence is a sentence existing in nature; calculating the total amount of the output sentence of the neural network as the first number, and calculating which belongs to the natural world The total number of output sentences of the sentences existing in the sentence is the second number; the ratio of the second number to the first number is calculated, and it is judged whether the ratio reaches a preset threshold. According to the obtained batch of output sentences, whether the proportion of "true sentences" reaches a preset threshold (for example, 90%, etc.), it is judged whether the neural network is trained to a better state.

In this application, the neural network may be a deep neural network, which is a multi-layer neural network, and the advantage of multi-layer is that it can express complex functions with fewer parameters. An effective method to build a multilayer neural network is simply divided into two steps. One is to train one layer of the network each time, and the other is to tune the high-level representation r generated from the original representation x upward and the high-level representation r generated downward The x'is as consistent as possible. the way is:

1) First, build a single-layer neuron layer by layer, so that a single-layer network is trained every time.

2) After all layers are trained, use Wake-Sleep algorithm for tuning. The weights between the other layers except the top layer are changed to bidirectional, so that the top layer is still a single-layer neural network, and the other layers become a graph model. The upward weight is used for "cognition" and the downward weight is used for "generation". Then use the Wake-Sleep algorithm to adjust all the weights. Reaching agreement between cognition and generation is to ensure that the top-level representation of the generation can restore the underlying nodes as accurately as possible. For example, a node on the top layer represents a human face, then the image of all faces should activate this node, and the image generated downward from this result should be able to represent a rough face image. The Wake-Sleep algorithm is divided into two parts: Wake and Sleep.

2.1), Wake stage, the cognitive process, through the characteristics of the outside world and upward weight (cognitive weight) to generate the abstract representation of each layer (node state), and use gradient descent to modify the downward weight between layers (generate weight) . That is, "If reality is different from what I imagined, changing my weight makes what I imagined is like this".

2.2), Sleep stage, the generation process, through the top-level representation (the concept learned when waking up) and the downward weight, the state of the bottom layer is generated, and the upward weight between the layers is modified at the same time. That is, "If the scene in the dream is not the corresponding concept in my mind, change my cognitive weight so that this scene is the concept in my opinion."

The construction method of the natural language processing system proposed in this application can convert the word features extracted from the received sentence text into a D-dimensional text vector, pass the text vector into a neural network, and obtain the neural network to receive the text The output sentence after the vector is calculated by the backpropagation algorithm to calculate the target weight parameter of the neural network, and the weight parameter of each node of the neural network is adjusted according to the target weight parameter, until the output sentence of the neural network meets the preset condition, through Automatically train neural networks and achieve better processing results, which improves the accuracy and convenience of building natural language processing systems.

The application further provides an electronic device. Refer to FIG. 7, which is a schematic diagram of program modules of the electronic device 20 according to an exemplary embodiment of the present application.

The electronic device 20 includes:

The extraction module 201 is adapted to extract several word features from the received sentence text;

The conversion module 202 is adapted to convert the word feature into a D-dimensional text vector, and pass the text vector into a neural network;

The calculation module 203 is adapted to obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network through a back propagation algorithm; and

The adjustment module 204 is adapted to adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets a preset condition.

Further, the extraction module 201 includes: a labeling unit adapted to label the part of speech of words in the sentence text; and a decomposition unit adapted to decompose the sentence text into word features according to the part of speech.

Further, the conversion module 202 includes: a matching unit, adapted to match D-1 entries related to the word feature according to a similarity measure; a setting unit, adapted to match the word feature with the word feature according to the The degree of relevance of the term sets D term weights; and the vector generating unit is adapted to generate a D-dimensional text vector according to the D term weights.

Further, the similarity measurement method includes cosine distance.

Further, the calculation module 203 is further adapted to calculate the error term of each hidden layer in reverse order starting from the output layer of the neural network; and calculate the weight parameter of each node of the hidden layer according to the error term.

Further, the preset condition includes: the proportion of sentences that belong to the natural world in the output sentences of the neural network reaches a preset threshold.

Further, the neural network is a deep neural network.

The electronic device 20 proposed in this application can convert the word features extracted from the received sentence text into a D-dimensional text vector, pass the text vector into a neural network, and obtain the output of the neural network after receiving the text vector Sentence, calculate the target weight parameter of the neural network through the backpropagation algorithm, adjust the weight parameter of each node of the neural network according to the target weight parameter, until the output sentence of the neural network meets the preset condition, through the automatic training of the neural network And achieve a better processing effect, improve the accuracy and convenience of building a natural language processing system.

To achieve the above objective, the present application also provides a computer device 20, including a memory 21, a processor 22, and computer-readable instructions stored on the memory 21 and running on the processor 22, and the processor 22 executes The computer-readable instructions implement the steps of the above method. The computer readable instructions can be stored in the memory 24.

To achieve the foregoing objective, the present application also provides a non-volatile computer-readable storage medium on which computer-readable instructions are stored, and the computer-readable instructions implement the steps of the foregoing method when executed by a processor.

This application also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a cabinet server (including independent servers, or more) that can execute programs. A server cluster composed of two servers) and so on. The computer device in this embodiment at least includes but is not limited to: a memory, a processor, etc., which can be communicably connected to each other through a system bus.

This embodiment also provides a non-volatile computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory ( SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, servers, App application malls, etc., on which storage There are computer-readable instructions, and the corresponding functions are realized when the program is executed by the processor. The non-volatile computer-readable storage medium of this embodiment is used to store the electronic device 20, and when executed by the processor 22, realizes the construction method of the natural language processing system of the present application.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims (20)

A method for constructing a natural language processing system, the method comprising the steps: Extract several word features from the received sentence text; Converting the word feature into a D-dimensional text vector, and passing the text vector into a neural network; Obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network by using a back propagation algorithm according to the error term between the output sentence and the text vector; and Adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets a preset condition, and a natural language processing system is obtained. 8. The method for constructing a natural language processing system according to claim 1, wherein the step of extracting several word features from the received sentence text further comprises: The sentence text is divided into several words according to the grammar, and the part of speech of the words in the sentence text is marked according to the part of speech tag set; and The sentence text is decomposed into word features according to the part of speech. 8. The method for constructing a natural language processing system according to claim 1, wherein the step of converting the word feature into a D-dimensional text vector further comprises: Match D-1 entries related to the word feature according to the similarity measurement method; Set D entry weights according to the degree of correlation between the character feature and the entry; and A D-dimensional text vector is generated according to the weights of the D entries. 8. The method for constructing a natural language processing system according to claim 3, wherein the similarity measurement method includes cosine distance. 8. The method for constructing a natural language processing system according to claim 1, wherein the step of calculating the target weight parameter of the neural network through a back propagation algorithm according to the error term of the output sentence and the text vector further comprises: Starting from the output layer of the neural network, calculate the error terms of each hidden layer in reverse order; and The weight parameter of each node of the hidden layer is calculated according to the error term. 2. The method for constructing a natural language processing system according to claim 1, wherein the preset conditions include: the proportion of the sentences that exist in the natural world in the output sentences of the neural network reaches a preset threshold; Before the step that the output sentence meets the preset conditions, it also includes: Determine whether the output sentence is a sentence that exists in nature; Calculating the total number of output sentences of the neural network as the first number, and calculating the total number of output sentences belonging to the sentences existing in nature as the second number; Calculate the ratio of the second number to the first number, and determine whether the ratio reaches a preset threshold. The method for constructing a natural language processing system according to claim 1, wherein the neural network is a deep neural network; The step of converting the word feature into a D-dimensional text vector and transmitting the text vector to a neural network includes: Converting the word feature into a D-dimensional text vector through the word2vec algorithm and/or the doc2vec algorithm; The text vector is passed into the deep neural network, and the text vector is processed through the hidden layer of the deep neural network. An electronic device, which includes: The extraction module is suitable for extracting several word features from the received sentence text; A conversion module, adapted to convert the word feature into a D-dimensional text vector, and pass the text vector into a neural network; The calculation module is adapted to obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network through the back propagation algorithm according to the error term between the output sentence and the text vector; and The adjustment module is adapted to adjust the weight parameter of each node of the neural network according to the target weight parameter, until the output sentence of the neural network meets a preset condition, and a natural language processing system is obtained. A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and capable of running on the processor. The method for constructing a natural language processing system when the processor executes the computer-readable instructions includes the steps : Extract several word features from the received sentence text; Converting the word feature into a D-dimensional text vector, and passing the text vector into a neural network; Obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network by using a back propagation algorithm according to the error term between the output sentence and the text vector; and Adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets a preset condition, and a natural language processing system is obtained. 9. The computer device of claim 9, wherein the step of extracting several word features from the received sentence text further comprises: The sentence text is divided into several words according to the grammar, and the part of speech of the words in the sentence text is marked according to the part of speech tag set; and The sentence text is decomposed into word features according to the part of speech. 9. The computer device of claim 9, wherein the step of converting the word feature into a D-dimensional text vector further comprises: Match D-1 entries related to the word feature according to the similarity measurement method; Set D entry weights according to the degree of correlation between the character feature and the entry; and A D-dimensional text vector is generated according to the weights of the D entries. The computer device according to claim 11, wherein the similarity measurement method includes a cosine distance. 9. The computer device according to claim 9, wherein the step of calculating the target weight parameter of the neural network through a backpropagation algorithm according to the error term between the output sentence and the text vector further comprises: Starting from the output layer of the neural network, calculate the error terms of each hidden layer in reverse order; and The weight parameter of each node of the hidden layer is calculated according to the error term. 9. The computer device according to claim 9, wherein the preset condition comprises: the proportion of the sentences in the natural world in the output sentences of the neural network reaches a preset threshold; and the output sentences up to the neural network meet the preset threshold Before the conditional step, it also includes: Determine whether the output sentence is a sentence that exists in nature; Calculating the total number of output sentences of the neural network as the first number, and calculating the total number of output sentences belonging to the sentences existing in nature as the second number; Calculate the ratio of the second number to the first number, and determine whether the ratio reaches a preset threshold. 9. The computer device of claim 9, wherein the neural network is a deep neural network; The step of converting the word feature into a D-dimensional text vector and transmitting the text vector to a neural network includes: Converting the word feature into a D-dimensional text vector through the word2vec algorithm and/or the doc2vec algorithm; The text vector is passed into the deep neural network, and the text vector is processed through the hidden layer of the deep neural network. A non-volatile computer-readable storage medium has computer-readable instructions stored thereon, and when the computer-readable instructions are executed by a processor, a method for constructing a natural language processing system is realized, including the following steps: Extract several word features from the received sentence text; Converting the word feature into a D-dimensional text vector, and passing the text vector into a neural network; Obtain the output sentence after the neural network receives the text vector, and calculate the target weight parameter of the neural network by using a back propagation algorithm according to the error term between the output sentence and the text vector; and Adjust the weight parameter of each node of the neural network according to the target weight parameter until the output sentence of the neural network meets the preset condition, and the natural language processing system is obtained. 15. The non-volatile computer-readable storage medium of claim 16, wherein the step of extracting several word features from the received sentence text further comprises: The sentence text is divided into several words according to the grammar, and the part of speech of the words in the sentence text is marked according to the part of speech tag set; and The sentence text is decomposed into word features according to the part of speech. 15. The non-volatile computer-readable storage medium of claim 16, wherein the step of converting the word feature into a D-dimensional text vector further comprises: Match D-1 entries related to the word feature according to the similarity measurement method; Set D entry weights according to the degree of correlation between the character feature and the entry; and A D-dimensional text vector is generated according to the weights of the D entries. The non-volatile computer-readable storage medium of claim 18, wherein the similarity measure includes cosine distance. The non-volatile computer-readable storage medium of claim 16, wherein the step of calculating the target weight parameter of the neural network through a backpropagation algorithm according to the error term between the output sentence and the text vector further comprises : Starting from the output layer of the neural network, calculate the error terms of each hidden layer in reverse order; and The weight parameter of each node of the hidden layer is calculated according to the error term.

Images