CN114757210A - Training method of translation model, sentence translation method, apparatus, equipment, program - Google Patents

Abstract

The invention provides a method for training a translation model, which comprises the following steps: obtaining a source terminal statement corresponding to the target translation memory statement in a translation memory library; forming training samples by the source statement and each target translation memory statement, and forming a training sample set by different training samples; processing different training samples in the training sample set through the translation model to determine an updating parameter of the translation model; and according to the updating parameters of the translation model, iteratively updating the encoder parameters and the decoder parameters of the translation model through different training samples in the training sample set. The invention also provides a device, equipment, a software program and a storage medium. The method and the device can enable the trained translation model to be higher in accuracy and better in translation effect, and the embodiment of the invention can also be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like.

Description

Training method of translation model, sentence translation method, apparatus, equipment, program

technical field

The present invention relates to machine translation (MT, Machine Translation) technology, in particular to a training method of a translation model, a sentence translation method, an apparatus, a device, a software program and a storage medium.

Background technique

At present, people often need to translate text or voice in their work and life. Generally, they can use special translation applications or translate web pages for machine translation (MT, Machine Translation), but machine translation sometimes results in translation errors. Therefore, when using machine translation technology in the industry, combining machine-aided translation (CAT, Computer-Aided Translation) is a widely used practice. With the progress and improvement of the MT system, various efficient CAT interaction methods have emerged.

With the development of machine translation, Neural Machine Translation (NMT) is commonly used as a new generation of translation technology. The neural network machine translation system is built based on the encoder-decoder framework. However, in the translation process of the neural network machine translation system, the decoder has multiple tasks, such as recording the content currently translated and the content that needs to be translated, and recording the translated content. Fluency-related information, etc. A translation memory (TMTranslation Memory) is a database that stores pairs of source and target language fragments. Translators can consult this database when translating to improve translation efficiency and consistency. In the machine translation community, early work focused on incorporating translation memory into statistical machine translation models. In recent years, as neural machine translation models have achieved excellent results in various translation tasks, more and more researches aim to integrate translation memory into neural translation models (NMT Neural Machine Translation), but the complex model structure And redundant translation memory affects the training accuracy and training speed of the translation model, which is not conducive to the widespread promotion and use of the translation model.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present invention provide a training method, device, equipment, software program and storage medium for a translation model, which can reduce the model complexity of the translation model, and select translation memory sentences similar to the sentences to be translated through comparison and retrieval. , which can reduce the complex network structure caused by the additional memory network in the related technology, the problem of affecting the training speed and the problem of too long translation time in use. Mechanism to capture the similarity of different translation mechanisms, to ensure the diversity of translation memory (that is, the diversity of training samples), so that the accuracy of the trained translation model is higher, the translation effect is better, and the user experience is improved.

The technical solution of the embodiment of the present invention is realized as follows:

An embodiment of the present invention provides a training method for a translation model, and the training method includes:

Get the target translation memory sentence;

Obtaining the source-end sentence corresponding to the target translation memory sentence in the translation memory;

Forming the source-end sentence and each target translation memory sentence into a training sample, and forming different training samples into a training sample set;

Get the initial parameters of the translation model;

In response to the initial parameters of the translation model, different training samples in the training sample set are processed by the translation model to determine the update parameters of the translation model;

According to the update parameters of the translation model, the encoder parameters and decoder parameters of the translation model are iteratively updated through different training samples in the training sample set.

The embodiment of the present invention provides a method comprising:

Determine at least one word-level latent variable corresponding to the sentence to be translated through the encoder of the translation model;

Through the decoder of the translation model, according to the at least one word-level latent variable, a translation word corresponding to the word-level latent variable and a selection probability of the translated word are generated;

According to the selected probability of the translation result, at least one translation word is selected to form a translation result corresponding to the to-be-translated sentence;

The translation result is output.

The embodiment of the present invention also provides a training device for a translation model, the training device includes:

The data transmission module is used to obtain the target translation memory sentence;

a translation model training module, used to obtain the source-end sentence corresponding to the target translation memory sentence in the translation memory;

The translation model training module is used to form a training sample from the source sentence and the target translation memory sentence, and form a training sample set from different training samples;

The translation model training module is used to obtain the initial parameters of the translation model;

the translation model training module, configured to process different training samples in the training sample set by the translation model in response to the initial parameters of the translation model, and determine the update parameters of the translation model;

The translation model training module is configured to iteratively update the encoder parameters and decoder parameters of the translation model through different training samples in the training sample set according to the update parameters of the translation model.

In the above scheme,

The translation model training module is used to obtain the maximum length of the statement to be translated and the maximum length of any translation memory statement;

The translation model training module is used to obtain the word element distance between the to-be-translated statement and any translation memory statement,

The translation model training module is used to determine the to-be-translated sentence and any of the translation-memory sentences based on the morphological distance, the maximum length of the sentence to be translated and the maximum length of any translation memory sentence similarity;

The translation model training module is configured to determine that any translation memory sentence is the original translation memory sentence corresponding to the to-be-translated sentence when the similarity is greater than or equal to a similarity threshold.

In the above scheme,

The translation model training module is used to calculate the attention value corresponding to each translation memory sentence through the attention function;

The translation model training module is used to fuse translation memory sentences with the same attention value into the same translation memory sentence; or

The translation model training module is used to fuse translation memory sentences with the same attention value into different training samples in the training sample subset.

In the above scheme,

The translation model training module is used to determine a dynamic noise threshold matching the use environment of the translation model;

the translation model training module, configured to perform denoising processing on the training sample set according to the dynamic noise threshold to form a denoising training sample set matching the dynamic noise threshold; or,

The translation model training module is configured to determine a fixed noise threshold corresponding to the translation model, and perform denoising processing on the training sample set according to the fixed noise threshold, so as to form a match with the fixed noise threshold The set of denoising training samples.

In the above scheme,

The translation model training module is configured to perform negative example processing on the training sample set to form a negative example sample set corresponding to the training sample set, wherein the negative example sample set is used for the translation Encoder parameter and decoder parameter tuning of the model.

In the above scheme,

The translation model training module is used to determine the training sample set;

The translation model training module is used to determine the supervision function corresponding to the translation model;

The translation model training module is used to adjust the temperature coefficient of the supervision function;

The translation model training module is configured to perform negative processing on the training sample set through the supervision function based on the vector similarity and different temperature coefficients of any two translation memory sentences in the training set to form A set of negative samples corresponding to the set of training samples.

In the above scheme,

The translation model training module is used to randomly combine the sentences to be output in the decoder of the translation model to form a negative sample set corresponding to the training sample set; or,

The translation model training module is configured to randomly delete or replace the sentences to be output in the decoder of the translation model to form a negative sample set corresponding to the training sample set.

In the above scheme,

The translation model training module is used to substitute different training samples in the training sample set into the loss function corresponding to the auto-encoding network formed by the encoder of the translation model and the decoder;

The translation model training module is used to determine that when the loss function satisfies the convergence condition, the parameters corresponding to the encoder and the corresponding decoder parameters in the translation model are used as update parameters of the translation model.

The embodiment of the present invention also provides a sentence translation device, the device includes:

The encoder module is used to determine at least one word-level latent variable corresponding to the sentence to be translated through the encoder of the translation model;

A decoder module, configured to generate a translation word corresponding to the word-level latent variable and a selected probability of the translated word according to the at least one word-level latent variable through the decoder of the translation model;

The decoder module is configured to select at least one translation word to form a translation result corresponding to the statement to be translated according to the selected probability of the translation result;

The decoder module is used for outputting the translation result.

The embodiment of the present invention also provides an electronic device, the electronic device includes:

memory for storing executable instructions;

The processor is configured to implement the training method of the pre-order translation model when executing the executable instructions stored in the memory.

Embodiments of the present invention also provide a computer-readable storage medium storing executable instructions, characterized in that, when the executable instructions are executed by a processor, a training method for a pre-order translation model, or a pre-order statement is implemented. translation method.

The embodiment of the present invention has the following beneficial effects:

The technical solution provided by the present invention is by obtaining sentences to be translated, and obtaining at least two original translation memory sentences through comparative retrieval based on the sentences to be translated; and performing translation memory fusion processing on the at least two obtained original translation memory sentences to obtain target translation memory sentence; based on each target translation memory sentence, obtain the corresponding source end sentence from the translation memory bank; form the source end sentence and each target translation memory sentence into a training sample, and combine different training samples forming a training sample set; obtaining the initial parameters of the translation model; in response to the initial parameters of the translation model, processing different training samples in the training sample set by the translation model to determine the update parameters of the translation model; According to the update parameters of the translation model, the encoder parameters and decoder parameters of the translation model are iteratively updated through different training samples in the training sample set. Translation memory sentences can reduce the complex network structure caused by the additional memory network in related technologies, the problems that affect the training speed, and the problem that the translation time is too long during use. The attention mechanism is used to capture the similarity of different translation mechanisms to ensure the diversity of translation memory (that is, the diversity of training samples), so that the trained translation model has higher accuracy, better translation effect, and improved user experience; At the same time, it can effectively make full use of the gain of the existing translation memory sentences for model training, so that the translation model can adapt to different usage scenarios.

Description of drawings

1 is a schematic diagram of a usage scenario of a translation model training method provided by an embodiment of the present invention;

2 is a schematic diagram of the composition and structure of a training device for a translation model provided by an embodiment of the present invention;

Fig. 3 is the schematic diagram of generating translation result in the traditional scheme;

4 is an optional schematic flowchart of a training method for a translation model provided by an embodiment of the present invention;

5 is an optional schematic flowchart of a training method for a translation model provided by an embodiment of the present invention;

6 is an optional schematic flowchart of a training method for a translation model provided by an embodiment of the present invention;

Fig. 7 is a schematic diagram of the accuracy test result of the translation model;

Fig. 8 is the schematic diagram of the training efficiency test result of the translation model;

9 is a schematic diagram of the test results of the number of different translation memory sentences in the translation model training method of the application;

FIG. 10 is a schematic diagram of a front-end display interface of a translation model provided by an embodiment of the present invention.

11 is an optional structural schematic diagram of a translation model in an embodiment of the present invention;

12 is a schematic diagram of an optional translation process of the translation model in the embodiment of the present invention;

13 is an optional structural schematic diagram of an encoder in a translation model according to an embodiment of the present invention;

14 is a schematic diagram of vector splicing of an encoder in a translation model according to an embodiment of the present invention;

15 is a schematic diagram of an encoding process of an encoder in a translation model according to an embodiment of the present invention;

16 is a schematic diagram of a decoding process of a decoder in a translation model according to an embodiment of the present invention;

FIG. 17 is a schematic diagram of an output effect of a translation model in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. All other embodiments obtained under the premise of creative work fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" can be the same or a different subset of all possible embodiments, and Can be combined with each other without conflict.

Before further describing the embodiments of the present invention in detail, the terms and terms involved in the embodiments of the present invention are described. The terms and terms involved in the embodiments of the present invention are applicable to the following explanations.

1) For the sentence to be translated, the sentence corresponding to a certain natural language of the translation model is input before the language conversion is performed.

2) The translation result, which is a sentence corresponding to a certain natural language output by the translation model after the language conversion of the source sentence.

3) Reference sentence, a preset reference standard corresponding to a certain natural language after the language conversion of the source sentence.

4) Loyalty, a parameter between 0 and 1 that characterizes the closeness of the content of the target sentence to the content of the source sentence, as a criterion for evaluating the accuracy of the translation of the source sentence. The closer the content is, the more accurate the translation will be.

5) Translation, converting sentences in one natural language into sentences in another natural language.

6) Neural Network (NN): Artificial Neural Network (ANN), referred to as neural network or neural-like network, in the field of machine learning and cognitive science, is a kind of imitation biological neural network (animal center) Mathematical or computational models of the structure and function of the nervous system, especially the brain, used to estimate or approximate functions.

7) Machine Translation (MT): It belongs to the category of computational linguistics, which studies the translation of text or speech from one natural language into another natural language by means of computer programs. Neural Machine Translation (NMT) is a technology that uses neural network technology for machine translation.

9) Speech Translation: Also known as automatic speech translation, it is a technology that translates speech in a natural language into text or speech in another natural language through a computer. Generally, it can be composed of two stages of speech recognition and machine translation. .

10) Encoder-decoder structure: The network structure commonly used in machine translation technology. It consists of an encoder and a decoder. The encoder converts the input text into a series of context vectors that can express the characteristics of the input text. The decoder receives the result of the encoder output as its own input and outputs the corresponding text in another language. text sequence.

11) Translation memory, a database that records natural language sentences (or sentence fragments) and their translations. As the core component of the machine-assisted translation system, it is established and expanded with the use of users to eliminate duplicate translations and improve work efficiency. For example, translation memory can store bilingual translation sentence pairs (sample pairs), in which The bilingual translation sentence pairs can be manually translated or collected through other means (such as translation information in web page information collected by web crawler programs). If the sentence to be translated provided by the user is "I want to eat Hamburger", the corresponding original translation memory sentences "I like to eat hamburger", "I want to eat French fries", and the source-side sentences corresponding to the original translation memory sentences "Ilike eat Hamburger", " I want to eat Frenchfries", the above retrieved {I like to eat hamburger, I like eat Hamburger}, {I want to eat French fries, Iwant to eat French fries} can be encoded into the translation model to guide the translation model to decode, In order to obtain the translation result "I want to eat Hamburger" outputted by the translation model and corresponding to the sentence to be translated. In some embodiments, the source sentence and the target translation memory sentence may also be composed of training samples to train the translation model, so as to improve the translation accuracy of the translation model.

FIG. 1 is a schematic diagram of a usage scenario of a translation model training method provided by an embodiment of the present invention. Referring to FIG. 1 , a terminal (including a terminal 10-1 and a terminal 10-2) is provided with a client of translation software. The software client can input the corresponding sentence to be translated, and the chat client can also receive the corresponding translation result, and display the received translation result to the user; the terminal is connected to the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, Or a combination of the two, using a wireless link for data transmission.

As an example, the server 200 is configured to deploy the translation model and train the translation model, so as to update the parameters of the encoder network and the decoder network in the translation model, so as to realize The network and the decoder network generate translation results for the target sentence to be translated, and display the translation results corresponding to the sentence to be translated generated by the translation model through the terminal (terminal 10-1 and/or terminal 10-2). In order to better understand the methods provided by the embodiments of the present application, artificial intelligence, various branches of artificial intelligence, and application fields, cloud technologies, and artificial intelligence cloud services involved in the methods provided by the embodiments of the present application will be described first.

Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning. Each direction will be described below.

Natural language processing (NLP, Nature Language processing) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, the language that people use on a daily basis, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

Machine Learning (ML, Machine Learning) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other techniques.

Cloud technology refers to a kind of hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing and sharing of data. Cloud technology is based on the general term of network technology, information technology, integration technology, management platform technology, application technology, etc. applied in the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. Background services of technical network systems require a lot of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the back-end system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need to be strong. The system backing support can only be achieved through cloud computing.

The so-called artificial intelligence cloud service, also known as AI as a Service (AIaaS, AI as a Service), is the current mainstream service mode of artificial intelligence platform. Specifically, the AIaaS platform will provide several types of common AI services. Split and provide standalone or packaged services in the cloud. This service model is similar to opening an AI-themed mall: all developers can access one or more artificial intelligence services provided by the platform through API interfaces, and some senior developers can also use the platform Provided AI framework and AI infrastructure to deploy and operate their own exclusive cloud AI services.

The solutions provided in the embodiments of the present application involve technologies such as artificial intelligence natural language processing, machine learning, and artificial intelligence cloud services, and are specifically described by the following embodiments.

The translation model-based training method provided by the embodiment of the present application will be described with reference to the exemplary application and implementation of the terminal provided by the embodiment of the present application.

Of course, before the target sentence to be translated is processed by the translation model to generate the corresponding translation result, the translation model needs to be trained, which specifically includes: obtaining the sentence to be translated, and obtaining at least two sentences based on the sentence to be translated through comparative retrieval original translation memory sentences; perform translation memory fusion processing on at least two obtained original translation memory sentences to obtain target translation memory sentences; based on each target translation memory sentence, obtain the corresponding source-end sentences from the translation memory; The source sentence and each target translation memory sentence form a training sample, and different training samples form a training sample set; obtain the initial parameters of the translation model; in response to the initial parameters of the translation model, through the translation model Process different training samples in the training sample set to determine the update parameters of the translation model; according to the update parameters of the translation model, encode the translation model through different training samples in the training sample set Iteratively update the decoder parameters and decoder parameters.

The structure of the training device for the translation model according to the embodiment of the present invention will be described in detail below. The training device for the translation model may be implemented in various forms. For example, the electronic device in the embodiment of the present application may be a dedicated terminal with a translation model training function. It may also be a server provided with a translation model training function, such as the server 200 in the preceding figure. FIG. 2 is a schematic diagram of the composition and structure of a training device for a translation model provided by an embodiment of the present invention. It can be understood that FIG. 2 only shows an exemplary structure of the training device for a translation model, but not the entire structure. part or all of the structure.

The apparatus for training a translation model provided by the embodiment of the present invention includes: at least one processor 201 , a memory 202 , a user interface 203 , and at least one network interface 204 . The various components in the training apparatus of the translation model are coupled together by the bus system 205 . It will be understood that the bus system 205 is used to implement the connection communication between these components. In addition to the data bus, the bus system 205 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the various buses are labeled as bus system 205 in FIG. 2 .

The user interface 203 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen, and the like.

It will be appreciated that the memory 202 may be either volatile memory or non-volatile memory, and may include both volatile and non-volatile memory. The memory 202 in the embodiment of the present invention can store data to support the operation of the terminal (eg 10-1). Examples of such data include: any computer program used to operate on the terminal (eg 10-1), such as operating systems and applications. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. Applications can contain various applications.

In some embodiments, the apparatus for training translation models provided by the embodiments of the present invention may be implemented in a combination of software and hardware. As an example, the apparatus for training translation models provided in the embodiments of the present invention may be processed in the form of hardware decoding processors. It is programmed to execute the translation model training method provided by the embodiment of the present invention. For example, the processor in the form of a hardware decoding processor may adopt one or more Application Specific Integrated Circuits (ASIC, Application Specific Integrated Circuit), DSP, Programmable Logic Device (PLD, Programmable Logic Device), Complex Programmable Logic Device (CPLD, Complex Programmable Logic Device), Field Programmable Gate Array (FPGA, Field-Programmable Gate Array) or other electronic components.

As an example in which the apparatus for training the translation model provided by the embodiment of the present invention adopts a combination of software and hardware, the apparatus for training the translation model provided in the embodiment of the present invention may be directly embodied as a combination of software modules executed by the processor 201, and the software modules may be located in the In the storage medium, the storage medium is located in the memory 202, and the processor 201 reads the executable instructions included in the software module in the memory 202, and implements the implementation of the present invention in combination with necessary hardware (for example, including the processor 201 and other components connected to the bus 205). Example of the translation model training method provided.

As an example, the processor 201 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices , discrete hardware components, etc., where a general-purpose processor may be a microprocessor or any conventional processor, or the like.

As an example in which the apparatus for training the translation model provided by the embodiment of the present invention is implemented in hardware, the apparatus provided by the embodiment of the present invention may be directly executed by the processor 201 in the form of a hardware decoding processor, for example, by one or more Application Specific Integrated Circuit (ASIC, Application Specific Integrated Circuit), DSP, Programmable Logic Device (PLD, Programmable Logic Device), Complex Programmable Logic Device (CPLD, Complex Programmable Logic Device), Field Programmable Gate Array (FPGA, Field- Programmable GateArray) or other electronic components to implement the translation model training method provided by the embodiment of the present invention.

The memory 202 in the embodiment of the present invention is used to store various types of data to support the operation of the training apparatus of the translation model. Examples of these data include: any executable instructions for operating on the translation model training device, such as executable instructions, a program implementing the method for training from translation models according to embodiments of the present invention may be included in the executable instructions.

In other embodiments, the training device for the translation model provided by the embodiments of the present invention may be implemented in software. FIG. 2 shows the training device for the translation model stored in the memory 202, which may be in the form of programs and plug-ins. The software includes a series of modules. As an example of the program stored in the memory 202, it may include a training device for a translation model. The training device for a translation model includes the following software modules: a data transmission module 2081, and a translation model training module 2082. When the software module in the translation model training device is read into the RAM by the processor 201 and executed, the translation model training method provided by the embodiment of the present invention will be implemented. The following describes each of the translation model training device in the embodiment of the present invention. software module functions, which,

The data transmission module 2081 is used to obtain the target translation memory sentence.

The translation model training module 2082 is configured to acquire the source-end sentence corresponding to the target translation memory sentence in the translation memory.

The translation model training module 2082 is configured to acquire the corresponding source sentence from the translation memory based on each target translation memory sentence.

The translation model training module 2082 is configured to compose the source sentence and each target translation memory sentence into a training sample, and compose different training samples into a training sample set.

The translation model training module 2082 is used to obtain initial parameters of the translation model.

The translation model training module 2082 is configured to, in response to the initial parameters of the translation model, process different training samples in the training sample set through the translation model to determine the update parameters of the translation model.

The translation model training module 2082 is configured to iteratively update the encoder parameters and decoder parameters of the translation model through different training samples in the training sample set according to the update parameters of the translation model.

In some embodiments of the present invention, when the trained translation model is deployed, the electronic device in the embodiment may further include a sentence translation apparatus. Specifically, the sentence translation apparatus includes:

The encoder module is used for determining at least one word-level latent variable corresponding to the sentence to be translated through the encoder of the translation model. The decoder module is configured to generate, through the decoder of the translation model, a translation word corresponding to the word-level latent variable and a selection probability of the translated word according to the at least one word-level latent variable. The decoder module is configured to select at least one translation word to form a translation result corresponding to the to-be-translated sentence according to the selected probability of the translation result. The decoder module is used for outputting the translation result.

The training method of the translation model provided by the embodiment of the present invention is described in conjunction with the training device of the translation model shown in FIG. 2 . Before introducing the training method of the translation model provided by the embodiment of the present invention, the translation model in this application is first introduced according to the sentence to be translated. In the process of generating the corresponding translation result, Figure 3 is a schematic diagram of generating the translation result in the traditional scheme, in which, taking the Transformer framework as the structure of the translation model as an example, it mainly includes the encoder-embeddings layer (encoder word vector layer), encoder/decoder ith (1≦ ith≦6) layer (encoder/decoder 1 to 6 layers), decoder-embeddings layer (decoder word vector layer) and decoder softmax layer (decoder output layer) and other functional components. Among them, each layer of encoder/decoder is also composed of some other basic units. All components are organically combined to form a layer in the network, and then stacked layer by layer to form the entire network. Among them, each layer of encoder converts the input sentence (source language) into a semantic vector, and each layer of the decoder converts the semantic vector into an output sentence (target language). In this process, when using translation memory to help training, TM retrieval is first performed. This step searches the database for translation fragments that are similar to the current input by means of a similarity measure. Then use an additional dual-encoder (Dual-Encoder) structure or directly combine TM and X with a unified vocabulary, and input it to the translation model for training. The disadvantage of this method is that the sentences most similar to the currently translated sentence are selected in turn from the database, but the retrieved sentences are ignored, resulting in all the retrieved sentences repeating each other, and redundant content brings less information gain information, which affects the efficiency of model training. At the same time, the translation model of the related art processes the retrieved multiple TMs individually instead of treating them as a unified whole, which affects the translation effect of the translation model.

In order to solve the defect in this related art, with reference to FIG. 4, FIG. 4 is an optional schematic flowchart of the training method of the translation model provided by the embodiment of the present invention. It is understood that the steps shown in FIG. 4 can be executed by running the translation model. Various electronic devices of the training apparatus are executed, such as a dedicated terminal with a model training function, a server with a translation model training function, or a server cluster. The steps shown in FIG. 4 will be described below.

Step 401: The translation model training apparatus obtains a target translation memory sentence, and obtains a source-end sentence corresponding to the target translation memory sentence in a translation memory database.

In some embodiments of the present invention, when obtaining the target translation memory sentence, first obtain the sentence to be translated; then search based on the sentence similarity between the sentence to be translated and the original translation memory sentence, and obtain a sentence matching the sentence to be translated At least two original translation memory sentences are obtained; finally, the obtained at least two original translation memory sentences are subjected to translation memory fusion processing to obtain the target translation memory sentences. Among them, the translation memory can store different original translation memory sentences. For example, the translation memory can store the translation information provided by historical users, that is, every time the user initiates a translation request, the translation memory can respond to the user's translation request. The translation information provided by the user is stored accordingly, and each stored translation information can be used as the original translation memory sentence; or the trained translation model automatically translates the web pages in different languages collected by the crawler, and saves the translation results In the translation memory, as the original translation memory sentence, there is no specific limitation on this in this application.

In some embodiments of the present invention, taking sentence translation in the game display interface as an example, for example, Japanese sentences and translated Chinese sentences displayed on the display interface in game A can be used as target translation memory sentences and corresponding The source sentence is stored in the translation memory. When you need to memorize the translation of the Japanese sentence displayed on the display interface in the B game, you can obtain the target translation in the translation memory that matches the to-be-translated sentence in the B game. The memorized sentences and the corresponding source-side sentences complete the training of the translation model.

In some embodiments of the present invention, retrieval is performed based on the sentence similarity between the sentence to be translated and the original translation memory sentence to obtain at least two original translation memory sentences matching the to-be-translated sentence, which can be achieved in the following manner :

Obtain the maximum length of the statement to be translated and the maximum length of any translation memory statement; obtain the morpheme distance between the statement to be translated and any translation memory statement, based on the morpheme distance, the statement to be translated The maximum length of the maximum length and the maximum length of the any translation memory sentence, determine the similarity between the to-be-translated sentence and the any translation memory sentence; when the similarity is greater than or equal to the similarity threshold, determine the any The translation memory sentence is the original translation memory sentence corresponding to the to-be-translated sentence. Wherein, the input sentence to be translated is x, and y is any translation memory sentence. According to formula 1, the method of comparative retrieval is used to retrieve the sentence similarity between the sentence to be translated and the original translation memory sentence to obtain the sentence similar to the sentence to be translated. At least two original translation memory sentences that match, for example, k (k is an integer greater than or equal to 2) translation memory sentences can be found in the translation memory:

Among them, (x, x ⁱ ) and x ⁱ , x ^j are the two groups of sentences to be translated and the original translation memory sentence. For example, the input sentence to be translated is "I want to eat an apple", and the original translation memory sentence is "I want to eat."Strawberry" and "I'm going to eat instant noodles". Through translation memory fusion processing, the target translation memory sentence can be obtained as "I want to eat strawberries and plan to eat instant noodles".

When performing similarity retrieval, the specific calculation method refers to formula 2, where Sim(x, y) is the similarity measurement function between the sentence to be translated and any translation memory sentence:

Among them, the numerator item D _edit (x, y) in Formula 2 is the edit distance of the token (Token) between any two sentences. The denominator term max(|x|,|y|) is the maximum length of the sentence to be translated and any translation memory sentence. The similarity can be normalized by the maximum length of the sentence to be translated and any translation memory sentence. Therefore, by comparing the sentence similarity between the sentence to be translated and different original translation memory sentences, a higher original translation memory sentence matching the sentence to be translated can be obtained, ensuring the accuracy of the translation model and reducing the errors of the translation model The impact of translation on users.

Step 402: The translation model training apparatus performs translation memory fusion processing on the at least two acquired original translation memory sentences to obtain a target translation memory sentence.

In some embodiments of the present invention, translation memory fusion processing is performed on at least two obtained original translation memory sentences to obtain a target translation memory sentence, which can be achieved in the following ways:

The attention value corresponding to each translation memory sentence is calculated by the attention function; the translation memory sentences with the same attention value are fused into the same translation memory sentence; or the translation memory sentences with the same attention value are fused into different training sample subsets Training samples. Among them, the attention value corresponding to each translation memory sentence can be calculated by the attention function. For details, please refer to formula 3:

为翻译记忆节点v_i在t+1时刻的状态，又该节点的邻居节点在t时刻的状态和该节点在t时刻的状态表示经过自注意力机制更新而来。同时可以为每个翻译记忆库配置设一个超节点，超节点的作用是为了让不同翻译记忆语句之间可以进行信息沟通和交互，同时通一个翻译记忆内部库的节点是无法看到周围节点的内容，保证了语义的完整性，便于在不同的使用环境中调用原始的翻译记忆语句。通过翻译记忆融合，利用注意力机制来捕捉不同翻译机制相似性，保证翻译记忆的多样性(也就是训练样本的多样性)，使得经过训练翻译模型的精确度更高，翻译效果更好。in

is the state of translation memory node v _i at time t+1, and the state of the node’s neighbor nodes at time t and the state of this node at time t are updated by the self-attention mechanism. At the same time, a super node can be configured for each translation memory. The function of the super node is to allow information communication and interaction between different translation memory sentences. At the same time, the nodes of a translation memory internal database cannot see the surrounding nodes. The content ensures the integrity of the semantics and facilitates the recall of the original translation memory sentences in different usage environments. Through translation memory fusion, the attention mechanism is used to capture the similarity of different translation mechanisms, so as to ensure the diversity of translation memory (that is, the diversity of training samples), so that the accuracy of the trained translation model is higher and the translation effect is better.

In some embodiments of the present invention, in order to speed up the training speed of the translation model, the translation memory sentences with the same attention value can also be discarded, and only the translation memory sentences with different and the same attention value are retained, so as to reduce the volume of the training sample set .

Step 403: The translation model training apparatus acquires the corresponding source-end sentence from the translation memory database based on each target translation memory sentence.

Step 404 : The translation model training apparatus composes the source sentence and each target translation memory sentence into a training sample, and composes different training samples into a training sample set.

In some embodiments of the present invention, when the number of training samples in the training sample set exceeds the threshold of the number of training samples, the training sample set needs to be de-noised, specifically including:

determining a dynamic noise threshold that matches the context in which the translation model is used;

Perform denoising processing on the training sample set according to the dynamic noise threshold to form a denoising training sample set matching the dynamic noise threshold; or,

A fixed noise threshold corresponding to the translation model is determined, and the training sample set is denoised according to the fixed noise threshold to form a denoised training sample set matching the fixed noise threshold. Wherein, due to the different usage environments of the translation models, the dynamic noise thresholds that match the usage environments of the translation models are also different. For example, in the usage environments of academic translations, the dynamic noise thresholds that match the usage environments of the translation models Need to be less than the dynamic noise threshold in the article reading environment.

In some embodiments of the present invention, when the translation model is fixed in the corresponding hardware mechanism, and the use environment is oral translation, by fixing the fixed noise threshold corresponding to the translation model, the training speed of the translation model can be effectively improved, and users can be reduced. waiting time.

After the training sample set is determined, step 405 is continued.

Step 405: The translation model training apparatus obtains initial parameters of the translation model.

Step 406 : The translation model training apparatus, in response to the initial parameters of the translation model, processes different training samples in the training sample set through the translation model, and determines the update parameters of the translation model.

Step 407: The translation model training device iteratively updates the encoder parameters and decoder parameters of the translation model through different training samples in the training sample set according to the update parameters of the translation model.

In some embodiments of the present invention, initializing the decoder network to update the parameters of the decoder network can be implemented in the following ways:

Encoding the sentence to be translated by the encoder of the decoder network to form the encoding result of the sentence to be translated; decoding the encoding result of the sentence to be translated by the decoder of the decoder network; When the selected probability of the translation result corresponding to the sentence to be translated is obtained by decoding, the parameters of the decoder network are determined. For example, the decoder network that initializes the trained translation model generates three words that generate translation results according to the corresponding sentences to be translated, namely translation result a (with probability 0.45), translation result b (with probability 0.5) and translation result c (probability is 0.45), then the probability distribution is {0.45, 0.5, 0.45}.

Thus, it can be realized that the translation model can output the translation result with the smallest loss value according to the corresponding sentence to be translated.

In some embodiments of the present invention, in response to the initial parameters of the translation model, different training samples in the training sample set are processed by the translation model, and the update parameters of the translation model are determined by the following way to achieve:

Substitute different training samples in the training sample set into the loss function corresponding to the auto-encoding network composed of the encoder and the decoder of the translation model; when it is determined that the loss function satisfies the convergence condition, it corresponds to the translation model. The parameters of the encoder and the corresponding decoder parameters are used as the update parameters of the translation model. Among them, the loss function of the encoder network is expressed as:

loss_A=∑(decoder_A(encoder(warp(x1)))-x1)2; specifically, decoder_A is the decoder A, warp is the function _of the sentence to be translated, x1 is the sentence to be translated, and encoder is the encoder.

In the process of iterative training, by substituting the sentence to be translated into the loss function of the encoder network, the parameters of encoder A and decoder A when the loss function descends according to the gradient (such as the maximum gradient) are solved. When the loss function converges (that is, determine the When the word-level latent variable corresponding to the sentence to be translated can be formed), the training is ended.

In the training process of the encoder network, the loss function of the encoder network is expressed as: loss_B=∑(decoder_B(encoder(warp(x2)))-x2)2; wherein, decoder_B is the decoder B, and warp is the sentence to be translated function, x2 is the sentence to be translated, and encoder is the encoder.

In the process of iterative training, by substituting the sentence to be translated into the loss function of the encoder network, the parameters of encoder B and decoder B are solved when the loss function descends according to the gradient (such as the maximum gradient); when the loss function converges (that is, when the loss function converges) When the selected probability of the translation result corresponding to the sentence to be translated is obtained by decoding), the training is ended.

Therefore, it can be realized that the translation model can output the translation result with the smallest loss value according to the corresponding sentence to be translated, so as to ensure the accuracy of the translation result.

In some embodiments of the present invention, the method further includes:

Negative example processing is performed on the training sample set to form a negative example sample set corresponding to the training sample set, wherein the negative example sample set is used to adjust the encoder parameters and decoder parameters of the translation model Adjustment.

In order to further illustrate the process of obtaining negative samples, continue to refer to FIG. 5 , which is an optional schematic flowchart of a training method for a translation model provided by an embodiment of the present invention. It is understood that the steps shown in FIG. 5 can be executed by running Various electronic devices of the translation model training apparatus can execute, for example, a dedicated terminal with a model training function, a server with a translation model training function, or a server cluster. The steps shown in FIG. 5 will be described below.

Step 501: Determine the supervision function corresponding to the translation model.

Step 502: Adjust the temperature coefficient of the supervisory function.

Step 503: Based on the vector similarity and different temperature coefficients of any two translation memory sentences in the training set, perform negative processing on the training sample set through the supervisory function to form the same as the training sample set. The corresponding negative sample set. Among them, the supervision function refers to formula 4:

Among them, sim(y, y') represents the similarity of the vectors of any two translation memory sentences, and the similarity calculation can be the standard method of machine translation evaluation (BLEU, Bilingual Evaluation Understudy), cosine similarity (Cosine Similarity) The specific similarity calculation method is not limited in this application. τ is the temperature coefficient, which is used to control the difficulty of distinguishing positive and negative samples in comparative learning. By adjusting the temperature coefficient, the proportion of negative samples can be flexibly controlled.

In some embodiments of the present invention, the encoder corresponding to the translation model and the corresponding decoder may also be a bidirectional network model. For example, the Bi-GRU bidirectional GRU model may be selected as the corresponding encoder and the corresponding decoder. Here The Bi-GRU bidirectional GRU model is a model that can recognize inverted sentence structures. When the user enters a dialogue sentence, the dialogue sentence may be an inverted sentence structure, which is different from the normal sentence structure. For example, the dialogue sentence input by the user is "how is the weather today", while the normal sentence structure is "today." How is the weather?”, the Bi-GRU bidirectional GRU model can identify the dialogue sentences with the inverted sentence structure, which can enrich the functions of the trained model and improve the robustness of the final trained target model.

Continuing to describe the text sentence processing method of the translation model provided by the embodiment of the present invention in conjunction with the translation model training device shown in FIG. 2, referring to FIG. 6, FIG. 6 is an optional text sentence processing method of the translation model provided by the embodiment of the present invention. A schematic flowchart, it is understood that the steps shown in FIG. 6 can be executed by various electronic devices running the translation model training device, for example, can be a dedicated terminal with a function of processing sentences to be translated, a server with a function of processing sentences to be translated, or a server cluster. The steps shown in FIG. 6 will be described below.

Step 601: Determine at least one word-level latent variable corresponding to the sentence to be translated through the encoder of the translation model.

Step 602 : Generate a translation word corresponding to the word-level latent variable and a selection probability of the translated word according to the at least one word-level latent variable through the decoder of the translation model.

Step 603: According to the selected probability of the translation result, select at least one translation word to form a translation result corresponding to the to-be-translated sentence.

Step 604: Output the translation result.

When testing the trained translation model, refer to Figures 7 to 9, wherein the model training methods of the related art are: Vaswani et al., 2017, Gu et al., 2018, Zhang et al., 2018, Xu et al., 2020*, Xia et al., 2019, He et al., 2021(@s), Cai et al., 2021(#2), the training method of this application is T-Full, and Figure 7 is the translation A schematic diagram of the accuracy test results of the model, FIG. 8 is a schematic diagram of the training efficiency test results of the translation model, and FIG. 9 is a schematic diagram of the test results of different translation memory sentence numbers in the translation model training method of the application, and the accuracy score of T-Full is 58.69 to 67.76, which is significantly higher than the training method in the related art, indicating that the present application can effectively make full use of the gains of the existing translation memory sentences for model training, so that the translation model can adapt to different usage scenarios.

FIG. 10 is a schematic diagram of a front-end display interface of a translation model provided by an embodiment of the present invention. The translation model shown in this embodiment can process sentences to be translated and generate corresponding translated texts. Among them, the target to be translated is "When I can find the time, I want to go camping and light a bonfire with my friends" entered by the user through the translation applet of the instant messaging client.

Through the processing of the translation model, corresponding translation texts are formed for the user to select and the selection probability of the translation result.

According to the selected probability of the described translation result, the selected translation result constitutes the translation result corresponding to the statement to be translated and includes the following three combinations:

1) "When I can find time, I want to go camping and light a bonfire with my friends".

2) "When I find time, go camping and light a bonfire with your friends".

3) "When you find time, go camping friends and light a campfire".

Thus, through the translation model provided by the present invention, it can be realized that a plurality of different translation results can be generated according to the same sentence to be translated.

The following describes the use process of the translation model provided by the embodiment of the present invention in conjunction with the specific structure of the translation model. Since the virtual objects and virtual scenes in the game are in English or Japanese, domestic users often cannot understand the meanings of virtual objects and virtual scenes in time. Therefore, through the translation model, the foreign language meaning of virtual objects and virtual scenes can be obtained in time. The trained translation model can translate the text information in the game scene of the Japanese game server. The translation model is a transformer structure.

Continuing to refer to FIG. 11, FIG. 11 is an optional structural schematic diagram of a translation model in an embodiment of the present invention, wherein the Encoder includes: N=6 identical layers, and each layer includes two sub-layers. The first sub-layer is a multi-head attention layer followed by a simple fully connected layer. Each of the sub-layers adds residual connection and normalisation.

Decoder includes: It consists of N=6 identical layers, where the layer and the encoder are not the same. The layer here contains three sub-layers, including a self-attention layer. The encoder-decoderattention layer is finally a fully connected layer. . The first two sub-layers are based on the multi-headattention layer. Specifically, Nx on the left represents the structure of one of the layers of the encoder, which includes two sublayers, the first sublayer is a multi-head attention layer, and the second sublayer is a forward propagation layer. The input and output of each sublayer are associated, and the output of the current sublayer is used as an input data for the next sublayer. Each sublayer is followed by a normalization operation, which can improve the convergence speed of the model. The Nx on the right represents the structure of one of the layers of the decoder. One layer of the decoder includes three sub-layers. The first sub-layer is the multi-head attention sub-layer controlled by the mask matrix, which is used to model the generated target. The sentence vector, in the training process, needs a mask matrix to control, so that each time the multi-head attention is calculated, only the first t-1 words are calculated. The second sub-layer is the multi-head attention sub-layer, which is the attention mechanism between the encoder and the decoder, that is, to find relevant semantic information in the source text. The calculation of this layer uses the dot product method. The third sublayer is the forward propagation sublayer, which is calculated in the same way as the forward propagation sublayer in the encoder. There is also an association between each sub-layer of the decoder, and the output of the current sub-layer is used as an input data for the next sub-layer. And each sub-layer of the decoder is also followed by a normalization operation to speed up the model convergence.

Continuing to refer to FIG. 12 , FIG. 12 is a schematic diagram of an optional translation process of the translation model in the embodiment of the present invention, wherein the encoder and decoder parts both include 6 encoders and decoders. The inputs to the first encoder are combined with embedding and positional embedding. After passing through 6 encoders, the output is output to each decoder in the decoder part; the input sentence to be translated is Japanese "Unparalleled の Demon Lu Bu は Yan っ た." After processing by the translation model, the original translation sentence "Cang Tian Xiang Long Zhao Yun は 言います" is stored in the translation memory. "The brave man's oath, life and death よ り も even だ し い! Fear れ な い and heart に hug い て, empty を flying べ ま す."; and the source sentence "Sky Xianglong Zhao Yun said: "The oath of the brave is more than life and death! Don't be afraid, you can soar in the sky! ”; after obtaining the Chinese source sentence corresponding to the target translation memory sentence that is also Japanese in the translation memory, the translation model is trained, and the final translation result output by the translation model is: “The Demon of Unparalleled Lu Bu said:” From this moment on, the battlefield will be dominated by me alone! But someone dares to fight me!"

Continuing to refer to FIG. 13, FIG. 13 is an optional structural schematic diagram of the encoder in the translation model in the embodiment of the present invention, wherein the input consists of a query (Q) and a key (K) with a dimension of d and a value ( V) composition, all keys compute the dot product of the query, and apply the softmax function to get the weights of the values.

Continue to refer to Fig. 13, Fig. 13 is a vector diagram of the encoder in the translation model in the embodiment of the present invention, wherein Q, K and V are multiplied by W^Q, W^K, W^V through the vector x of the input encoder Get Q, K and V. The dimension of W^Q, W^K, W^V in the article is (512, 64), and then it is assumed that the dimension of the available inputs is (m, 512), where m represents the number of words. So the dimensions of Q, K and V obtained by multiplying the input vector with W^Q, W^K, W^V are (m, 64).

Continuing to refer to FIG. 14, FIG. 14 is a schematic diagram of vector splicing of the encoder in the translation model in the embodiment of the present invention, wherein Z0 to Z7 are the corresponding 8 parallel heads (dimensions are (m, 64)), and then concat the 8 After one head, the (m, 512) dimension is obtained. Finally, after multiplying with W^O, the output matrix of dimension (m, 512) is obtained, then the dimension of this matrix is consistent with the dimension of entering the next encoder.

Continuing to refer to FIG. 15, FIG. 15 is a schematic diagram of the encoding process of the encoder in the translation model in the embodiment of the present invention, wherein the tensor passing through the self-attetion also needs to go through the processing of the residual network and Later Norm, and then enter the full connection In the feed-forward network of , the feed-forward network needs to perform the same operation, the residual processing and normalization. The final output tensor can be entered into the next encoder, and then such an operation is iterated 6 times, and the result of the iterative processing enters the decoder.

Continue to refer to Figure 16, Figure 16 is a schematic diagram of the decoding process of the decoder in the translation model in the embodiment of the present invention, wherein, the input and output of the decoder and the decoding process:

Output: the probability distribution of the output word corresponding to the i position;

Input: the output of the encoder & the output of the decoder corresponding to the i-1 position. So the middle attention is not self-attention, its K and V come from the encoder, and Q comes from the output of the previous position decoder.

17 is a schematic diagram of the output effect of the translation model in the embodiment of the present invention. After the translation model is trained by the translation model training method provided by the present application, the user input in the game scene and the foreign language content displayed to the user can be translated, which is convenient User's understanding and use.

To sum up, the embodiments of the present invention have the following technical effects:

The technical solution provided by the present invention is by obtaining sentences to be translated, and obtaining at least two original translation memory sentences through comparative retrieval based on the sentences to be translated; and performing translation memory fusion processing on the at least two obtained original translation memory sentences to obtain target translation memory sentence; based on each target translation memory sentence, obtain the corresponding source end sentence from the translation memory bank; form the source end sentence and each target translation memory sentence into a training sample, and combine different training samples forming a training sample set; obtaining the initial parameters of the translation model; in response to the initial parameters of the translation model, processing different training samples in the training sample set by the translation model to determine the update parameters of the translation model; According to the update parameters of the translation model, the encoder parameters and decoder parameters of the translation model are iteratively updated through different training samples in the training sample set. Translation memory sentences can reduce the complex network structure caused by the additional memory network in related technologies, the problems that affect the training speed, and the problem that the translation time is too long during use. The attention mechanism is used to capture the similarity of different translation mechanisms to ensure the diversity of translation memory (that is, the diversity of training samples), so that the trained translation model has higher accuracy, better translation effect, and improved user experience; At the same time, it can effectively make full use of the gain of the existing translation memory sentences for model training, so that the translation model can adapt to different usage scenarios.

The above descriptions are only examples of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (15)

1. A method for training a translation model, the method comprising:

acquiring a target translation memory statement;

obtaining a source terminal statement corresponding to the target translation memory statement in a translation memory library;

forming training samples by the source-end sentences and the target translation memory sentences, and forming training sample sets by different training samples;

acquiring initial parameters of a translation model;

responding to initial parameters of the translation model, processing different training samples in the training sample set through the translation model, and determining updating parameters of the translation model;

and according to the updating parameters of the translation model, iteratively updating the encoder parameters and the decoder parameters of the translation model through different training samples in the training sample set.

2. The method of claim 1, wherein obtaining the target translation memory statement comprises:

obtaining a sentence to be translated;

retrieving based on the sentence similarity between the sentence to be translated and the original translation memory sentence to obtain at least two original translation memory sentences matched with the sentence to be translated;

and performing translation memory fusion processing on the obtained at least two original translation memory sentences to obtain the target translation memory sentence.

3. The method according to claim 2, wherein the retrieving based on the sentence similarity between the sentence to be translated and the original translation memory sentence to obtain at least two original translation memory sentences matched with the sentence to be translated comprises:

acquiring the maximum length of the statement to be translated and the maximum length of any translation memory statement;

obtaining the word element distance between the sentence to be translated and any translation memory sentence,

determining the similarity between the sentence to be translated and any translation memory sentence based on the word element distance, the maximum length of the sentence to be translated and the maximum length of any translation memory sentence;

and when the similarity is greater than or equal to a similarity threshold value, determining that any translation memory statement is an original translation memory statement corresponding to the statement to be translated.

4. The method according to claim 1, wherein the performing translation memory fusion processing on the obtained at least two original translation memory statements to obtain a target translation memory statement comprises:

calculating an attention value corresponding to each translation memory statement through an attention function;

fusing the translation memory sentences with the same attention value into the same translation memory sentence; or

The translated memory statements of the same attention value are fused into different training samples in the subset of training samples.

5. The method of claim 1, further comprising:

determining a dynamic noise threshold value matched with the use environment of the translation model;

denoising the training sample set according to the dynamic noise threshold value to form a denoising training sample set matched with the dynamic noise threshold value; or,

and determining a fixed noise threshold corresponding to the translation model, and denoising the training sample set according to the fixed noise threshold to form a denoising training sample set matched with the fixed noise threshold.

6. The method of claim 1, further comprising:

and carrying out negative example processing on the training sample set to form a negative example sample set corresponding to the training sample set, wherein the negative example sample set is used for adjusting the encoder parameters and the decoder parameters of the translation model.

7. The method of claim 6, wherein the negative case processing the set of training samples comprises:

determining a supervision function corresponding to the translation model;

adjusting a temperature coefficient of the supervisory function;

and carrying out negative example processing on the training sample set through the supervision function based on the vector similarity and different temperature coefficients of any two translation memory sentences in the training sample set to form a negative example sample set corresponding to the training sample set.

8. The method of claim 6, wherein the negative case processing the set of training samples comprises:

randomly combining statements to be output in a decoder of the translation model to form a negative sample set corresponding to the training sample set; or,

and carrying out random deletion processing or replacement processing on the sentences to be output in a decoder of the translation model to form a negative example sample set corresponding to the training sample set.

9. The method of claim 1, wherein the determining updated parameters of the translation model by processing different training samples in the set of training samples by the translation model in response to initial parameters of the translation model comprises:

substituting different training samples in the training sample set into a loss function corresponding to a self-coding network formed by an encoder and a decoder of the translation model;

and determining parameters corresponding to an encoder and corresponding decoder parameters in the translation model when the loss function meets the convergence condition as update parameters of the translation model.

10. A sentence translation method, the method comprising:

determining at least one word-level hidden variable corresponding to a sentence to be translated through an encoder of a translation model;

generating, by a decoder of the translation model, a translated term corresponding to the word-level hidden variable and a selected probability of the translated term according to the at least one word-level hidden variable;

selecting at least one translation word to form a translation result corresponding to the sentence to be translated according to the selection probability of the translation result;

outputting the translation result;

wherein the translation model is trained based on the method of any one of claims 1 to 9.

11. A training apparatus for a translation model, the training apparatus comprising:

the data transmission module is used for acquiring a target translation memory statement;

the translation model training module is used for acquiring a source end sentence corresponding to the target translation memory sentence from a translation memory library;

the translation model training module is used for forming training samples by the source-end sentences and the target translation memory sentences and forming training sample sets by different training samples;

the translation model training module is used for acquiring initial parameters of a translation model;

the translation model training module is used for responding to initial parameters of the translation model, processing different training samples in the training sample set through the translation model, and determining updating parameters of the translation model;

and the translation model training module is used for carrying out iterative updating on the encoder parameter and the decoder parameter of the translation model through different training samples in the training sample set according to the updating parameter of the translation model.

12. A sentence translation apparatus, characterized in that the apparatus comprises:

the encoder module is used for determining at least one word-level hidden variable corresponding to the sentence to be translated through an encoder of the translation model;

a decoder module, configured to generate, by a decoder of the translation model, a translated term corresponding to the hidden variable at the term level and a selected probability of the translated term according to the hidden variable at the term level;

the decoder module is used for selecting at least one translation word to form a translation result corresponding to the sentence to be translated according to the selection probability of the translation result;

and the decoder module is used for outputting the translation result.

13. An electronic device, characterized in that the electronic device comprises:

a memory for storing executable instructions;

a processor for implementing the method of training a translation model according to any one of claims 1 to 9 or implementing the method of sentence translation according to claim 10 when executing the executable instructions stored in the memory.

14. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the method of training a translation model according to any one of claims 1 to 9 or implement the method of sentence translation according to claim 10.

15. A computer readable storage medium storing executable instructions which, when executed by a processor, implement the method for training a translation model according to any one of claims 1 to 9, or implement the method for sentence translation according to claim 10.

Images