CN117056720A - Learning fine tuning method for pre-training language, computer device and computer readable storage medium - Google Patents

Abstract

The present invention provides a learning fine-tuning method for pre-training language, a computer device and a computer-readable storage medium. The method includes acquiring a pre-training model and producing a vertical data set in a vertical field. The vertical data set includes multiple groups of logical relationships. a training statement, after expanding and shuffling the first training statement, the pre-training model is trained to obtain the initial training model; and the first training statement is used to test the initial training model, and the first training model is tested according to the initial training model. The response accuracy of the training statement determines whether the initial training model is under-fitting; and determines whether the initial training model is over-fitting, and the expansion multiple of the first training statement is calculated based on the under-fitting or over-fitting situation. Adjustment. The invention also provides a computer device and a computer-readable storage medium for implementing the above method. The present invention can avoid over-fitting or under-fitting of the training model.

Description

Pre-trained language learning fine-tuning method, computer device and computer-readable storage medium

Technical field

The present invention relates to the technical field of large-scale language training. Specifically, it is a reinforcement learning fine-tuning method for large-scale pre-trained language models. It also relates to a computer device and a computer-readable storage medium that implement the method.

Background technique

Large-scale pre-training language model is an artificial intelligence technology that has made breakthrough progress in recent years. The most famous model is Open AI's GPT. This model uses unlabeled large-scale text data for pre-training to learn rich languages. Representations can then be adapted to domain-specific tasks or problems by fine-tuning these pre-trained models.

In the field of reinforcement learning, researchers have proposed various methods to combine pre-trained language models with reinforcement learning. For example, a commonly used method is to use a reinforcement learning algorithm based on reward signals to fine-tune a pre-trained model through interaction with the environment. This method allows the model to self-learn and optimize in a specific task.

Regarding domain-specific fine-tuning methods, some existing literature proposes different techniques and strategies. For example, Smith et al. in their 2019 paper "Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems" introduced an approach to offline reinforcement learning that can be fine-tuned without real-time interaction. Additionally, Li et al. introduced a self-supervised fine-tuning method in their 2020 paper "Train Your Own Model (TYOM): ASelf-Supervised Model for Speech Recognition" that uses automatically generated labels to fine-tune pre-training model to fit a specific speech recognition task.

Although large-scale pre-trained language models have achieved great success in the field of natural language processing, they also have some problems and shortcomings. First, these models often require extensive computing resources and time to pre-train. Second, pre-trained general representations may not directly adapt to domain-specific details and features. In addition, the fine-tuning process may face the challenge of insufficient domain-specific data, especially for some specific vertical fields or tasks. These problems and difficulties limit the effectiveness and performance of large-scale pre-trained language models in specific domain applications.

All in all, the core capabilities of current large-scale pre-trained language models come from a large amount of training data and huge network model parameters. However, this large-scale effect not only brings intelligence, but also makes the model difficult to be deployed locally and difficult to modify. In this context, fine-tuning training is widely used to enhance the knowledge of large-scale pre-trained models. However, there is currently no very effective and specific way to fine-tune training.

The invention patent application with the publication number CN115423118A discloses a pre-trained language model fine-tuning method. This method formulates text prompt templates through the initial pre-trained language model for each type of task; organizes the training data into batches of data and merges them; Shuffle the order of the merged batch data; fine-tune the parameters of the pre-trained language model through multi-task learning. However, this method does not take into account the handling of over-fitting or under-fitting in the model, and in particular does not conduct effective tests for over-fitting, resulting in over-fitting in the model obtained by training, affecting the quality of the trained model. .

Contents of the invention

The first object of the present invention is to provide a learning fine-tuning method for pre-training language that can avoid over-fitting or under-fitting of the model obtained through training.

The second object of the present invention is to provide a computer device that implements the above-mentioned fine-tuning method for learning a pre-trained language.

The third object of the present invention is to provide a computer-readable storage medium that implements the above-mentioned fine-tuning method of learning a pre-trained language.

In order to achieve the first object of the present invention, the pre-trained language learning fine-tuning method provided by the present invention includes obtaining a pre-training model and creating a vertical data set in a vertical field. The vertical data set includes multiple sets of first training statements with logical relationships, After the first training statement is expanded and shuffled, the pre-training model is trained to obtain an initial training model; and the first training statement is used to test the initial training model, and the response to the first training statement is based on the initial training model The accuracy rate determines whether the initial training model is under-fitting; if the initial training model is under-fitting, increase the expansion multiple of the first training statement in the vertical data set and retrain the pre-training model; establish a common sense data set , the common sense data set contains multiple sets of second training statements. The second training statements are incremented into the vertical data set to form an incremental vertical data set. The incremental vertical data set is used to train the initial training model to obtain the incremental training model. Determine whether the initial training model is overfitting based on the accuracy of the incremental training model's response to the training sentences in the incremental vertical data set; if the initial training model is overfitting, reduce the number of first training steps in the vertical data set. The expansion multiple of the statement is used to retrain the pre-trained model.

It can be seen from the above scheme that after training to obtain the initial training model, it is necessary to test whether the initial training model has overfitting and underfitting, and dynamically adjust the expansion multiple according to the overfitting and underfitting conditions. In this way, more training models can be obtained and the quality of the models obtained through training can be improved.

In addition, for the over-fitting test, a common sense data set is set up. By adding the data set after the second training statement to test the incremental training model, it can effectively determine whether the initial training model is over-fitting, so as to accurately Make adjustments to the initially trained model.

A preferred solution is to use a front-end interface to input the first training sentence when creating a vertical data set in a vertical field. The front-end interface has a prompt input box, a question input box, and an answer input box.

It can be seen that during training, users can quickly input the first training statement through the front-end interface. The operation is simple and can improve the training efficiency of the model.

A further solution is that the front-end interface is also equipped with data import buttons, data expansion buttons, and data shuffle buttons.

In this way, users can use the data import button, data expansion button, and data shuffle button to automatically expand and shuffle the first training statement to improve testing efficiency.

A further solution is to use the front-end interface to input the second training sentence when creating a knowledge data set.

It can be seen that using the same front-end interface, you can also enter a second training sentence, making the production of common sense data sets very simple and convenient.

A preferred solution is that the front-end interface is also provided with an under-fitting detection button and an over-fitting detection button; the method also includes: when obtaining a signal that the under-fitting detection button is pressed, performing an under-fitting detection operation, and When the signal that the over-fitting detection button is pressed is obtained, the over-fitting detection operation is performed.

It can be seen that the over-fitting test and under-fitting test are performed through the corresponding operation buttons on the front-end interface. The over-fitting test and under-fitting test can be realized through the front-end interface, and the initial training model is modified. The test operation is very convenient.

A preferred solution is that the front-end interface is also provided with a model export button; the method further includes: when obtaining a signal that the model export button is pressed, exporting the fine-tuned training model.

In this way, when it is determined that the training model meets the requirements, that is, there is no overfitting or underfitting, the model obtained by training can be quickly exported through the model export button.

A further solution is that when the expansion multiple of the first training statement is increased or decreased, the expansion multiple after the increase or decrease is an integer multiple.

In this way, the first training statement and the second training statement used for training are expanded by an integer multiple, and the training effect is better.

A further solution is that the first training sentence is in a question-and-answer format; and/or the second training sentence is in a question-and-answer format.

In order to achieve the above second object, the computer device provided by the present invention includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, each step of the above-mentioned pre-training language learning fine-tuning method is implemented.

In order to achieve the above third object, the present invention provides a computer program stored on a computer-readable storage medium. When the computer program is executed by a processor, each step of the above-mentioned fine-tuning method for learning a pre-trained language is implemented.

Description of the drawings

Figure 1 is the first part of the flow chart of an embodiment of the pre-training language learning fine-tuning method of the present invention.

Figure 2 is the second part of the flow chart of an embodiment of the pre-training language learning fine-tuning method of the present invention.

Figure 3 is a schematic diagram of the front-end interface in the embodiment of the pre-training language learning fine-tuning method of the present invention.

The present invention will be further described below in conjunction with the accompanying drawings and examples.

Detailed ways

The pre-training language learning fine-tuning method of the present invention is mainly applied to the training of large-scale pre-trained language models, especially for the training of large-scale pre-trained language models in vertical fields. The computer device provided by the present invention has a processor and a memory. The processor can execute a computer program and implement the above-mentioned fine-tuning method for learning a pre-training language.

Example of learning fine-tuning method for pre-trained language:

Referring to Figures 1 and 2, this embodiment first performs step S1 to obtain a pre-trained model that needs to be trained in a vertical domain, and download the pre-trained model to a local computer, for example, store the pre-trained model on the local computer. on the memory. In addition, the designed fine-tuning algorithm model also needs to be deployed on the local computer, and various dependency packages must be installed on the local computer. Then, by implementing the Low-Rank Adaptation (lora) function on the local computer, a new additional model can be formed while ensuring that the pre-trained original model remains unchanged. This model contains vertical domain knowledge and is in the form of parameters. Save on local computer.

Then, step S2 is performed to create data in vertical fields that require training to form a vertical data set. When making a vertical data set, it is necessary to prepare a variety of knowledge in the vertical field, and the accuracy of the training data needs to be ensured. After determining the data for training, the data is made into logically related statements, for example, in the form of questions and answers for each item of training data. Furthermore, the description related to the questions and answers of each item of training data can also be set in the form of prompt words or instructions in the front part of the question and answer pair, and placed in a json file composed of a predetermined format in order.

Preferably, the process of organizing data information into standard json files is directly completed in this embodiment by designing a standard front-end interface and back-end program. The user only needs to enter prompt words, questions and standard answers at the corresponding positions of the front-end interface, and click to form json file is enough.

Referring to Figure 3, the front-end interface is provided with multiple buttons and three input boxes, namely prompt input box 11, question input box 12 and answer input box 13. The input box is displayed above each input box. name so that users can correctly identify each input box and enter correct information in each input box.

In addition, the front-end interface is also equipped with six buttons, namely data import button 21, data expansion button 22, data shuffle button 23, underfitting detection button 24, overfitting detection button 25 and model output button 26. Among them, the data import button 21 is used to import model data, for example, import a pre-trained model into the test system. The data expansion button 22 is used to expand the first training statement and the second training statement. Preferably, the expansion multiple is set by the user, such as 5 times or 6 times expansion. Preferably, the expansion multiple is an integer, that is, the first training statement and the second training statement are expanded by an integer multiple. The data shuffle button 23 is used to shuffle the expanded data, that is, randomly sort the expanded data, thereby realizing the expansion of the data set.

The underfitting detection button 24 is used to issue an instruction for underfitting detection. When the user clicks this button, the system will automatically test whether there is underfitting in the initial training model. The overfitting detection button 25 is used to issue an instruction for overfitting detection. When the user clicks this button, the system will automatically test whether the initial training model has overfitting. The model output button 26 is used to output the tested model, which means that the output model is saved in the memory of the local computer.

Referring to Figure 1, after acquiring multiple sets of first training statements with logical relationships from the vertical data set, step S3 is executed to expand and reorder the first training statements. Preferably, a default expansion multiple can be set. For example, the default expansion multiple is 5 times. In addition, in the initial state of the first training statement of the vertical data set, there are 1,000 question and answer pairs, then the amplitude is performed 5 times to form 5,000 question and answer pairs. Yes, but only 1,000 pieces of content are involved. Moreover, step S3 also performs shuffle processing on the expanded first training statement and puts it into the fine-tuning model for training.

Then, step S4 is executed, and the first training sentence after the expansion and out-of-order processing is used for training to obtain an initial training model. After obtaining the initial training model, you need to determine whether the initial training model is overfitting or underfitting after initial training, and fine-tune the initial training model based on the test results.

Then, step S5 is executed to test whether there is underfitting on the initial training model. Specifically, first randomly select a certain proportion of data from the vertical data set, for example, randomly select 10% of the data as underfitting test data, extract the problems and input them into the trained initial training model, and then obtain the initial training model. For the answers to the input test questions, compare the obtained answers with the standard answers in the first training sentence, and determine the similarity between the answers generated by the initial training model and the standard answers. If the similarity is higher than 90%, then It is considered that the initial training model can obtain the correct answer. In this way, the number of answers to 10% of the randomly selected questions of the initial training model that are more than 90% similar to the standard answer is calculated one by one. If the number of answers that are more than 90% similar to the standard answer exceeds the preset threshold , for example, if the threshold is 95%, it is considered that the initial training model does not have underfitting.

Therefore, in step S6, it is necessary to determine whether the initial training model is under-fitted. If the initial sequence model is not under-fitted, step S9 is executed. If the initial training model is under-fitted, it means The expansion multiple of the first training statement in step S3 is not enough, and step S7 needs to be executed to increase the expansion multiple of the first training statement. If the expansion multiple is 5 times in the initial state, then step S7 can set the expansion multiple to 6 times or 7 times, and execute step S8 to process the expanded first training statement out of order again, and process the predetermined training statements again. Train the model to train.

In step S9, it is necessary to test whether the initial training model has overfitting. In this embodiment, a common sense data set needs to be created. The common sense data set contains a plurality of second training sentences. The second training sentences are common sense sentences and are also presented in the form of questions and answers, such as questions of the second training sentences. It's "How big is China's land area" and so on. Preferably, the size of the second training statement is similar to the size of the first training statement. For example, the number of the first training statement is 1,000, and the number of the second training statement is also approximately 1,000, preferably between 1,100 of 900.

After making the common sense data set, add the second training statement of the common sense data set to the vertical data set to form an incremental vertical data set, and use the incremental vertical data set to train the initial training model to obtain an incremental training model. Then, a certain proportion of data is randomly obtained from the increased vertical data set as overfitting test data, and the similarity between the answer generated by the incremental training model and the standard answer is judged. If the similarity is greater than 90%, the incremental training model is considered The generated answer is correct. Finally, it is necessary to count the proportion of correct answers. If the proportion of correct answers is greater than the preset threshold, it is considered that the initial training model is not overfitting. Otherwise, the initial training model is considered to be overfitting.

In step S10, if the judgment result is yes, it means that the initial training model is under-fitting, then step S11 is executed to reduce the expansion multiple of the first training statement. For example, the expansion multiple of the first training statement in the initial state is 5 times, then step S11 will set the expansion multiple to 3 times or 4 times, and execute step S10 to shuffle the expanded first training statements again, and train the pre-training model again.

If the judgment result of step S10 is no, which means that the initial training model is neither overfitting nor underfitting, step S13 is executed to output the trained initial training model and use the initial training model as the final The trained model is saved in the local memory of the local computer.

The present invention proposes a method for under-fitting testing and over-fitting testing for the initial training model. Especially for the case of over-fitting testing, by adding common-sense second test statements to increment the vertical data set, it can effectively Test whether there is overfitting in the initial training model. In addition, for under-fitting and over-fitting situations, the initial training model is adjusted by increasing or decreasing the expansion multiple, so that the quality of the final training model is better.

In addition, the present invention also constructs a front-end interface through which the user can quickly input the first training statement and the second training statement, and can start processing such as expansion and reordering of the training statements in the data, and can also click The under-fitting detection button 24, the over-fitting detection button 25 and the model output button 26 implement under-fitting testing, over-fitting detection and the export of the training model, which are very convenient to operate.

Computer Device Examples:

The computer device in this embodiment can be an intelligent terminal device or a desktop computer. The computer device has a processor, a memory, and a computer program stored in the memory and capable of running on the processor, for example, used to implement the above information processing method. An information processing program, when the processor executes the computer program, implements each step of the above-mentioned pre-training language learning fine-tuning method.

For example, a computer program can be divided into one or more modules, and one or more modules are stored in a memory and executed by a processor to complete each module of the present invention. One or more modules may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the execution process of the computer program in the terminal device.

It should be noted that the terminal device can be a computing device such as a desktop computer, a notebook, a handheld computer, and a cloud server. Terminal devices may include, but are not limited to, processors and memories. Those skilled in the art can understand that the schematic diagram of the present invention is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. , for example, the terminal device may also include input and output devices, network access devices, buses, etc.

The processor referred to in the present invention can be a central processing unit (CPU), or other general-purpose processor, digital signal processor (Digital Signal Processor, DSP), or application specific integrated circuit (Application Specific Integrated Circuit, ASIC). , off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor can be a microprocessor or the processor can be any conventional processor, etc. The processor is the control center of the terminal device and uses various interfaces and lines to connect various parts of the entire terminal device.

The memory can be used to store computer programs and/or modules, and the processor implements various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory, and calling data stored in the memory. The memory may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, at least one application required for a function, etc.; the stored data area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card , a flash memory card (FlashCard), at least one disk storage device, flash memory device, or other volatile solid-state storage device.

Computer-readable storage media embodiments:

If the computer program stored in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program can be stored in a computer-readable storage medium. When executed by the processor, each step of the above-mentioned pre-trained language learning fine-tuning method can be realized.

Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. Computer-readable media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, mobile hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM, Read-Only Memory), random access Memory (RAM, Random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include Electrical carrier signals and telecommunications signals.

Finally, it should be emphasized that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications, as long as the spirit and principles of the present invention are maintained. Within the scope of this invention, any modifications, equivalent substitutions, improvements, etc. shall be included in the protection scope of the present invention.

Claims (10)

1. A method of learning fine tuning of a pre-training language, comprising:

obtaining a pre-training model, and manufacturing a vertical data set in the vertical field, wherein the vertical data set comprises a plurality of groups of first training sentences with logic relations, and training the pre-training model after the first training sentences are subjected to expansion and disorder treatment to obtain an initial training model;

the method is characterized in that:

testing the initial training model by applying the first training sentence, and judging whether the initial training model has a lack-fitting condition according to the response accuracy of the initial training model to the first training sentence; if the initial training model has the condition of under fitting, increasing expansion multiples of the first training sentences in the vertical data set, and retraining the pre-training model;

establishing a common sense data set, wherein the common sense data set comprises a plurality of groups of second training sentences, adding the second training sentences into the vertical data set to form an increment vertical data set, training an initial training model by using the increment vertical data set to obtain an increment training model, and judging whether the initial training model has fitting conditions according to the response accuracy of the increment training model to the training sentences in the increment vertical data set; and if the initial training model is fitted, reducing expansion times of the first training sentences in the vertical data set, and retraining the pre-training model.

2. The method for learning and fine tuning of a pre-training language according to claim 1, wherein:

when the vertical data set in the vertical field is manufactured, a front-end interface is applied to input the first training sentence, and the front-end interface is provided with a prompt input box, a question input box and an answer input box.

3. The method for learning and fine tuning of a pre-training language according to claim 2, wherein:

the front-end interface is also provided with a data import button, a data expansion button and a data disorder button.

4. The method for learning and fine tuning of a pre-training language according to claim 2, wherein:

and when the common sense data set is established, the front-end interface is applied to input the second training statement.

5. The method for learning and fine tuning of a pre-training language of claim 4, wherein:

the front end interface is also provided with an under fitting detection button and an over fitting detection button;

the method further comprises the steps of: and executing the under-fitting detection operation when the signal of the under-fitting detection button pressed is acquired, and executing the over-fitting detection operation when the signal of the over-fitting detection button pressed is acquired.

6. The method for learning and fine tuning of a pre-training language according to claim 2, wherein:

the front end interface is also provided with a model export button;

the method further comprises the steps of: and when the signal of the model export button pressed is acquired, exporting a training model after fine adjustment.

7. A method of fine tuning the learning of a pre-training language as claimed in any one of claims 1 to 6, wherein:

when the expansion multiple of the first training sentence is increased or reduced, the increased or reduced expansion multiple is an integer multiple.

8. A method of fine tuning the learning of a pre-training language as claimed in any one of claims 1 to 6, wherein:

the first training sentence is in a question-answering mode; and/or, the second training sentence is in a question-answering mode.

9. Computer device, characterized in that it comprises a processor and a memory, said memory storing a computer program which, when executed by the processor, implements the steps of the method for learning fine tuning of a pre-training language according to any one of claims 1 to 8.

10. A computer readable storage medium having stored thereon a computer program characterized by: the computer program, when executed by a processor, performs the steps of a method of learning fine tuning of a pre-training language according to any one of claims 1 to 8.