CN116756555A - A method for generating a training model tool, a model training method and a device - Google Patents

Abstract

This application discloses a method for generating a training model tool, a model training method and a device, and belongs to the field of computer technology. The method includes: obtaining a first function from a first function library, and obtaining a second function with the same function as the first function from a second function library, wherein the first function is run on a CPU function, the second function is a function running on the GPU; obtain the first parameter information of the first function and the second parameter information of the second function; according to the first parameter information and the second parameter information, Generate a parameter set; adjust the first function and/or the second function according to the parameter set; generate a basis function according to the adjusted first function and the adjusted second function; obtain a basis function library according to the basis function tool.

Description

A method for generating a training model tool, a model training method and a device

Technical field

The present application belongs to the field of computer technology, and specifically relates to a method for generating a training model tool, a model training method and a device.

Background technique

With the development of society, the amount of data is increasing at a rate of doubling every year, and the need to accelerate machine learning model training is becoming more and more important.

In related technologies, the central processing unit (CPU) of the training terminal is usually used to process the corresponding training data; the graphics processing unit (GPU) of the training terminal is used to process the corresponding training data, and the GPU has faster The data processing speed plays a role in accelerating the training process. The above-mentioned software toolkits required for CPU and GPU runtime can only process their corresponding training data types. Therefore, in the existing technology, to implement the same functional module, it is necessary to write corresponding module codes for training data respectively for the CPU or GPU. The code writing workload is large, resulting in high labor costs.

Contents of the invention

The purpose of the embodiments of the present application is to provide a method for generating a training model tool, a model training method and a device, which can solve the problems of heavy code writing workload and high labor cost in the existing model training methods.

In a first aspect, embodiments of the present application provide a method for generating a training model tool. The method includes:

Obtain a first function from the first function library, and obtain a second function with the same function as the first function from the second function library, wherein the first function is a function running on the CPU, and the The second function is a function that runs on the GPU;

Obtain the first parameter information of the first function and the second parameter information of the second function;

Generate a parameter set according to the first parameter information and the second parameter information;

adjusting the first function and/or the second function according to the parameter set;

Generate a basis function based on the adjusted first function and the adjusted second function;

Based on the basis functions, a basis function library tool is obtained.

Optionally, generating a parameter set according to the first parameter information and the second parameter information includes:

Compare the first parameter information and the second parameter information to obtain a comparison result;

A parameter set is generated according to the comparison result, the first parameter information and the second parameter information.

Optionally, the first parameter information and/or the second parameter information includes at least one of the following: function name, parameter number, and parameter type.

Optionally, generating a parameter set based on the comparison result, the first parameter information and the second parameter information includes:

If the comparison result is the same, use the first parameter information or the second parameter information as a parameter set;

If the comparison results are different, obtain the one containing a larger number of parameters from the first parameter information and the second parameter information as the reference parameter information;

According to the reference parameter information, the parameter set is obtained.

Optionally, adjusting the first function and the second function according to the parameter set includes:

Based on the parameter set, function parameter enhancement adjustment is performed on the first function or the second function that contains a smaller number of parameters.

Optionally, the method also includes:

When the first function library does not include the first function corresponding to the second function, compile and generate the first function corresponding to the second function according to the second parameter information of the second function; or,

When the second function library does not include a second function corresponding to the first function, a second function corresponding to the first function is compiled and generated according to the first parameter information of the first function.

In a second aspect, embodiments of the present application provide a model training method, which method includes:

In response to the model training task, obtain training data and configuration information of the model training workflow;

Call the basis function associated with the model training workflow according to the configuration information, wherein the basis function is configured as the basis function described in the first aspect;

Through the basis function and the training data, the model is iteratively trained.

Optionally, the iterative training of the model through the basis function and the training data includes:

Determine the target running resource to be called according to the data type of the training data, where the target running resource includes CPU resources or GPU resources;

Based on the base function, the target running resource is called to iteratively train the model.

In a third aspect, embodiments of the present application provide a tool generation device for training a model. The device includes:

The first acquisition module is used to acquire the first function from the first function library, and acquire the second function with the same function as the first function from the second function library, wherein the first function is configured on the CPU. A function that runs on the GPU, and the second function is a function that runs on the GPU;

The second acquisition module is used to acquire the first parameter information of the first function and the second parameter information of the second function;

A first generation module, configured to generate a parameter set according to the first parameter information and the second parameter information;

an adjustment module, configured to adjust the first function and/or the second function according to the parameter set;

The second generation module is used to generate the basis function according to the adjusted first function and the adjusted second function;

The third generation module is used to obtain a basis function library tool according to the basis function.

In a fourth aspect, embodiments of the present application provide a model training device, which includes:

The acquisition module is used to obtain training data and configuration information of the model training workflow in response to the model training task;

A calling module, configured to call a base function associated with the model training workflow according to the configuration information, wherein the base function is configured as the base function described in the first aspect;

A training module, configured to iteratively train the model through the basis function and the training data.

In a fifth aspect, embodiments of the present application provide an electronic device. The electronic device includes a processor and a memory. The memory stores programs or instructions that can be run on the processor. The program or instructions are processed by the processor. When executed by the processor, the steps of the method for generating a training model tool as described in the first aspect or the method for model training as described in the second aspect are implemented.

In a sixth aspect, embodiments of the present application provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the training model tool as described in the first aspect is implemented. The steps of the generation method or the model training method as described in the second aspect.

In this embodiment of the present application, by obtaining the first parameter information of the first function running on the CPU and the second parameter information of the second function running on the GPU, the first parameter information and the second parameter are used Information is generated to generate a parameter set, thereby using the parameter set to adjust the first function and the second function, and obtain the basis function. The solution of the present invention abstracts and obtains a basic function library tool by adjusting the functions running on the CPU and the functions running on the GPU. Then, when using the CPU or GPU for automatic model training, it uses the basic functions in the basic function library. By calling the first function or the second function, there is no need to write module codes for model training separately for the CPU and GPU, which reduces repeated workload and cost, avoids the problem of inconvenient conversion, and improves efficiency.

Description of the drawings

Figure 1 is a schematic flowchart of a tool generation method for training a model provided by an embodiment of the present application;

Figure 2 is a schematic flow chart of the model training method provided by the embodiment of the present application;

Figure 3 is a schematic diagram of the application of a training model tool;

Figure 4 is a schematic structural diagram of a tool generation device for training models provided by an embodiment of the present application.

Figure 5 is a schematic structural diagram of a model training device provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art fall within the scope of protection of this application.

The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first," "second," etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the related objects are in an "or" relationship.

The tool generation method for training models provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios.

As shown in Figure 1, the model training method provided by the embodiment of this application includes the following steps:

Step S11: Obtain a first function from the first function library, and obtain a second function with the same function as the first function from the second function library, where the first function is a function that runs on the CPU. , the second function is a function that runs on the GPU.

During the model training process, an automatic machine learning framework can be built on the CPU or GPU through algorithms to automatically handle a series of model training processes such as data cleaning, feature engineering, hyperparameter optimization, model selection, model evaluation, and model fusion. The tool libraries and functions called when performing model training on the CPU are different from the tool libraries and functions called when performing model training on the GPU. The CPU tool library mainly includes Pandas, Numpy and Scikit-learn. The GPU tool library mainly includes cuDF (data processing library), cuPY (matrix mathematics library), and cuML (machine learning library). The GPU tool library has an Application Program Interface (API) interface similar to the CPU tool library.

Select the first function that needs to be called during the model training process from the CPU tool library, and the first function is included in the first function library. Select the second function that needs to be called during the model training process from the GPU tool library, and the second function is included in the second function library. The first function and the second function with the same function function correspond to each other.

Step S12: Obtain the first parameter information of the first function and the second parameter information of the second function.

The first parameter information includes the function interface and function functions of the first function. Similarly, the second parameter information includes the function interface and function functions of the second function.

Step S13: Generate a parameter set based on the first parameter information and the second parameter information.

The parameter set contains the first parameter information corresponding to all the first functions in the first function library, that is, the function interface and function function corresponding to the first function. Similarly, the parameter set also contains the first parameter information corresponding to all the second functions in the second function library. Two function information.

For a pair of first function and second function with the same function, compare the corresponding first parameter information and second parameter information. When the first parameter information and the second parameter information are the same, the first parameter information and the second parameter information do not need to be Revise. When the first parameter information and the second parameter information are different, the first parameter information or the second parameter information is adjusted so that the first parameter information and the second parameter information are the same.

Step S14: Adjust the first function and/or the second function according to the parameter set.

It can be understood that when the target first parameter information or the target second parameter information in the parameter set is adjusted, the target first function or the target second function in the first function library needs to be adjusted accordingly, so that the adjusted target first function The first parameter information of the function is consistent with the target first parameter information or the second parameter information of the target second function is consistent with the target second parameter information.

Step S15: Generate a basis function based on the adjusted first function and the adjusted second function.

After adjustment, the first parameter information and second parameter information of each pair of the first function and the second function with the same function function are also the same, that is, the function interface is also the same. For each pair of the first function with the same function function and the same function interface Abstract with the second function to form a corresponding base function, which is used to call the corresponding first function or second function.

Step S16: Obtain a basis function library tool based on the basis function.

The base function library tool includes a plurality of base functions, and the base functions are used to call the corresponding first function or the corresponding second function. Therefore, through the base function library tool, the first function can be called on the first function library or on the first function library. The second function is called in the second function library.

The following example illustrates the process of generating a basis function. To implement the addition function, the CPU tool Pandas and the GPU tool library cuDF both use the add() function. The corresponding first function is pandas.DataFrame.add( other, axis='columns', level=None, fill_value=None).

The corresponding second function is cudf.DataFrame.add(other, axis='columns', level=None, fill_value=None).

Abstract the first and second function codes that implement addition to obtain the base function of a unified interface:

basetoolbox.DataFrame.add(other, axis='columns', level=None, fill_value=None).

It should be noted that each basis function includes corresponding function functions.

Through the above steps, the first parameter information of the first function running on the CPU and the second parameter information of the second function running on the GPU are obtained, and parameters are generated using the first parameter information and the second parameter information. set, thereby using the parameter set to adjust the first function and the second function, and obtain the basis function. The solution of the present invention abstracts and obtains a basic function library tool by adjusting the functions running on the CPU and the functions running on the GPU, and then uses the basic functions in the basic function library tool when using the CPU or GPU for automatic model training. By calling the first function or the second function, there is no need to write module codes for model training separately for the CPU and GPU, which reduces repeated workload and cost, avoids the problem of inconvenient conversion, and improves efficiency.

Optionally, generating a parameter set based on the first parameter information and the second parameter information includes:

Compare the first parameter information and the second parameter information to obtain a comparison result;

A parameter set is generated according to the comparison result, the first parameter information and the second parameter information.

Compare the first parameter information and the second parameter information of a pair of the first function and the second function with the same function to obtain the comparison result, and determine whether the first parameter information or the second parameter information needs to be modified based on the comparison result, In order to make the first parameter information and the second parameter information of each pair of the first function and the second function with the same function consistent, thereby ensuring that the base function library tool can be accurately called in the first function library and the second function library through the base function The first function and the second function have the same function.

Optionally, the first parameter information and/or the second parameter information includes at least one of the following: function name, parameter number, and parameter type.

The function name corresponds to the function's function, making it easy to call the base function. The number of parameters indicates the number of parameters that the function can handle, and the parameter type indicates the data type of the parameters that the function can handle. Compare the first parameter information and the second parameter information by comprehensively considering the above-mentioned function name, number of parameters, and parameter types, and adjust the first parameter information and/or the second parameter information according to the comparison results, so as to adjust the third parameter information in the subsequent process. The first function and/or the second function further ensures that the base function library tool can accurately call the first function and the second function with the same function in the first function library and the second function library through the base function.

Optionally, generating a parameter set based on the comparison result, the first parameter information and the second parameter information includes:

When the comparison result is the same, the first parameter information or the second parameter information is used as a parameter set,

When the comparison result is the same, that is, when a pair of the first function and the second function with the same function function also have the same function interface, that is, the number of parameters that the function can handle, the data type of the parameters that can be processed, and the name of the function are all the same. At the same time, there is no need to adjust the first parameter information and the second parameter information, and there is no need to adjust the first function corresponding to the first parameter information and the second function corresponding to the second parameter information, and the corresponding basis function is directly generated. In the subsequent model training process, you only need to call the corresponding first function in the first function library or the corresponding second function in the second function library through the basic function in the basic function library tool to complete the data processing. There is no need to target the CPU and GPU. to rewrite the training module code.

When the comparison results are different, the one containing a larger number of parameters is obtained from the first parameter information and the second parameter information as the reference parameter information, and the parameter set is obtained based on the reference parameter information. .

For a pair of first function and second function with the same function, the comparison results are different, that is, the first parameter information and the second parameter information of the pair of functions are different, that is, when the function interfaces are different, the first parameter information or the second parameter information is different. Parameter information is adjusted to generate parameter sets. Specifically, the different function interfaces may be due to the different number of parameters processed by the function, different function data types, or different function names. If the function interface parameters of the first parameter information are less than the function interface parameters of the second parameter information, the second parameter information is the base parameter information, and the function interface of the first parameter information is added according to the base parameter information; if it is the function interface of the second parameter information For function interface parameters whose parameters are less than the first parameter information, the first parameter information is the reference parameter information, and the second parameter information is added according to the reference parameter information. The party containing a smaller number of parameters is supplemented according to the benchmark parameter information, so as to facilitate subsequent adjustments to the function corresponding to the party containing a smaller number of parameters based on the benchmark parameter information.

Optionally, adjusting the first function and the second function according to the parameter set includes:

Based on the parameter set, function parameter enhancement adjustment is performed on the first function or the second function that contains a smaller number of parameters.

For example, the first function is a function in Scikit-learn. According to the first parameter information corresponding to the first function, the first function can support data types such as integers, floating point numbers, and text, while the second function is a function in cuML. function, according to the second parameter information corresponding to the second function, the second function only supports integers and floating point numbers. In this case, the first parameter information of the first function is the reference parameter information. According to the reference parameter information, the function interface parameter in the second parameter information of the second function needs to be added, that is, the function interface parameter in the second parameter information needs to be added. Enhancements to parameter encoding for tags. When text features appear, the text is classified into categories, and then label encoding is performed to replace the text with numeric labels. The function functions and function interfaces of the enhanced first parameter information and the second parameter information are consistent, which facilitates abstract processing of the first function and the second function, and obtains corresponding base functions to generate base function library tools.

Optionally, the tool generation method for training the model provided by the embodiment of this application also includes:

When the first function library does not include the first function corresponding to the second function, compile and generate the first function corresponding to the second function according to the second parameter information of the second function; or,

When the second function library does not include a second function corresponding to the first function, a second function corresponding to the first function is compiled and generated according to the first parameter information of the first function.

It should be noted that since the software ecosystem that supports CPU operation is more mature than the software ecosystem that supports GPU operation, generally speaking, the first function in the first function library can implement more functions than the second function in the second function library. function, therefore, in most cases, each second function in the second function library can find the corresponding first function in the first function library, but the first function may not necessarily be able to find the corresponding first function in the second function library. The second function therefore needs to be compiled to generate the corresponding second function based on the first parameter information of the first function.

The compilation process is explained through the following example. Some functions of cuML only support processing ndarray data, but do not support DataFrame, the mainstream data format of machine learning algorithms. When the target first function is used to perform a specific function on data in DataFrame format When processing, cuML only supports processing ndarray data and there is no second function corresponding to the first function of the target. Therefore, you can only refer to the algorithm logic corresponding to the first parameter information of the first function of the target in the GPU tool library. Implement it again in the form of DataFrame, generate the second target function to correspond to the first target function, and then generate the target base function based on the first target function and the second target function, ensuring that every step in the model training process can This is achieved by calling base functions.

The embodiment of the present application also provides a model training method. As shown in Figure 2, the model training method includes:

Step S21: In response to the model training task, obtain training data and configuration information of the model training workflow.

The execution subject of the model training method may be a model training application, and the model training application may be loaded on a mobile phone, computer, or other devices. Based on the model training task, the execution subject of the model training method responds and obtains training data and configuration information of the model training workflow. The general requirements for training data are: the data sample should be as large as possible, the data should be diverse, and the data sample should be of high quality. Configuration information includes the operating relationship between operator modules and operator models. Operator models include: data cleaning, missing value processing, normalization processing, feature engineering, hyperparameter optimization, model selection, model evaluation, and model fusion. Input training data and configuration information into the model training application to perform full-process automatic machine learning.

Step S22: Call the base function associated with the model training workflow according to the configuration information, wherein the base function is configured as the base function described in the production method of the above training model tool,

Determine the base function associated with the model training workflow according to the configuration information. If the training data needs to be cleaned, the base function with the data cleaning function is called, and then the base function is used in the first function library and the second function library. Call the first function and the second function of the corresponding function.

Step S23: Iteratively train the model through the basis function and the training data.

According to the corresponding relationship between the base function and the first function and the second function with the same function, the corresponding first function can be called in the first function library or the corresponding second function can be called in the second function library to process the training data. , for model training. Therefore, when processing training data, you can choose either CPU or GPU. The CPU has a small number of computing units with powerful computing capabilities, and is suitable for a small amount of calculations with high complexity. The GPU has hundreds or thousands of cores and can perform a large number of parallel calculations. The calculation speed is much greater than that of the CPU, but the complexity of the computational logic it can handle is low. When processing training data, according to the algorithm logic and data volume required for the training step, it is reasonable to choose to call the first function for processing on the CPU or the second function for processing on the GPU. Through the invocation of the above-mentioned base function and the invocation of the first function or the second function corresponding to the base function, both the CPU and the GPU can complete the full-process automatic machine learning training task. In addition, due to the characteristics of GPU, usually the speed performance of GPU is better than that of CPU, and the larger the amount of training data, the more obvious the acceleration advantage of GPU.

Optionally, iteratively training the model through the basis function and the training data includes:

Determine the target running resource to be called according to the data type of the training data, where the target running resource includes CPU resources or GPU resources;

Based on the base function, the target running resource is called to iteratively train the model.

Understandably, when the data type of the training data is a type more suitable for CPU processing, the first function is called through the base function to perform training on the CPU; when the data type of the training data is a type more suitable for GPU processing, the first function is called through the base function. The function calls the second function to perform training on the GPU.

As shown in Figure 3, it is a schematic diagram of the application of a training model tool. Input the training data set and train data to complete the training of the automatic machine learning model. The model training application includes a base function library determined by the adjusted first function and the adjusted second function, that is, the abstraction layer shown in Figure 3. The above abstraction layer is also called the base function library tool (BaseToolbox). BaseToolbox identifies the data type of the training set data. The application also includes a toolkit, which includes a CPU function library cpuToolbox, which is the first function library, and a GPU function library gpuToolbox, which is the second function library. The basis function library tool determines the basis function of the corresponding function based on the training set data and the configuration information of the model training workflow, and then selects to call the first function or GPU function in the CPU function library based on the identified data type of the training set data. Second function in the library. For example, when the format of the training set data is cuDFDaraFrame supported by the GPU, the second function is called in the GPU function library and the automatic machine learning model training process is executed in the GPU.

Both CPU and GPU can be used for model training, which is more flexible. The use of CPU or GPU is determined according to the data type of the training data, further ensuring the stability of the model training process.

For the training model tool generation method provided by the embodiment of the present application, the execution subject may be a training model tool generation device.

In the embodiment of the present application, a method for generating a training model tool executed by a training model tool generating device is taken as an example, and the training model tool generating device 400 provided by the present application is explained with reference to FIG. 4 . The generating device 400 of the training model tool includes:

The first acquisition module 401 is used to acquire the first function from the first function library, and acquire the second function with the same function as the first function from the second function library, wherein the first function is A function that runs on the CPU, and the second function is a function that runs on the GPU;

The second acquisition module 402 is used to acquire the first parameter information of the first function and the second parameter information of the second function;

The first generation module 403 is used to generate a parameter set according to the first parameter information and the second parameter information;

Adjustment module 404, configured to adjust the first function and/or the second function according to the parameter set;

The second generation module 405 is used to generate a basis function according to the adjusted first function and the adjusted second function;

The third generation module 406 is used to obtain a basis function library tool according to the basis function.

Optionally, the first generation module 403 is also used to:

Compare the first parameter information and the second parameter information to obtain a comparison result;

A parameter set is generated according to the comparison result, the first parameter information and the second parameter information.

Optionally, the first parameter information and/or the second parameter information includes at least one of the following: function name, parameter number, and parameter type.

Optionally, the first generation module 403 is also used to:

If the comparison result is the same, use the first parameter information or the second parameter information as a parameter set;

If the comparison results are different, obtain the one containing a larger number of parameters from the first parameter information and the second parameter information as the reference parameter information;

According to the reference parameter information, the parameter set is obtained.

Optionally, the adjustment module 404 is also used to:

Based on the parameter set, function parameter enhancement adjustment is performed on the first function or the second function that contains a smaller number of parameters.

Optionally, the tool generation device 400 for training the model also includes a compilation module, and the compilation module is used for:

When the first function library does not include the first function corresponding to the second function, compile and generate the first function corresponding to the second function according to the second parameter information of the second function; or,

When the second function library does not include a second function corresponding to the first function, a second function corresponding to the first function is compiled and generated according to the first parameter information of the first function.

The above-mentioned tool generation device for training a model can generate a corresponding base function library tool based on the first function in the first function library and the second function in the second function library, and then call the corresponding base function library tool through the base function in the base function library tool. first function or second function.

It should be noted that the tool generation device for training a model provided by the embodiment of the present application can implement all the technical processes of the tool generation method for training a model and can achieve the same technical effect. To avoid duplication, here No longer.

For the model training method provided by the embodiments of the present application, the execution subject may be a model training device. In the embodiment of the present application, a model training device executing a model training method is used as an example, and the model training device 500 provided by the present application is described with reference to FIG. 5 . The model training device includes:

The acquisition module 501 is used to obtain training data and configuration information of the model training workflow in response to the model training task;

The calling module 502 is configured to call the base function associated with the model training workflow according to the configuration information, wherein the base function is configured as the base function described in the method for generating the training model tool;

The training module 503 is used to iteratively train the model through the basis function and the training data.

Optionally, the training module 503 is also used to:

Determine the target running resource to be called according to the data type of the training data, where the target running resource includes CPU resources or GPU resources;

Based on the base function, the target running resource is called to iteratively train the model.

The above-mentioned model training device 500 can call the corresponding first function or second function on the CPU or GPU according to the basis function and training data to perform fully automatic machine learning training.

It should be noted that the model training device provided by the embodiment of the present application can implement all the technical processes of the above model training method and can achieve the same technical effect. To avoid duplication, the details will not be described here.

The device in the embodiment of the present application may be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than the terminal. For example, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle-mounted electronic device, a mobile Internet device (MID), or augmented reality (AR)/virtual reality (VR). Equipment, robots, wearable devices, ultra-mobile personal computers (UMPC), netbooks or personal digital assistants (PDA), etc. Non-mobile electronic devices can also be servers, network-attached storage (Network Attached Storage (NAS), personal computer (PC), television (TV), teller machine or self-service machine, etc., are not specifically limited in the embodiments of this application.

Optionally, as shown in Figure 5, this embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores programs or instructions that can be run on the processor 601. The program or When the instructions are executed by the processor 601, each step of the above training model tool generation method or model training method embodiment is implemented, and the same technical effect can be achieved. To avoid duplication, the details will not be described here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

Embodiments of the present application also provide a readable storage medium, which stores a program or instructions. When the program or instructions are executed by a processor, the method for generating a training model tool or the method for training a model can be implemented. Each process can achieve the same technical effect. To avoid duplication, it will not be described again here.

Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage media, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or apparatus that includes that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions may be performed, for example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk , CD), including several instructions to cause a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of this application.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, and all of them should be covered. within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (10)

1. A method for generating a training model tool, which is characterized by including: Obtain a first function from the first function library, and obtain a second function with the same function as the first function from the second function library, wherein the first function is a function running on the CPU, and the The second function is a function that runs on the GPU; Obtain the first parameter information of the first function and the second parameter information of the second function; Generate a parameter set according to the first parameter information and the second parameter information; adjusting the first function and/or the second function according to the parameter set; Generate a basis function based on the adjusted first function and the adjusted second function; Based on the basis functions, a basis function library tool is obtained. 2. The method according to claim 1, wherein generating a parameter set according to the first parameter information and the second parameter information includes: Compare the first parameter information and the second parameter information to obtain a comparison result; A parameter set is generated according to the comparison result, the first parameter information and the second parameter information. 3. The method according to claim 2, wherein the first parameter information and/or the second parameter information includes at least one of the following: function name, parameter number, and parameter type. 4. The method according to claim 3, wherein generating a parameter set according to the comparison result, the first parameter information and the second parameter information includes: If the comparison result is the same, use the first parameter information or the second parameter information as a parameter set; If the comparison results are different, obtain the one containing a larger number of parameters from the first parameter information and the second parameter information as the reference parameter information; According to the reference parameter information, the parameter set is obtained. 5. The method of claim 4, wherein adjusting the first function and the second function according to the parameter set includes: Based on the parameter set, function parameter enhancement adjustment is performed on the first function or the second function that contains a smaller number of parameters. 6. The method according to claim 1, characterized in that, the method further comprises: When the first function library does not include the first function corresponding to the second function, compile and generate the first function corresponding to the second function according to the second parameter information of the second function; or, When the second function library does not include a second function corresponding to the first function, a second function corresponding to the first function is compiled and generated according to the first parameter information of the first function. 7. A model training method, characterized by comprising: In response to the model training task, obtain training data and configuration information of the model training workflow; Call the basis function associated with the model training workflow according to the configuration information, wherein the basis function is configured as the basis function according to any one of claims 1 to 6; Through the basis function and the training data, the model is iteratively trained. 8. The method according to claim 7, wherein the iterative training of the model through the basis function and the training data includes: Determine the target running resource to be called according to the data type of the training data, where the target running resource includes CPU resources or GPU resources; Based on the base function, the target running resource is called to iteratively train the model. 9. A device for generating a training model tool, characterized in that the device includes: The first acquisition module is used to acquire the first function from the first function library, and acquire the second function with the same function as the first function from the second function library, wherein the first function is configured on the CPU. A function that runs on the GPU, and the second function is a function that runs on the GPU; The second acquisition module is used to acquire the first parameter information of the first function and the second parameter information of the second function; A first generation module, configured to generate a parameter set according to the first parameter information and the second parameter information; an adjustment module, configured to adjust the first function and/or the second function according to the parameter set; The second generation module is used to generate the basis function according to the adjusted first function and the adjusted second function; The third generation module is used to obtain a basis function library tool according to the basis function. 10. A model training device, characterized in that the device includes: The acquisition module is used to obtain training data and configuration information of the model training workflow in response to the model training task; A calling module, configured to call a base function associated with the model training workflow according to the configuration information, wherein the base function is configured as the base function according to any one of claims 1 to 6; A training module, configured to iteratively train the model through the basis function and the training data.