CN114861870A - A method, apparatus and device for configuring a neural network architecture - Google Patents

Abstract

The invention discloses a method, device and device for configuring a neural network architecture, wherein the method comprises: accessing a decision problem of a neural network to be trained; obtaining a first environment of the decision problem according to the decision problem; Encapsulate the decision problem with the first environment to obtain an encapsulated second environment; access the neural network to be trained; access the architecture algorithm according to the second environment and the neural network to be trained; The second environment, the neural network to be trained and the architecture algorithm are adapted to generate trajectory data attributes; the neural network to be trained is optimized according to the trajectory data attributes to obtain the configuration architecture to be trained Neural Networks. In the above manner, the present invention improves the versatility and expansibility of the neural network architecture configuration.

Description

A method, apparatus and device for configuring a neural network architecture

technical field

The present invention relates to the technical field of machine learning, and in particular, to a method, apparatus and device for configuring a neural network architecture.

Background technique

Deep reinforcement learning is a combination of deep learning and reinforcement learning with decision-making ability to form an optimal decision-making method that can directly process high-dimensional complex information as input. Deep reinforcement learning can not only bring the convenience of end-to-end automatic optimization for reinforcement learning, but also make reinforcement learning no longer limited to low-dimensional space, and can solve more complex decision-making problems, which greatly expands the application scope of reinforcement learning. .

Deep reinforcement learning has been increasingly applied to many intelligent decision-making fields in recent years, but at present, deep reinforcement learning is still in the stage of rapid development, and various types of algorithms have emerged, forming a variety of problem environments. At the algorithm level, it is difficult to unify the forms and implementation details of various algorithms. The deep reinforcement learning algorithm is not suitable for deep reinforcement because its computing mode is not fixed and requires high concurrency, so mature training frameworks in the fields of computer vision, natural language processing, and speech processing are not suitable Learning field; at the environmental level, on the one hand, the problem environment that needs to be solved is diverse, so that there is no unified standard for the application of deep reinforcement learning to various intelligent decision-making problems, and it is often necessary to start from scratch, and the workload is huge; on the other hand , many problems only provide an adjudication engine, but do not provide a standardized process for how to transform the adjudication engine into the problem environment, so that there is no unified framework process for the implementation process from the adjudication engine to the environment as a reference.

In response to the above-mentioned problems at the two levels of algorithm and environment, some framework projects have tried to solve them, but the current frameworks are abstract and complex, and all have problems such as lack of versatility and scalability. Due to these shortcomings, the ease of use of each framework implementation is also difficult to guarantee.

SUMMARY OF THE INVENTION

In order to solve the above problems, a method, an apparatus, and a device for configuring a neural network architecture according to embodiments of the present invention are proposed.

According to an aspect of the embodiments of the present invention, a method for configuring a neural network architecture is provided, including:

Access the decision-making problem of the neural network to be trained;

According to the decision-making problem, obtain the first environment of the decision-making problem;

Encapsulating the decision problem and the first environment to obtain an encapsulated second environment;

access the neural network to be trained;

Access the architecture algorithm according to the second environment and the neural network to be trained;

Adapting the second environment, the neural network to be trained, and the architecture algorithm to generate trajectory data attributes;

The neural network to be trained is optimized according to the attributes of the trajectory data to obtain the neural network to be trained after the configuration architecture.

Optionally, according to the decision-making problem, the first environment of the decision-making problem is obtained, including:

If there is an environment in the decision-making problem, obtaining the first environment according to the environment in the decision-making problem;

If there is no environment in the decision-making problem, define a new environment for the decision-making problem, obtain a defined third environment, and then obtain the first environment according to the third environment.

Optionally, if there is an environment in the decision-making problem, the first environment is obtained according to the environment in the decision-making problem, including:

If the environment included in the decision-making problem satisfies the preset condition, determining the environment included in the decision-making problem as the first environment;

If the environment included in the decision-making problem does not satisfy the preset condition, the environment included in the decision-making problem is compatible with the general environment, and the compatible environment is determined as the first environment.

Optionally, obtaining the first environment according to the third environment, including:

The third environment is compatible with the general environment, and the compatible environment is determined as the first environment.

Optionally, after obtaining the encapsulated second environment, it further includes:

According to the second environment, the state space of the second environment is obtained, and the state space refers to the space of states generated when the neural network to be trained is running.

Optionally, after accessing the architecture algorithm, it further includes:

According to the architecture algorithm, the attributes of the pre-generated data of the neural network to be trained are determined.

Optionally, the second environment, the neural network to be trained, and the architecture algorithm are adapted to generate trajectory data attributes, including:

Adapting the second environment, the neural network to be trained and the architecture algorithm, and then according to the state space of the second environment after adaptation, the neural network to be trained after adaptation, and the adaptation The latter neural network to be trained pre-generates data to generate trajectory data attributes.

According to another aspect of the embodiments of the present invention, there is provided an apparatus for configuring a neural network architecture, the apparatus comprising:

The first access module is used to access the decision problem of the neural network to be trained;

a processing module, configured to obtain a first environment of the decision-making problem according to the decision-making problem; encapsulate the decision-making problem and the first environment to obtain an encapsulated second environment;

a second access module, configured to access the neural network to be trained;

a third access module, configured to access the architecture algorithm according to the second environment and the neural network to be trained;

an adaptation module, configured to adapt the second environment, the neural network to be trained and the architecture algorithm to generate trajectory data attributes; optimize the neural network to be trained according to the trajectory data attributes to obtain The neural network to be trained after configuring the architecture.

According to yet another aspect of the embodiments of the present invention, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete each other through the communication bus communication between;

The memory is used for storing at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above method for configuring a neural network architecture.

According to yet another aspect of the embodiments of the present invention, a computer storage medium is provided, where at least one executable instruction is stored in the storage medium, and the executable instruction causes a processor to execute a method corresponding to the above-mentioned method for configuring a neural network architecture. operate.

According to the solution provided by the above embodiments of the present invention, the decision problem of the neural network to be trained is accessed by accessing the decision problem; according to the decision problem, the first environment of the decision problem is obtained; the decision problem and the first environment are encapsulated , obtain the encapsulated second environment; access the neural network to be trained; access the architecture algorithm according to the second environment and the neural network to be trained; connect the second environment and the neural network to be trained and the architecture algorithm is adapted to generate trajectory data attributes; the neural network to be trained is optimized according to the trajectory data attributes to obtain the neural network to be trained after the configuration architecture, which improves the versatility of the neural network architecture configuration. Extensibility.

The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to understand the technical means of the embodiments of the present invention more clearly, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and The advantages can be more obvious and easy to understand, and the following specific implementations of the embodiments of the present invention are given.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for the purpose of illustrating the preferred embodiments, and are not considered to be limitations of the embodiments of the present invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 shows a flowchart of a method for configuring a neural network architecture provided by an embodiment of the present invention;

2 shows a flowchart of a specific method for configuring a neural network architecture provided by an embodiment of the present invention;

Fig. 3 shows a schematic diagram of architecture abstraction in a specific method for configuring a neural network architecture shown in Fig. 2;

4 shows a schematic structural diagram of an apparatus for configuring a neural network architecture provided by an embodiment of the present invention;

FIG. 5 shows a schematic structural diagram of a computing device provided by an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present invention will be more thoroughly understood, and will fully convey the scope of the present invention to those skilled in the art.

FIG. 1 shows a flowchart of a method for configuring a neural network architecture provided by an embodiment of the present invention. As shown in Figure 1, the method includes the following steps:

Step 11, construct the decision problem of accessing the neural network to be trained;

Step 12, obtaining the first environment of the decision-making problem according to the decision-making problem;

Step 13, encapsulating the decision-making problem and the first environment to obtain an encapsulated second environment;

Step 14, accessing the neural network to be trained;

Step 15, according to the second environment and the neural network to be trained, access the architecture algorithm;

Step 16, adapting the second environment, the neural network to be trained and the architecture algorithm to generate trajectory data attributes;

Step 17: Optimizing the neural network to be trained according to the attributes of the trajectory data to obtain the neural network to be trained after the configuration architecture.

In this embodiment, the decision problem of the neural network to be trained is accessed; according to the decision problem, the first environment of the decision problem is obtained; the decision problem and the first environment are encapsulated to obtain the encapsulated second environment; access the neural network to be trained; access the architecture algorithm according to the second environment and the neural network to be trained; connect the second environment, the neural network to be trained and the architecture algorithm Perform adaptation to generate trajectory data attributes; optimize the neural network to be trained according to the trajectory data attributes to obtain the neural network to be trained after the configuration architecture, which improves the versatility and scalability of the configuration of the neural network architecture.

In an optional embodiment of the present invention, step 12 may include:

Step 121, if the decision-making problem has an environment, obtain the first environment according to the environment included in the decision-making problem, and the first environment is a runnable environment;

Step 122, if there is no environment in the decision-making problem, define a new environment for the decision-making problem, obtain a defined third environment, and then obtain the first environment according to the third environment.

Fig. 2 shows a flowchart of a specific method for configuring a neural network architecture provided by an embodiment of the present invention. As shown in Fig. 2, in this embodiment, the environment is determined according to the existing environment or engine of the decision problem. access process. If the current decision-making problem scenario provides only the adjudication engine, the environment needs to be implemented according to the specific needs of the problem. During the implementation process, if the problem needs to be based on whether the problem itself is complete information, whether it is a turn-based system, whether it is a real-time system, etc. Layer access, but not limited to the above, can be contextualized according to the fully informative and turn-based properties of the problem. For example, for a specific backgammon problem scenario, if the decision problem scenario has not yet met the environment implementation of the standard gym interface, but there is already "a party's five chess pieces are arranged in a row on a horizontal line, a vertical line or an oblique line to win" The implementation engine of the adjudication rule can make the adjudication rule that "a party's five chess pieces are arranged in a row on a horizontal line, a vertical line or an oblique line to win" is compatible with the standard gym interface.

In yet another optional embodiment of the present invention, step 121 may include:

Step 1211, if the environment included in the decision-making problem satisfies a preset condition, determine the environment included in the decision-making problem as the first environment;

Step 1212: If the environment included in the decision-making problem does not satisfy the preset condition, the environment included in the decision-making problem is compatible with the general environment, and the compatible environment is determined as the first environment.

In this embodiment, the preset condition can be set to satisfy the gym standard interface, but it is not limited to the above. If the environment in the decision-making problem satisfies the gym standard interface, the decision problem is directly The environment is used as the first environment, and then the environment encapsulation layer is connected to realize the intercommunication between the environment and the agent; if the environment with the decision problem does not meet the gym standard interface, then the decision problem with the The environment is compatible with the gym standard interface, and the compatible environment is obtained, and then it is determined as the first environment, and then the environment encapsulation layer is connected to realize the interaction between the environment and the agent, or the compatibility and encapsulation can be carried out at the same time. , that is, the environment in the decision problem is directly encapsulated in the gym-compatible environment layer to realize the interaction between the environment and the agent.

In yet another optional embodiment of the present invention, in step 122, obtaining the first environment according to the third environment may include:

In step 1221, the third environment is compatible with the general environment, and the compatible environment is determined as the first environment.

In this embodiment, the third environment is first compatible with the gym standard interface to obtain a compatible environment, and then it is determined as the first environment, and then the environment encapsulation layer is connected to realize the interaction between the environment and the agent. For intercommunication, compatibility and encapsulation can also be performed at the same time, that is, the third environment is directly encapsulated in the gym-compatible environment layer to realize the intercommunication between the environment and the agent.

In yet another optional embodiment of the present invention, after step 13, it may further include:

Step 131: Obtain a state space of the second environment according to the second environment, where the state space refers to the space of states generated when the neural network to be trained is running.

Specifically, the state includes the state of the agent itself and the actions generated by the agent, but is not limited to the above.

In this embodiment, after step 13 and before step 14, it is necessary to determine the state space according to the type and scale of the second environment, and to determine the structure of the neural network to be trained. It is also necessary to determine the logical structure of the rule according to whether the knowledge rule is accessed, but it is not limited to the above. After step 14, the neural network and rules to be trained can also be connected to the inference layer.

In yet another optional embodiment of the present invention, after step 15, it may further include:

Step 151: According to the architecture algorithm, determine the attributes of the pre-generated data of the neural network to be trained.

In this embodiment, the attribute of the pre-generated data of the neural network to be trained includes the number of pre-generated actions in the pre-generated data of the neural network to be trained, but is not limited to the above. Taking the number of pre-generated actions as an example, for example, when using a single-agent algorithm, the number of actions output by the training party in the neural network to be trained is 1; when using a multi-agent algorithm, the number of actions output by the training party is greater than 1.

In yet another optional embodiment of the present invention, after step 151, it may further include:

Step 152: According to the architecture algorithm, obtain the dimension of the attribute of the pre-generated data of the neural network to be trained.

In yet another optional embodiment of the present invention, step 16 may include:

Step 161: Adapt the second environment, the neural network to be trained and the architecture algorithm, and then according to the state space of the second environment after adaptation, the neural network to be trained after adaptation and the pre-generated data of the neural network to be trained after adaptation to generate trajectory data attributes.

In this embodiment, each attribute of the pre-generated data is determined according to the state space of the second environment after adaptation, the neural network to be trained after adaptation, and the pre-generated data of the neural network to be trained after adaptation Then construct the (A, T, N, D) dimensional data of each attribute, and then determine the sampling logic of the action according to the algorithm and state space, access the agent adaptation layer, and finally connect according to the optimization logic in the algorithm. Enter the training layer, where A represents the dimension of the number of actions output by the agent to be trained in the neural network to be trained. For example, for a single agent, A is 1, and for a multi-agent, A is greater than 1 and A is equal to the number of agents ; T represents the step dimension of the interaction between the agent to be trained and the environment in the neural network to be trained, for example, if 100 steps are used to train the trajectory data once, then T is 100; N represents the quantity dimension of simultaneous parallel environment interaction, which can be equal to or Greater than 1, but not limited to the above, for example, for training in a single-machine and single-environment, N can be 1, for training in a single-machine multi-parallel environment, N can be the number of parallel environments, and for training in a multi-machine, multi-parallel environment , N can be the number of all parallel environments; D represents the dimension of a specific trajectory attribute data, for example, the dimension of the state information in the trajectory is (2, 3, 4, ), then the D stored for the state information is (2, 3, 4,); when constructing the (A, T, N, D) dimensional data of each attribute, for each attribute in the trajectory data, such as environmental status information, reward information, whether a round is completed, etc., Buffers all use a consistent (A, T, N, D) quadruple to represent the data dimension of specific attributes, which can meet the general expression requirements of various trajectory attributes and facilitate storage and reading in a general way.

In yet another optional embodiment of the present invention, after step 17, it may further include:

Step 18: Train the neural network to be trained after the configuration architecture.

In this embodiment, in the actual training process of the neural network to be trained after the configuration architecture, the agent adaptation layer directly interacts with the environment, inputs actions, and obtains observation data. Direct output, by mapping the direct output and sampling the action distribution, the action of interacting with the environment at the next moment is obtained. The trajectory data in the interaction process is stored in the Buffer, and the number of steps to be interacted is generally a hyperparameter, which needs to be dynamically adjusted according to the problem. The training layer uses the batch trajectory data to optimize the parameters of the deep neural network model, and updates the optimization results to the inference layer. At this point, the one-step training process is completed. This training process is repeated continuously until the evaluation indicators that need to be optimized converge, and the overall training process is completed.

FIG. 2 shows a flowchart of a specific method for configuring a neural network architecture provided by an embodiment of the present invention. As shown in FIG. 2 , the method implements a general training architecture based on deep reinforcement learning training decisions through the following four points. The architecture can provide standard engine/environment, model and algorithm access methods, reducing the difficulty and workload of applying deep reinforcement learning to various intelligent decision-making problems:

First, establish a common environment access method;

Second, in a scenario where there is only an adjudication engine but no specific environment, a general engine access method is provided;

Third, establish a general algorithm access method;

Fourth, establish a general strategy abstraction to meet the access needs of various deep neural network models and artificial rule models.

Specifically, a general access method is provided for engines and environments; a general access method is provided for various deep enhancement algorithms; and a general access method is provided for various deep neural network models or artificial rule models.

The following uses a specific scenario to illustrate how the neural network to be trained implements the overall process from the start of the access to the completion of the access through the specific method of configuring the neural network architecture shown in FIG. 2 :

"XX Soccer Environment" is a novel open source reinforcement learning environment that provides physics-based 3D soccer simulations modeled on popular soccer video games. Agents controlling players, learning how to pass the ball, and how to overcome opponents' defenses to score goals are challenging decision-making problems.

Based on the specific method of configuring the neural network architecture shown in Figure 2, the football game decision problem can be solved using the single-agent PPO algorithm according to the following standard steps:

The first step, environment access: The football environment itself satisfies the gym standard interface, and does not need to access the engine layer, the problem definition layer and the Gym compatible environment layer. Connect the existing step and reset methods of the environment to the step and reset standard interfaces provided by the environment encapsulation layer, and the step interface returns the reward, status and other information after interacting with the environment one step. According to the definition of action space and state space, construct two standard interfaces for obtaining environment action space and state space, including get_observationspace (observation space) and get_actionspace (action space).

The second step, model access: You can choose a variety of deep neural network models to solve the football problem. For example, considering the continuity of the football game process, the model is constructed based on the Long Short Term Memory network (Long Short Term, LSTM). Based on the constructed model, define the standard interfaces of the inference layer such as initialization (init), obtaining parameters (get_parameter), setting parameters (set_parameter), obtaining gradients (get_gradient), setting gradients (set_gradient), and inference (inference), and completing model access .

The third step, algorithm access: The algorithm uses trajectory data for training. There are generally multiple attributes in the trajectory data, and the data dimensions of each attribute are different. According to the selected algorithm, define the data dimension of each track attribute in the data buffer (Buffer) data dimension (A, T, N, D). For the general data dimensions of reinforcement learning such as state and action attributes in the trajectory data, it can be determined from the two standard interfaces for obtaining the environment action space and state space defined in the first step; for the additional attribute data required by the algorithm, you can press The definition of algorithm requirements, such as the PPO algorithm, requires the value attribute to represent the output of the critic network. Assuming that the output data dimension is (123,), the data dimension defined for the value attribute is (123,). Since the single-agent algorithm is selected, the number of action outputs in the Buffer data dimension (A, T, N, D) is 1. According to the current computing resources, the number of parallel running environments in the Buffer data dimension (A, T, N, D) can be determined. T is how many steps to interact with the football environment to perform a training, which can be adjusted as a hyperparameter of the training process. A variety of deep reinforcement algorithms can be used to solve the football problem. For example, considering the training efficiency and convergence stability, PPO is selected as the training algorithm. Based on the core logic of the training algorithm, define the update (_update) standard interface of the training layer. Based on the action space, define the standard interface of the action sampling (handle_sample) of the agent adaptation layer. So far, the algorithm access is completed.

The fourth step, training convergence: After the above three steps, the access to the training framework for solving the football problem is completed. After the connection is completed, the training process can be started, and the algorithm hyperparameters can be adjusted until the score converges, and the solution to the football problem is completed.

Fig. 3 shows a schematic diagram of an abstract layered architecture in a specific method for configuring a neural network architecture shown in Fig. 2. As shown in Fig. 3, in the above-mentioned embodiment, including but not limited to corresponding to the In the embodiment, the framework for configuring the neural network is abstracted in layers, and is divided into seven layers in total:

The first layer, training layer: the training layer implements the deep reinforcement learning algorithm, uses the sampled trajectory data stored in the Buffer to optimize the network model, and updates the network parameters to the inference layer;

The second layer, inference layer: the inference layer abstracts the deep neural network model and rules uniformly, and is responsible for executing the policy inference output;

The third layer, the agent adaptation layer: the adaptation layer calls the inference layer for inference, and performs policy mapping and action sampling on the inference results, and interacts with the environment. The interactive trajectory data is stored in the Buffer, and the agent adaptation layer When interacting with the environment, shield the concepts of single-agent and multi-agent in the deep reinforcement learning algorithm, and take all the agents of a party as a whole to achieve interaction with the environment;

The fourth layer, the environment encapsulation layer: the environment encapsulation layer realizes the common standard interaction with the agent adaptation layer;

The fifth layer, Gym compatible environment layer: compatible with the widely used OpenAI gym (a toolkit for developing and comparing reinforcement learning algorithms) standard interface;

The sixth layer, the problem definition layer: defines the problem to be solved, and defines and realizes the problems of complete information, turn-based and real-time system;

The seventh layer, the engine layer: responsible for accessing the simulator engine.

The combination of the seven layers abstracted above and the architecture of the configuration neural network can achieve the following:

1. Standardize the intelligent decision-making training process based on deep reinforcement learning, and abstract the environment, algorithms and models in the training process on a unified architecture, with scalability. The training architecture adopts single-machine or distributed, synchronous or asynchronous without mandatory constraints for the training process, and adapts to the future access to new deep reinforcement learning algorithms.

2. Through clear and detailed stratification, the data flow between layers is determined, which simplifies the complexity of applying deep reinforcement learning to solve intelligent decision-making problems, and provides a solution for intelligent body training that "sees the picture according to the picture".

3. Based on the architecture of the neural network in Figure 2, through the flexible combination of abstract components, it is possible to realize the distributed reinforcement learning framework that supports various mainstreams.

Fourth, the deep neural network model and the rule model are unified and abstracted, which has better generality in game problems based on knowledge and experience.

5. By abstracting different concepts such as single agent and multi-agent in the deep reinforcement learning algorithm as a whole, it simplifies the complexity of the interaction between different types of agents and the environment.

The beneficial effects of the method for configuring a neural network architecture provided by the embodiments of the present invention are described below with three specific scenarios:

Scenario 1: Solving an intelligent decision-making problem based on a general architecture: When faced with an intelligent decision-making problem, a set of training programs can be constructed according to the method for configuring a neural network architecture provided in the embodiment of the present invention, and a deep reinforcement learning algorithm is used to generate an agent model . In the training architecture, the definition and behavior of each component and the data flow between each other are constrained, so that the training plan can be completed with "laws to follow" and "follow the charts".

Scenario 2: Use existing deep reinforcement learning algorithms to solve new intelligent decision-making problems: When faced with new intelligent decision-making problems, the method for configuring the neural network architecture provided by the embodiment of the present invention does not need to access new deep reinforcement learning The initial verification of the problem can be achieved only by using the deep reinforcement learning algorithm that has been accessed. Dramatically reduces the effort to build problem-solving processes.

Scenario 3: Use an existing intelligent decision-making problem to verify a new deep reinforcement learning algorithm: When a new algorithm needs to be researched and verified, the method for configuring the neural network architecture provided by the embodiment of the present invention is used, and there is no need to access a new environment, The verification process of the new algorithm can be built only by using the previously connected intelligent decision-making environment.

In the above-mentioned embodiment of the present invention, the agent training process based on deep reinforcement learning is comprehensively abstracted, and the whole training process such as engine/environment access, algorithm access, model access, and buffer data access provides a general analysis The layer division and access method greatly reduces the difficulty and workload of accessing new environments and new algorithms; abstracts the interaction between different algorithms and environments such as single agent and multi-agent as the interaction between the overall agent and the environment , which greatly expands the applicability of the training architecture in the interaction of different algorithms and environments; the unified model abstraction of deep neural networks and rules in the inference layer can be applied to various knowledge-based game confrontation intelligent decision-making problems.

FIG. 4 shows a schematic structural diagram of an apparatus 40 for configuring a neural network architecture provided by an embodiment of the present invention. As shown in Figure 4, the device includes:

The first access module 41 is used to access the decision problem of the neural network to be trained;

The processing module 42 is configured to obtain the first environment of the decision-making problem according to the decision-making problem; encapsulate the decision-making problem and the first environment to obtain an encapsulated second environment;

a second access module 43, configured to access the neural network to be trained;

A third access module 44, configured to access the architecture algorithm according to the second environment and the neural network to be trained;

The adaptation module 45 is configured to adapt the second environment, the neural network to be trained and the architecture algorithm to generate trajectory data attributes; optimize the neural network to be trained according to the trajectory data attributes, Get the neural network to be trained after configuring the architecture.

Optionally, the processing module 42 is further configured to obtain the first environment according to the environment in the decision-making problem if there is an environment in the decision-making problem;

If there is no environment in the decision-making problem, define a new environment for the decision-making problem, obtain a defined third environment, and then obtain the first environment according to the third environment.

Optionally, the processing module 42 is further configured to determine the environment included in the decision-making problem as the first environment if the environment included in the decision-making problem satisfies a preset condition;

If the environment included in the decision-making problem does not satisfy the preset condition, the environment included in the decision-making problem is compatible with the general environment, and the compatible environment is determined as the first environment.

Optionally, the processing module 42 is further configured to make the third environment compatible with the general environment, and determine the compatible environment as the first environment.

Optionally, the processing module 42 is further configured to obtain the state space of the second environment according to the second environment, where the state space refers to the space of states generated when the neural network to be trained is running. .

Optionally, the third access module 44 is further configured to determine attributes of the pre-generated data of the neural network to be trained according to the architecture algorithm.

Optionally, the adaptation module 45 is further configured to adapt the second environment, the neural network to be trained, and the architecture algorithm, and then according to the state space of the second environment after adaptation, The adapted neural network to be trained and the pre-generated data of the adapted neural network to be trained are used to generate trajectory data attributes.

It should be understood that the above descriptions of the method embodiments illustrated in FIGS. 1 to 3 are merely illustrative of the technical solutions of the present invention by way of optional examples, and do not limit the method for configuring the neural network architecture involved in the present invention. In other embodiments, the execution steps and sequence of the method for configuring a neural network architecture involved in the present invention may be different from the foregoing embodiments, which are not limited in this embodiment of the present invention.

It should be noted that this embodiment is an apparatus embodiment corresponding to the foregoing method embodiment, and all implementation manners in the foregoing method embodiment are applicable to this apparatus embodiment, and can achieve the same technical effect.

An embodiment of the present invention provides a non-volatile computer storage medium, where the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the method for configuring a neural network architecture in any of the foregoing method embodiments.

FIG. 5 shows a schematic structural diagram of a computing device provided by an embodiment of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

As shown in FIG. 5 , the computing device may include: a processor (processor), a communications interface (Communications Interface), a memory (memory), and a communication bus.

Among them: the processor, the communication interface, and the memory communicate with each other through the communication bus. The communication interface is used to communicate with network elements of other devices such as clients or other servers. The processor is configured to execute a program, and specifically, may execute the relevant steps in the above-mentioned embodiments of the method for configuring a neural network architecture for a computing device.

Specifically, the program may include program code, the program code including computer operation instructions.

The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be the same type of processors, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

memory for storing programs. The memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

The program can specifically be used to cause the processor to execute the method for configuring the neural network architecture in any of the above method embodiments. For the specific implementation of the steps in the program, reference may be made to the corresponding descriptions in the corresponding steps and units in the above-mentioned embodiments of the method for configuring a neural network architecture, which will not be repeated here. Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and modules, reference may be made to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated here.

The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with teaching based on this. The structure required to construct such a system is apparent from the above description. Furthermore, embodiments of the present invention are not directed to any particular programming language. It is to be understood that various programming languages can be used to implement the contents of the embodiments of the invention described herein and that the above descriptions of specific languages are intended to disclose the best mode of carrying out the embodiments of the invention.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in order to simplify the embodiments of the invention and to aid in the understanding of one or more of the various aspects of the invention. in a single embodiment, figure, or description thereof.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies.

Furthermore, it will be understood by those skilled in the art that although some of the embodiments herein include certain features, but not others, included in other embodiments, that combinations of features of the different embodiments are intended to be within the scope of the present invention And form different embodiments.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. Embodiments of the present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing embodiments of the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit embodiments of the invention, and that the word "a" or "an" preceding an element does not preclude the presence of a plurality of such elements. Embodiments of the invention may be implemented by means of hardware comprising several different elements and by means of suitably programmed computers. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names. The steps in the above embodiments should not be construed as limitations on the execution order unless otherwise specified.

Claims (10)

1. A method for configuring a neural network architecture, wherein the method comprises: Access the decision-making problem of the neural network to be trained; According to the decision-making problem, obtain the first environment of the decision-making problem; Encapsulating the decision problem and the first environment to obtain an encapsulated second environment; access the neural network to be trained; Access the architecture algorithm according to the second environment and the neural network to be trained; Adapting the second environment, the neural network to be trained, and the architecture algorithm to generate trajectory data attributes; The neural network to be trained is optimized according to the attributes of the trajectory data to obtain the neural network to be trained after the configuration architecture. 2. The method for configuring a neural network architecture according to claim 1, wherein, according to the decision-making problem, obtaining the first environment of the decision-making problem, comprising: If there is an environment in the decision-making problem, obtaining the first environment according to the environment in the decision-making problem; If there is no environment in the decision-making problem, define a new environment for the decision-making problem, obtain a defined third environment, and then obtain the first environment according to the third environment. 3. The method for configuring a neural network architecture according to claim 2, wherein if there is an environment in the decision-making problem, the first environment is obtained according to the environment in the decision-making problem, comprising: If the environment included in the decision-making problem satisfies the preset condition, determining the environment included in the decision-making problem as the first environment; If the environment included in the decision-making problem does not satisfy the preset condition, the environment included in the decision-making problem is compatible with the general environment, and the compatible environment is determined as the first environment. 4. The method for configuring a neural network architecture according to claim 2, wherein obtaining the first environment according to the third environment comprises: The third environment is compatible with the general environment, and the compatible environment is determined as the first environment. 5. The method for configuring a neural network architecture according to claim 1, wherein after obtaining the encapsulated second environment, further comprising: According to the second environment, the state space of the second environment is obtained, and the state space refers to the space of states generated when the neural network to be trained is running. 6. The method for configuring a neural network architecture according to claim 5, characterized in that, after accessing the architecture algorithm, further comprising: According to the architecture algorithm, the attributes of the pre-generated data of the neural network to be trained are determined. 7. The method for configuring a neural network architecture according to claim 6, wherein the second environment, the neural network to be trained and the architecture algorithm are adapted to generate trajectory data attributes, comprising: Adapting the second environment, the neural network to be trained and the architecture algorithm, and then according to the state space of the second environment after adaptation, the neural network to be trained after adaptation, and the adaptation The latter neural network to be trained pre-generates data to generate trajectory data attributes. 8. A device for configuring a neural network architecture, wherein the device comprises: The first access module is used to access the decision problem of the neural network to be trained; a processing module, configured to obtain a first environment of the decision-making problem according to the decision-making problem; encapsulate the decision-making problem and the first environment to obtain an encapsulated second environment; a second access module, configured to access the neural network to be trained; a third access module, configured to access the architecture algorithm according to the second environment and the neural network to be trained; an adaptation module, configured to adapt the second environment, the neural network to be trained and the architecture algorithm to generate trajectory data attributes; optimize the neural network to be trained according to the trajectory data attributes to obtain The neural network to be trained after configuring the architecture. 9. A computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; The memory is used for storing at least one executable instruction, and when the at least one executable instruction is executed, the processor executes the method for configuring a neural network architecture according to any one of claims 1-7. 10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and when the executable instruction is executed, the computing device executes the method of configuring the neural network architecture according to any one of claims 1-7. method.

Images