CN111694557B

CN111694557B - Data processing method and device, image processing method and device and electronic equipment

Info

Publication number: CN111694557B
Application number: CN201910199174.8A
Authority: CN
Inventors: 李周洋; 张行程; 颜深根; 何家傲
Original assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Intelligent Technology Co Ltd
Priority date: 2019-03-15
Filing date: 2019-03-15
Publication date: 2024-04-16
Anticipated expiration: 2039-03-15
Also published as: CN111694557A

Abstract

The disclosure relates to a data processing method and device, an image processing method and device and an electronic device, wherein the data processing method comprises the following steps: acquiring a neural network code; performing code translation processing on the neural network codes to obtain intermediate representation codes, wherein the intermediate representation codes abstract the realization of different hardware devices for the same operation; the intermediate representation code is executed by an execution engine. The present disclosure can improve the scalability of neural network codes.

Description

Data processing method and device, image processing method and device and electronic equipment

Technical Field

The disclosure relates to the field of data processing, and in particular relates to a data processing method and device, an image processing method and device, and an electronic device.

Background

In the field of machine learning, the implementation of neural networks (e.g., deep learning networks) is a process that solves a complex mathematical model. The structured representation of complex mathematical models is a precondition for the realization of machine learning tasks.

At present, a typical deep learning framework uses Python and other languages as a programming interface, and an upper description language directly corresponds to a back-end implementation, so that the deep learning framework is difficult to convert to a different back-end.

Disclosure of Invention

The embodiment of the disclosure provides a data processing method and device, an image processing method and device and electronic equipment, which can improve the code expandability of a neural network.

According to an aspect of the present disclosure, there is provided a data processing method, including:

acquiring a neural network code;

performing code translation processing on the neural network codes to obtain intermediate representation codes, wherein the intermediate representation codes abstract the realization of different hardware devices for the same operation;

the intermediate representation code is executed by an execution engine.

In some possible embodiments, the performing a code translation process on the neural network code to obtain an intermediate representation code includes:

running the neural network code;

in response to a function object in the neural network code being executed, a generator interface is invoked to generate an intermediate representation code corresponding to the function object.

In some possible implementations, the calling generator interface, in response to a function object in the neural network code being executed, includes:

in response to a function object in the neural network code being executed, a generator interface corresponding to the function object is invoked based on a type of the function object to generate intermediate representation code corresponding to the function object.

In some possible implementations, the generating, by the generator interface, intermediate representation code corresponding to the function object includes:

and in response to the condition that the same function object is repeatedly operated, calling the generator interface to generate an intermediate representation code corresponding to the same function object each time the same function object is operated.

when the function object is repeatedly called, the saved intermediate representation code corresponding to the function object is run,

the stored intermediate representation code corresponding to the function object is the intermediate representation code generated by calling the generator interface when the function object is called for the first time, or the intermediate representation code generated by calling the generator interface before the function object is actually called.

In some possible implementations, the running the neural network code includes: the neural network code is interpreted by an interpreter and/or compiled by a compiler.

In some possible embodiments, the intermediate representation code corresponding to the function object in the neural network includes: the function object corresponds to the operation name, the operation additional attribute, the operation input and the operation output.

In some possible implementations, the intermediate representation code is configured to represent, in an abstract symbolic manner, a function object included in the neural network code, and the intermediate representation code corresponding to the function object is independent of a context of the function object.

In some possible implementations, the executing, by the execution engine, the intermediate representation code includes:

and calling a computing module and/or a communication module to execute the intermediate representation code through the execution engine.

In some possible embodiments, the method further comprises:

optimizing the intermediate representation code;

the executing, by the execution engine, the intermediate representation code, comprising:

and executing the optimized intermediate representation code through an execution engine.

In some possible implementations, the optimizing the intermediate representation code includes at least one of:

optimizing a memory usage mode corresponding to the intermediate representation code;

Merging at least a portion of the intermediate representation code for an arithmetic operation;

at least a portion of the intermediate representation code is merged for use in a communication operation.

In some possible embodiments, the method further comprises:

and storing the intermediate representation codes according to the generation sequence of the intermediate representation codes.

In some possible embodiments, the method further comprises:

forming a visualization structure corresponding to the intermediate representation code by using a visualization tool;

and displaying the visual structure.

In some possible embodiments, the method further comprises:

acquiring an image to be processed;

the step of performing code translation processing on the neural network code to obtain an intermediate representation code comprises the following steps:

operating the neural network code by taking the image to be processed as an operation object to obtain an intermediate representation code corresponding to the neural network code;

and executing the intermediate representation code by using the execution engine to obtain a processing result of the image to be processed.

In some possible embodiments, the image to be processed is a sample image, the method further comprising:

Determining network loss based on the processing result of the image to be processed;

based on the network loss, network parameters of the neural network are adjusted.

According to a second aspect of the present disclosure, there is provided an image processing method including:

acquiring an image to be processed;

and operating an intermediate representation code corresponding to the neural network code by taking the image to be processed as an operation object to obtain a processing result of the image to be processed, wherein the intermediate representation code is generated by the method according to any one of the first aspect.

According to a fifth aspect of the present disclosure, there is provided an electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: performing the method of any of the first aspects or performing the method of the second aspect.

According to a sixth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method of any one of the first aspects, or implement the method as described in the second aspect.

According to the embodiment of the disclosure, when the neural network code corresponding to the neural network code is operated, the intermediate representation code corresponding to the function object in the neural network code can be generated at the same time, the intermediate representation code abstracts the realization of different hardware devices for the same operation, namely, the neural network code is abstracted, and the method can be applied to different types of hardware devices, and the intermediate representation code is the code irrelevant to the type of the rear-end execution hardware of the neural network, so that the method can be applied to different rear-ends. The embodiment of the disclosure has the characteristics of simplicity and convenience, and can improve the expandability of the neural network codes.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the technical aspects of the disclosure.

FIG. 1 illustrates a flow chart of a data processing method according to an embodiment of the present disclosure;

FIG. 2 illustrates a flow chart of code translation in a data processing method according to an embodiment of the present disclosure;

FIG. 3 illustrates a flow chart of a visualized intermediate representation code in a data processing method of an embodiment of the present disclosure;

FIG. 4 illustrates another flow chart of a data processing method according to an embodiment of the present disclosure;

FIG. 5 illustrates another flow chart of a data processing method according to an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of one example of a data processing method according to an embodiment of the present disclosure;

FIG. 7 shows a flowchart of an image processing method according to an embodiment of the present disclosure;

FIG. 8 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

Fig. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure;

FIG. 10 illustrates a block diagram of an electronic device according to an embodiment of the disclosure;

fig. 11 illustrates another block diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Furthermore, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, components, and circuits that are well known to those skilled in the art have not been described in detail in order to not obscure the subject matter of the present disclosure.

The embodiment of the disclosure provides a data processing method, which can convert a neural network code into an intermediate representation code irrelevant to the back-end execution hardware of the neural network, so that the coupling between the front-end and the back-end of the neural network can be decoupled, and the method can be suitable for different back-end processing. The embodiment of the disclosure also has the characteristics of simplicity and convenience, can improve the applicability of codes, and has the characteristic of more flexibility. In addition, the data processing method provided by the embodiment of the present disclosure may be applied to any device having a machine learning function, such as a mobile terminal device such as a mobile phone, a computer device, a PAD, an intelligent wearable device such as a bracelet, or a server device, which is not illustrated herein.

It should be understood that the data processing method provided in the embodiments of the present disclosure may be applied to a training process of a neural network, or may be applied to a testing process of a neural network or a process of performing image processing in an actual scene, which is not limited in the embodiments of the present disclosure.

Fig. 1 shows a flowchart of a data processing method according to an embodiment of the present disclosure, wherein the data processing method may include:

s100: acquiring a neural network code;

in the embodiment of the disclosure, abstract representation of the neural network code can be realized, and because the neural network code in different languages cannot be executed in different hardware devices at the same time due to the diversification of programming languages of the neural network code, the embodiment of the disclosure provides a data processing method capable of abstracting and translating the neural network code in different programming languages into intermediate representation code.

First, neural network code that needs to be translated into intermediate representation code may be obtained. In some possible implementations, the neural network code may be a neural network code that implements any function, such as a neural network code that performs image processing (image recognition, image segmentation, image optimization, etc.), or a neural network code that also performs language word processing, or a neural network code that performs audio processing, which are not illustrated herein, but rather are embodiments of the disclosure.

In addition, in some possible implementations, the neural network code may be a code program written in a high-level language. For example, the high-level languages include at least one of the following: java, C, c++, c#, python, lisplua. In other embodiments, the neural network code may be written in other languages, as well, which is not specifically limited by the present disclosure.

In some possible embodiments, the manner of acquiring the neural network code may include: the neural network code to be translated is read from a database, or the neural network code transmitted by other devices is received, wherein the database may include a local database and/or a cloud database. The foregoing is merely exemplary, and the neural network code may be obtained in other embodiments in other ways.

S200: performing code translation processing on the neural network codes to obtain intermediate representation codes, wherein the intermediate representation codes abstract the realization of different hardware devices for the same operation;

as described in the above embodiments, the embodiments of the present disclosure may implement the translation of the neural network code, that is, may translate the neural network code from the original language form to the language form of the intermediate representation code. The intermediate representation code of the embodiment of the disclosure can be a self-defined language structure, and the operation front end of the intermediate representation code is irrelevant to the reasoning back end, so that the coupling between the front end (model analysis) and the back end (mathematical calculation) in the neural network learning framework can be released, and the same network model can be conveniently executed on different back ends. In addition, the intermediate representation code of the embodiment of the present disclosure may be used to represent the function objects included in the neural network code in an abstract symbol manner, so that the intermediate representation code may be executed on different types of hardware devices, and the applicability and the scalability of the neural network code are increased.

In some possible embodiments, the translation of the function objects in the neural network code may be implemented when the neural network code is executed, for example, when the function objects in the neural network code are called, intermediate representation code corresponding to the called function objects may be generated. The intermediate representation code may be used to interpret the corresponding function object.

In the embodiments of the present disclosure, the neural network code may include at least one function object, where the function object may be an operation function, a control function, an indication function, or any other function, which is not illustrated herein, and may be a function object in the embodiments of the present disclosure as long as the function object is a function in the neural network code.

In the embodiment of the disclosure, when the neural network code is executed, the function object in the neural network code is called, and at this time, the operation corresponding to the function object can be translated into the intermediate representation code. In other words, in the process that the neural network code is executed, the embodiments of the present disclosure may generate intermediate representation code corresponding to the executed operation of the function object according to the call of each function object of the neural network code. Through the translation process, the neural network codes in the form of high-level languages can be converted into intermediate representation codes, and the embodiment of the disclosure can translate the control and calculation logic of the high-level language description into the predefined intermediate representation codes in real time by means of the operation mechanism of the high-level language (such as Python). The intermediate representation code is a self-defined code for explaining each operation process of the neural network code, namely, the function objects in the neural network code are represented in an abstract symbol mode, and the intermediate representation code formed by the translation process is independent of the context of the function objects, so that the association between the front end and the back end of the neural network can be released, and the translated code can be suitable for different back end processing.

S300: the intermediate representation code is executed by an execution engine.

In the disclosed embodiments, after translating the neural network code into the intermediate representation code, the intermediate representation code may be executed by an execution engine. The intermediate representation code may have different functional implementations for different neural network codes, for example, may be used to perform one or more of arithmetic operations, communication operations, control operations, or may be used to implement or perform other operations, as the disclosure is not limited in this regard. According to the operation corresponding to the intermediate representation code, the execution engine can control the corresponding device or equipment to execute the corresponding operation. For example, for intermediate representation code that performs arithmetic operations, the execution engine may execute the intermediate representation code through a computing module (e.g., CPU, GPU), or for intermediate representation code of communication operations, the execution engine may execute the intermediate representation code through a communication module, i.e., the execution engine may perform the corresponding operations through devices, means, modules capable of performing the operations corresponding to the intermediate representation code.

Additionally, in some possible implementations, after the neural network code is translated into the predefined intermediate representation code, the translated intermediate representation code may be stored for subsequent visual transformations or performing operations, among other things. The translated intermediate representation codes can be exported and used for deployment of different back ends and visualization of models, and the stored intermediate representation codes can be stored in a text form and/or a picture form. Wherein, each intermediate representation code can be stored according to the generation sequence of each generated intermediate representation code in the operation process of the neural network code. The stored intermediate representation code may be executed when the intermediate representation code is executed by the execution engine.

According to the embodiment of the disclosure, the neural network code can be conveniently converted into the form of the intermediate representation code, and the intermediate representation code can abstract the realization mode of different devices on the same operation, so that the neural network code can be executed on different hardware devices, the applicability of the neural network code can be improved, and the neural network code translation method has the characteristics of simplicity and convenience in translation process.

The embodiments of the present disclosure are described in detail below. Fig. 2 shows a flow chart of code translation in a data processing method according to an embodiment of the present disclosure. Wherein, based on the performing code translation processing on the neural network code, an intermediate representation code is obtained (step S200), which may include:

s201: running the neural network code;

s202: in response to a function object in the neural network code being executed, a generator interface is invoked to generate an intermediate representation code corresponding to the function object.

As described in the above embodiments, the embodiments of the present disclosure may generate the intermediate representation code corresponding to the function object when the function object in the neural network code is called. I.e. the original code in the neural network code may be represented by a predefined intermediate representation code.

In some possible implementations, the neural network code in the embodiments of the present disclosure may be interpreted by an interpreter. The interpreter is capable of directly translating and running the high-level programming language corresponding to the neural network code. For different types of high-level languages, execution may be interpreted by different types of interpreters. For example, when the neural network code acquired in step S100 is written based on the Python language, the neural network code may be interpreted and executed by a Python language interpreter. For other types of languages, which may also be interpreted by a corresponding interpreter, the present disclosure is not illustrated. Alternatively, in other embodiments, the neural network code may be compiled and executed by a compiler, e.g., for a c++ language type of neural network code, the neural network code may be compiled into executable code by the compiler and then the executable code is re-executed. The following embodiments are described by taking an interpreter as an example, and a procedure for calling a generator interface by executing a neural network code by a compiler is the same as a procedure for interpreting the execution of the neural network code by the interpreter and striping the generator interface, and will not be repeated.

Wherein, when the neural network code is interpreted and executed by the interpreter, each function object in the neural network code can be translated into a corresponding intermediate representation code in sequence according to the sequence of the operation of the execution of the neural network code. The neural network code may include a plurality of function objects, each of which may be used to perform a different operation, such as may be used to perform arithmetic operations, control operations, instruction operations, and the like. The operation may include addition, subtraction, multiplication, division, convolution, and the like, and the operation object may be a scalar or a tensor. The control operations may include loop operations, branching operations, etc., which may correspond to logical processes of if, while, etc. in the high-level language. The indication operation may include indication of unequal allocation and reclamation of resources such as memory allocation, memory transfer, memory release, etc.

Correspondingly, when the code segments corresponding to the function objects are interpreted and called through the interpreter, the intermediate representation codes corresponding to the function objects can be generated through the interface of the call generator. The generator interface is a function interface of a generator which can be used for generating intermediate representation codes, and when a function object in the neural network codes is operated and called, the generator can call the generator interface to generate the intermediate representation codes corresponding to the called function object.

When the interpreter executes the neural network code, the number of lines of the code executed by each interpretation may be determined according to the type of the high-level language, and may be one line or multiple lines, which is not specifically limited in the present disclosure.

In some possible implementations, various types of producer interfaces may be included in embodiments of the disclosure, each type of producer interface corresponding to a type of function object. For example, in embodiments of the present disclosure, a generator interface defined in a generator may include; at least one of a producer interface of an arithmetic operation type, a producer interface of a control operation type, and a producer interface of a communication operation type, in other embodiments other types of producer interfaces may also be included. The generator interfaces of each type included in the embodiments of the present disclosure may correspond to the type of each function object in the neural network code, and when the function object is executed, the generator may call the generator interface corresponding to the type of the function object to generate the intermediate representation code corresponding to the function object. That is, embodiments of the present disclosure may enable the generation of respective intermediate representation code through a generator interface corresponding to a type of a function object when running different types of function objects.

In some possible implementations, the generator interface may define the generated intermediate representation code in a preset manner. Wherein, the intermediate representation code generated by the call generator interface may include: the function object corresponds to the operation name, the operation input and the operation output. In other embodiments, additional operational attributes, or other neural network code-related parameter information may also be included in the intermediate representation code.

The operation name may include at least one of an arithmetic operation, a control operation, a communication operation, and the like, or may be a name of a function object, and the like. Additional attributes of an operation refer to operational parameters employed to execute the function object, such as convolution parameters, weights, coefficients, etc., which may be determined based on neural network parameters. The input and output may be the input and output of a function object, respectively, and may be specifically determined according to the function of the function object and the operation procedure of the neural network.

When the generator interface in the embodiment of the disclosure generates the intermediate representation code, the corresponding intermediate representation code may be generated according to the above-defined manner, where the operation names, the attributes, the input and output identifiers may be determined according to a preset relationship, where the preset relationship may include intermediate symbols corresponding to each operation, intermediate symbols corresponding to each control operation, intermediate symbols corresponding to each communication operation, intermediate symbols corresponding to operands and attribute parameters, and so on. That is, the embodiment of the disclosure can query the intermediate symbols corresponding to the codes in the function objects of the neural network codes through the preset corresponding relation, and determine the intermediate representation codes corresponding to the code segments corresponding to the functions based on the queried intermediate symbols.

In addition, in some possible embodiments, when the neural network code is executed, there may be a case where the same function object is called and executed multiple times, for which the generator interface may be called each time the function object is executed to generate the intermediate representation code corresponding to the same function object. And the intermediate representation code generated each time can cover the intermediate representation code generated in the previous time, so that the occupation of the memory is reduced.

Alternatively, in other embodiments, when the same function object is called and run multiple times, the saved intermediate representation code corresponding to the function object may be run, where the saved intermediate representation code corresponding to the function object is an intermediate representation code generated by calling the generator interface when the function object is called for the first time, or is an intermediate representation code generated by calling the generator interface before the function object is actually called. In some embodiments, the intermediate representation code corresponding to the function object may be generated through the call generator interface when the function object is executed for the first time, and the intermediate representation code corresponding to the function object is saved, and meanwhile, when the function object is repeatedly called, the saved intermediate representation code corresponding to the function object is executed. Or before the function object is actually called, the corresponding generator interface can be called to generate and store the intermediate representation code corresponding to the function object, so that the intermediate representation code can be directly executed in actual application. That is, in the embodiment of the present disclosure, for the same function object, only the intermediate representation code corresponding to the same function object is generated when the function object is executed for the first time, and when the function object is called for multiple times, the stored intermediate representation code can be executed, so that the execution speed can be increased.

In some possible implementations, the embodiments of the present disclosure may also optimize each intermediate representation code after translating the neural network code into a corresponding intermediate representation code. In the embodiment of the disclosure, the optimization of the intermediate representation code refers to equivalent (meaning that the running result of the program is not changed) transformation of the intermediate representation code. Equivalent transformation is to make the transformed code running result identical to the pre-transformation code running result and function implementation. That is, embodiments of the present disclosure may provide space for static analysis and optimization of the neural network model translated into intermediate representation code prior to actual use of the model. For example, static analysis and optimization which are irrelevant to the running time, such as memory allocation optimization, calculation merging, communication merging and the like, can be performed on the translated intermediate representation code. Embodiments of the present disclosure may enable resource occupation of intermediate representation code during execution to be reduced by optimization of the intermediate representation code, for example, generated intermediate representation code may be optimized to be shorter (run time shorter, occupation space smaller), or space-time efficiency optimization may be performed. The manner in which the embodiments of the present disclosure optimize each intermediate representation code may include at least one of the following:

a: optimizing a memory usage mode corresponding to the intermediate representation code;

the memory usage mode corresponding to the optimized code in the embodiment of the disclosure may include: the allocation and release of the memory resources are regulated, the sequence of the memory allocation, the transfer code, the memory multiplexing and other modes are regulated, so that the time and sequence of the memory allocation and release and the reasonable multiplexing can be reasonably arranged, and the occupation of the memory resources can be reduced. In other embodiments, the memory layout of the intermediate representation code may be optimized in other ways as well.

b: merging at least a portion of the intermediate representation code for an arithmetic operation;

by combining the codes of the operation operations, a plurality of operation operations can be fused into one operation with little milk, so that the codes are shortened, and the code running speed is improved. In addition, the operation result is ensured under the condition of reducing the length of the code, and on the other hand, the resources occupied by the operation can be reduced. The code for merging the operation may include merging a plurality of code segments implementing a plurality of operation into one code, and may be capable of implementing the same operation result.

c: at least a portion of the intermediate representation code is merged for use in a communication operation.

By combining the codes of the communication operation, the communication requirement can be met under the condition of reducing the length of the codes, and on the other hand, the resources occupied by the communication operation and the delay caused by frequent communication can be reduced. The mode of combining the codes of the communication operation may include combining a plurality of code segments for implementing a plurality of communication operations into one code, and simultaneously, the same communication purpose can be implemented.

By the method, the optimization of the codes can be performed, the length of the intermediate representation codes, occupied resources and the like are further reduced, and meanwhile, the running efficiency of the intermediate representation codes can be improved.

In addition, when step S300 is executed, the optimized intermediate representation code may be executed by the execution engine, so that the effects of improving the operation speed, optimizing the memory use, and reducing the communication delay may be achieved according to the optimized intermediate representation code.

In another embodiment of the present disclosure, a visualization effect of the intermediate representation code may also be achieved. FIG. 3 illustrates a flow chart of visualizing intermediate representation code in a data processing method of an embodiment of the present disclosure. The data processing method of the embodiment of the present disclosure may further include:

S401: forming a visualization structure corresponding to the intermediate representation code by using a visualization tool;

s402: and displaying the visual structure.

As described in the above embodiment, after the intermediate representation codes corresponding to the neural network codes are generated, the respective intermediate representation codes may be subjected to the visualization processing, so as to obtain the corresponding visualization structure. Among other things, embodiments of the present disclosure may utilize visualization tools to form visualization structures corresponding to each intermediate representation code. Visualization tools may be used to create dialog boxes or other interfaces to display variables or objects in a manner appropriate to the type of data of the variables or objects. For example, the visualization tool of embodiments of the present disclosure may include a tensorboard. In other embodiments, the visualization of the intermediate representation code may also be implemented by other visualization tools.

After the visualization structure of the intermediate representation code is obtained, the obtained visualization structure can be displayed, and the visualization display of the intermediate representation code is realized.

In addition, as described in the above embodiments, the neural network code of the embodiments of the present disclosure may be a code that realizes an arbitrary function, and the intermediate representation code corresponding to the neural network code may be generated by translating the neural network code. The application of the data processing method of the embodiment of the present disclosure is explained below based on an example in the field of image processing.

Fig. 4 illustrates another flow chart of a data processing method according to an embodiment of the present disclosure. The data processing method comprises the following steps:

s501: acquiring an image to be processed;

the neural network code according to the embodiment of the present disclosure may implement processing operations of an image, for example, processing operations of extracting image features, image deblurring processing, image recognition, and the like, which are not specifically limited in this disclosure. Correspondingly, an image to be processed can be obtained first, and the image to be processed is used as an operand of the neural network code, so that the neural network code of the embodiment of the disclosure is executed, and a processing process of the image to be processed is realized.

S502: performing code translation processing on the neural network code to obtain an intermediate representation code, wherein the intermediate representation code comprises an intermediate representation code corresponding to the neural network code obtained by operating the neural network code by taking the image to be processed as an operand;

according to the embodiment of the disclosure, the acquired image to be processed can be input into the neural network as an operand, namely, the acquired image to be processed is input into the neural network, and the neural network code is executed at the same time, and at the moment, when the neural network code is called, the intermediate representation code corresponding to the function object can be generated through the generator interface. The specific procedure is the same as that of the above embodiment, and a repetitive description thereof will not be given here.

S503: executing the intermediate representation code by an execution engine, wherein the intermediate representation code is executed by the execution engine to obtain a processing result of the image to be processed.

After the corresponding intermediate representation code is generated, the generated intermediate representation code may be stored, for example, stored in a local database or a cloud database, which is not limited.

In addition, the intermediate representation code obtained or stored in step S502 may be executed by the execution engine, thereby obtaining a processing result of the image to be processed.

After the intermediate representation code of each function object of the neural network code is generated, each intermediate representation code may be executed by using the execution engine, and since the embodiments of the present disclosure use the operand corresponding to the image to be processed as the processing object, after the intermediate representation code is executed, the processing result corresponding to the image to be processed may be generated.

In addition, in other embodiments of the present disclosure, the neural network code may be a code of a neural network testing stage or a code of an application stage, which is not specifically limited in the embodiments of the present disclosure.

Fig. 5 shows another flowchart of a data processing method according to an embodiment of the present disclosure, wherein the method further comprises:

S601: determining network loss based on the processing result of the image to be processed;

s602: based on the network loss, network parameters of the neural network are adjusted.

In some possible embodiments, the above-mentioned image to be processed may be used as a training sample to train the neural network corresponding to the neural network code, that is, a plurality of images to be processed may be used as operands, input into the neural network, and convert the neural network code into an intermediate representation code, and execute the intermediate representation code by an execution engine to obtain a prediction result (processing result) corresponding to each image to be processed, determine a loss value of the neural network by using a loss between the processing result and a real result for supervision, stop training of the network when the network loss value is less than a loss threshold, and adjust parameters of the neural network when the network loss value is greater than or equal to the loss threshold until the network loss is less than the loss threshold.

In order to more clearly embody the embodiments of the present disclosure, the translation process of the embodiments of the present disclosure is illustrated below. Fig. 6 shows a schematic diagram of one example of a data processing method according to an embodiment of the present disclosure. The neural network written in Python language is described as an example.

The method comprises the steps of firstly acquiring a neural network code written in a Python language from a code library or a database, then interpreting and executing the neural network code through a Python interpreter, calling an API (interface of a generator) in an IR (intermediate representation code) generator based on the operation of a function object in the neural network code in the interpretation process, calling the interface of the generator based on the IR generator, generating the intermediate representation code corresponding to the called function object, and storing the intermediate representation codes according to the order of generating the intermediate representation codes. And meanwhile, the visualization structure can be obtained by performing visualization processing on each intermediate representation code through a visualization tool. The execution engine may execute the obtained intermediate expression codes to realize the functions. In other embodiments, the intermediate representation code generated by the IR optimization module may be optimized to improve the efficiency of operation of the intermediate representation code, reduce its length, memory space for applications, etc. The visualization process may also be performed on the optimized intermediate representation code and the optimized intermediate representation code may be executed by the execution engine when the visualization process and the intermediate representation code are executed by the execution engine.

In summary, in the embodiments of the present disclosure, when the neural network code corresponding to the neural network code is executed, the intermediate representation code corresponding to the function object in the neural network code may be generated at the same time, where the intermediate representation code abstracts the implementation of different hardware devices for the same operation, that is, abstracting the neural network code, and may be applicable to different types of hardware devices, where the intermediate representation code is a code unrelated to the type of hardware executed by the back end of the neural network, and thus may be applicable to different back ends. The embodiment of the disclosure has the characteristics of simplicity and convenience, and can improve the expandability of the neural network codes.

In addition, the embodiment of the disclosure also provides an image processing method, which can obtain a processing result through the execution of the intermediate representation code corresponding to the neural network code. The image processing method of the embodiment of the present disclosure may be applied to any device having an image processing function, such as a mobile phone, an image capturing device, and an intelligent wearable device, or may also be applied to a server, which is not specifically limited in this disclosure.

Fig. 7 shows a flowchart of an image processing method according to an embodiment of the present disclosure, wherein the image processing method includes:

S10: acquiring an image to be processed;

in the embodiment of the present disclosure, an image may be processed by executing an intermediate representation code, wherein the image to be processed may be acquired first. The image to be processed can be obtained by directly acquiring the image, or can be obtained by reading the stored image, or can be received from other devices to serve as the image to be processed. In other embodiments, the image to be processed may be acquired in other ways, which is not illustrated in one-to-one manner by the present disclosure.

S20: and operating an intermediate representation code corresponding to the neural network code by taking the image to be processed as an operation object to obtain a processing result of the image to be processed, wherein the intermediate representation code is generated by the method according to any one of the embodiments.

As in the above embodiments, the embodiments of the present disclosure may translate the neural network code into the intermediate representation code, and the corresponding neural network code having the image processing function may be translated into the corresponding intermediate representation code.

Correspondingly, the acquired image to be processed can be used as an operation object, and the intermediate representation code is executed, so that a processing result corresponding to the image to be processed is obtained, wherein the processing result can be determined according to the specific function of the neural network. With the above-described embodiment, since the intermediate representation code is in the form of a code that is independent of the backend, it can be applied to various types of image processing apparatuses with better scalability.

It will be appreciated by those skilled in the art that in the above-described method of the specific embodiments, the written order of steps is not meant to imply a strict order of execution but rather should be construed according to the function and possibly inherent logic of the steps.

It will be appreciated that the above-mentioned method embodiments of the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the description of the present disclosure.

In addition, the disclosure further provides a data processing apparatus, an image processing apparatus, an electronic device, a computer readable storage medium, and a program, where the foregoing may be used to implement any one of the image/data processing methods provided in the disclosure, and corresponding technical schemes and descriptions and corresponding descriptions referring to method parts are not repeated.

Fig. 8 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure, as shown in fig. 8, including:

a first acquisition module 10 for acquiring a neural network code;

the generating module 20 is configured to perform code translation processing on the neural network code to obtain an intermediate representation code, where the intermediate representation code abstracts implementation of different hardware devices for the same operation;

An execution module 30 for executing the intermediate representation code by an execution engine.

In some possible embodiments, the generating module includes:

an operation unit for operating the neural network code;

and the generating unit is used for calling a generator interface to generate an intermediate representation code corresponding to the function object in response to the function object in the neural network code being executed.

In some possible implementations, the generating unit is further configured to, in response to a function object in the neural network code being executed, invoke a generator interface corresponding to the function object based on a type of the function object to generate an intermediate representation code corresponding to the function object.

In some possible implementations, the generating unit is further configured to, in response to a situation in which the same function object is repeatedly executed, call the generator interface to generate the intermediate representation code corresponding to the same function object each time the same function object is executed.

In some possible implementations, the generating unit is further configured to run the saved intermediate representation code corresponding to the function object when the function object is repeatedly called,

In some possible embodiments, the execution unit is further configured to interpret and execute the neural network code by an interpreter, and/or to compile and execute the neural network code by a compiler.

In some possible embodiments, the intermediate representation code corresponding to the function object in the neural network includes: and the function object corresponds to the operation name, the operation input and the operation output.

In some possible implementations, the execution module is further configured to invoke, by the execution engine, a calculation module and/or a communication module to execute the intermediate representation code.

In some possible implementations, the apparatus further includes an optimization module for optimizing the intermediate representation code;

the execution module is further configured to execute the optimized intermediate representation code by an execution engine.

In some possible implementations, the optimizing module optimizing the intermediate representation code includes at least one of:

In some possible embodiments, the apparatus further comprises a storage module for storing the intermediate representation codes in order of their generation.

In some possible embodiments, the apparatus further comprises:

and the visualization module is used for forming a visualization structure corresponding to the intermediate representation code by utilizing a visualization tool and displaying the visualization structure.

In some possible embodiments, the first acquiring module is further configured to acquire an image to be processed;

the generating module is also used for operating the neural network code by taking the image to be processed as an operation object to obtain an intermediate representation code corresponding to the neural network code;

The execution module is also used for executing the intermediate representation code by utilizing the execution engine to obtain a processing result of the image to be processed.

In some possible embodiments, the image to be processed is a sample image, and the apparatus further includes a training module for determining a network loss based on a processing result of the image to be processed; and

Fig. 9 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure, wherein the image processing apparatus includes:

a second acquisition module 100 for acquiring an image to be processed;

and the image processing module 200 is configured to run an intermediate representation code corresponding to the neural network code with the image to be processed as an operation object, to obtain a processing result of the image to be processed, where the intermediate representation code is generated by a method according to any one of the first aspect.

In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.

The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a non-volatile computer readable storage medium.

The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the method described above.

The electronic device may be provided as a terminal, server or other form of device.

Fig. 10 shows a block diagram of an electronic device according to an embodiment of the disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 10, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.

Fig. 11 illustrates another block diagram of an electronic device according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 11, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.

The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for performing operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method of data processing, comprising:

acquiring a neural network code, wherein the neural network code is written based on Python;

executing the intermediate representation code by an execution engine;

running the neural network code;

in response to a function object in the neural network code being executed, invoking a generator interface to generate an intermediate representation code corresponding to the function object;

The calling generator interface in response to a function object in the neural network code being executed, comprises:

in response to a function object in the neural network code being executed, invoking a generator interface corresponding to the function object based on a type of the function object to generate an intermediate representation code corresponding to the function object;

the intermediate representation code is used for representing the function object contained in the neural network code in an abstract symbol mode, and the intermediate representation code corresponding to the function object is irrelevant to the context of the function object.

2. The method of claim 1, wherein the generating, by the generator interface, intermediate representation code corresponding to the function object comprises:

3. The method of claim 1, wherein the generating, by the generator interface, intermediate representation code corresponding to the function object comprises:

4. A method according to any of claims 2-3, wherein said running the neural network code comprises: the neural network code is interpreted by an interpreter and/or compiled by a compiler.

5. The method of claim 1, wherein the intermediate representation code corresponding to the function object in the neural network comprises: and the function object corresponds to the operation name, the operation input and the operation output.

6. The method of claim 1, wherein the executing, by an execution engine, the intermediate representation code comprises:

7. The method according to claim 1, wherein the method further comprises:

optimizing the intermediate representation code;

8. The method of claim 7, wherein the optimizing the intermediate representation code comprises at least one of:

9. The method according to any one of claims 1, further comprising:

10. The method according to any one of claims 1, further comprising:

and displaying the visual structure.

11. The method according to any one of claims 1, further comprising:

Acquiring an image to be processed;

12. The method of claim 11, wherein the image to be processed is a sample image, the method further comprising:

13. An image processing method, comprising:

acquiring an image to be processed;

and operating an intermediate representation code corresponding to the neural network code by taking the image to be processed as an operation object to obtain a processing result of the image to be processed, wherein the intermediate representation code is generated by the method as set forth in any one of claims 1-12.

14. A data processing apparatus, comprising:

A first acquisition module for acquiring a neural network code, the neural network code being written based on Python;

the generation module is used for carrying out code translation processing on the neural network codes to obtain intermediate representation codes, and the intermediate representation codes abstract the realization of different hardware devices for the same operation;

an execution module for executing the intermediate representation code by an execution engine;

the generation module comprises:

an operation unit for operating the neural network code;

a generating unit for calling a generator interface to generate an intermediate representation code corresponding to a function object in the neural network code in response to the function object being executed;

the generating unit is further configured to, in response to a function object in the neural network code being executed, invoke a generator interface corresponding to the function object based on a type of the function object to generate an intermediate representation code corresponding to the function object;

15. The apparatus according to claim 14, wherein the generating unit is further configured to, in response to a situation in which the same function object is repeatedly executed, call the generator interface to generate the intermediate representation code corresponding to the same function object each time the same function object is executed.

16. The apparatus according to claim 14, wherein the generating unit is further configured to run the saved intermediate representation code corresponding to the function object when the function object is repeatedly called,

17. The apparatus according to any of claims 14-16, wherein the execution unit is further configured to interpret the execution of the neural network code by an interpreter and/or to compile the execution of the neural network code by a compiler.

18. The apparatus of claim 14, wherein the intermediate representation code corresponding to the function object in the neural network comprises: and the function object corresponds to the operation name, the operation input and the operation output.

19. The apparatus of claim 14, wherein the execution module is further to invoke a calculation module and/or a communication module to execute the intermediate representation code by the execution engine.

20. The apparatus of claim 14, further comprising an optimization module for optimizing the intermediate representation code;

21. The apparatus of claim 20, wherein the optimization module optimizing the intermediate representation code comprises at least one of:

22. The apparatus of claim 14, further comprising a storage module for storing the intermediate representation codes in an order in which they were generated.

23. The apparatus of claim 14, wherein the apparatus further comprises:

24. The apparatus of claim 14, wherein the first acquisition module is further configured to acquire an image to be processed;

25. The apparatus of claim 24, wherein the image to be processed is a sample image, the apparatus further comprising a training module for determining network loss based on a result of processing the image to be processed; and

26. An image processing apparatus, comprising:

the second acquisition module is used for acquiring the image to be processed;

and the image processing module is used for operating an intermediate representation code corresponding to the neural network code by taking the image to be processed as an operation object to obtain a processing result of the image to be processed, wherein the intermediate representation code is generated by the method as claimed in any one of claims 1 to 12.

27. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to: performing the method of any one of claims 1 to 12 or performing the method of claim 13.

28. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 12 or the method of claim 13.