WO2012062081A1 - Method for online updating java classes based on reconstruction of byte-code files - Google Patents

Abstract

A method for online updating Java classes based on the reconstruction of byte-code files is provided. The method analyzes and reconstructs the binary files when it performs dynamic updates on the Java programs, combining a new mechanism of dynamically loading the classes by a virtual machine (VM), to reduce the costs brought by the dynamic updates recompiling the machine code on running time, and improve the update efficiency.

Description

 A Java class online update method based on bytecode file reconstruction

Technical field

 The invention belongs to the field of computer applications, and is specifically an efficient Java class online updating method based on bytecode file reconstruction. Background technique

 Software is the product of human brain work. There is no perfect software in the world, and it is difficult for developers to complete a software system that does not require improvement. Software always requires constant maintenance by developers, such as bug fixes, efficiency improvements, feature enhancements, and code refactoring.

 Traditional software updates require the termination of running software, and in some cases, especially the more important software systems, the cost of terminating the running software is high. To this end, an online software update technology is very important. A software update system based on JVM (Java Virtual Machine) has high research value due to the easy-to-modify features of virtual machines.

 To run a Java program on a machine, you need to write the source code, compile the source code, and start the JVM to load the class file to run. The Java language is mainly composed of classes, and the classes are mainly composed of domain and method definitions. After the JVM loads the class file, the JVM will map the read bytecode file to the corresponding runtime metadata. This is the first layer of processing of the binary at runtime. When a method needs to be executed, the JVM calls his online compiler to compile the bytecode into the corresponding machine code, and the CPU reads these machine instructions to perform the corresponding functions. The JVM creates an object in the memory allocated heap area. The type information corresponding to the object is related to the class metadata. The information mainly describes the domain corresponding to each offset of the object space, and the instance method that the object can run.

 The dynamic update system defines a basic semantic for maintaining consistency of updates: new code is not run before the update, and the old code is not run after the update. In addition to this, basic type safety is also guaranteed during the update process.

 Dynamic update systems generally require an update preparation step, mainly by analyzing changes made to the new and old versions of the program and outputting the information to a file, and the programmer may need to write some update patches.

 According to the changes made by the new and old versions of the Java class, the Java classes that need to be processed are mainly divided into the following three types in the analysis phase:

1. When the class's parent class, interface collection, domain collection, and method collection change, the class needs to be reloaded into the virtual machine using the JVM's dynamic loading mechanism, and the class definition and domain definition and method definition are mapped to corresponding operations. Time metadata. An update to this type of definition change is called a class update.

 2. When the class definition has not changed, only the implementation of the method changes. In this case, you only need to update the VM metadata corresponding to the method. This class definition is unchanged but there is a method bytecode change update called method body update.

3. When there is no change in the class definition and method bytecode, but because there is a class that references the class update class, you need to update the machine code of the corresponding method. This bytecode has not changed, but a method update that references a class update class is called a reference method update. Different updates cause the runtime metadata to change, and dynamic updates replace the runtime metadata. Different updates to the above analysis require different approaches when running online updates:

 1. Class update:

 The class update causes the runtime class metadata to be regenerated, and at the same time the method bytecode changes, and the method machine code changes. Due to the uniqueness of the type, replacing the new class requires removing the old class definition from the runtime type management mechanism. Specifically:

 1 Remove old class metadata from running system 2 Dynamically load new class 3 Generate new class metadata, domain metadata, method metadata 4 Compilation method

 2. Method body update:

 The method body update causes the bytecode of the method to change, and at the same time the bytecode of the method changes. Specifically:

 1 Replacement method bytecode part in metadata 2 Recompilation method

 3. Reference method update:

 The operand of some of the machine code in the reference method has changed. A simple solution at this time is to recompile the whole method, but this is obviously not the best method. Specifically:

 1 Recompile the method.

 The update requires the program to run to a secure point, and the security point needs to be able to stop all user threads and scan the user thread, determining that there are no restricted methods in the thread's run stack. The method of changing the machine code is a restricted method. In addition, the user can also specify the restricted method according to the program logic. When a method is removed, these methods are also marked as restricted methods to ensure the semantics of the update.

 The machine code that references the method update needs to be fixed. The general practice is to recompile. The characteristics of this type of method: First, the bytecode has not changed, and the second is to reference the class update class. Although its bytecode has not changed, the referenced methods are also present in the new and old versions, but because the dynamic class loading mechanism of the JVM will assign the metadata corresponding to the new class to the new offset, this leads to the reference method update. The machine code of the method needs to be recompiled. If you can ensure that this offset update does not change before and after, the recompilation caused by the reference method update can be eliminated, improving the efficiency of the update. This paper proposes a method to combine and analyze and reconstruct the bytecode binary file to adjust the dynamic loading class mechanism at runtime to reduce the recompilation caused by the update of the reference method. Summary of the invention

 The technical problem to be solved by the present invention is to provide a Java class online update method based on bytecode file reconstruction, which reduces the change of runtime metadata by reducing the dynamic loading class mechanism at runtime, and reduces the reference method. The recompilation brought by the update improves the efficiency of the update.

 The invention relates to a Java class online updating method based on bytecode file reconstruction, which mainly comprises the following steps:

1) Analyze the binary bytecode file in the old and new versions of the program, and find out the class update, class method body update, and class reference method update. Class

 2) Find suitable update points at runtime according to the analysis results;

 3) At runtime, according to the update of the class, the corresponding update operation is performed, specifically:

 3.1) For classes that are updated by the class, replace the old class with the old one;

 3.2) For the class updated by the class method body, re-read and initialize the methods defined by it;

 3.3) For classes whose class reference method is updated, recompile the method with the corresponding reference;

 4) Convert the class instance of the type change in the heap to a new type instance.

 The process of step 1) above is:

 The JVM-based update analysis tool traverses all classes twice,

 First traversal:

 1.1) According to the situation of the class in the old version in the new version, classify the old version class, mark the classes that exist in the reduced class and the old and new versions; further mark the class definition changes for the classes that exist in the new and old versions. And classes whose class definitions do not change;

 1.2) According to the situation in the new version of the methods in the class in the old version, the methods of reducing, the methods that exist but the method body changes, and the methods that exist and the method body are unchanged are marked;

 The second traversal:

 1.3) For those methods that exist in the old and new versions and whose method body remains the same, mark the method that references the class whose class definition changed; Mark the output:

 1.4) Reduce the class in step 1.1), all the methods in the class whose class definition changes, the method in which the method body in the class definition is unchanged, the method whose method body is unchanged but the class that references the class definition change is marked as Restricted method

 15) In the restricted method marked in step 1.4), the bytecode invariant method is marked as a restricted method with the bytecode unchanged.

 The specific steps of the above step 1.1) markup class are: Take a class C in the old version, and judge whether it has the same C class in the new version, if it does not exist, mark c as a reduced class; if it exists, further compare Whether the definition of the class has changed, and the steps to determine whether the class definition has changed are:

 1.1.1) Determine whether the basic information of the class in the binary file has changed. If the parent class is a reduced class, mark the class inheritance relationship change;

 1.1.2) Determining and marking whether the domain existence domain can remain unchanged at runtime, and calculating the offset of the instance domain in the heap object instance space according to the layout of the instance domain of the corresponding VM platform, the offset and the domain Appear the order in the binary file, reconstruct the binary bytecode file generated by the compiler, and adjust the order in which the fields appear;

 1.1.3) After adjusting the layout in 1.1.2), judge and mark if the instance size of the class has changed if necessary;

1.1.4) Judging and Marking Method Existence Method Whether the offset can be kept unchanged at runtime, the offset is calculated according to the corresponding VM platform instance method offset algorithm, and the offset is related to the order in which the method appears in the binary file. The binary bytes generated by the compiler The code file is reconstructed, and the order in which the methods appear is adjusted;

 1.1.5) If one of the above changes, it is determined that the definition of the class changes; otherwise, the class definition is unchanged, and the adjusted bytecode is written into the file.

 The detailed steps of adjusting the above steps 1丄 2) and determining whether the domain offset has changed are:

 Prerequisites: If you find that filling the gaps during the adjustment process increases the size of the object instance, stop the adjustment.

 a) Layout the old class domain and sort the fields by offset;

 b) sequentially fetch a field f from the sorted list, and continue to fetch the next domain if there is no identical domain f' in the new class; c) otherwise, if it can destroy the premise without destroying the premise Domain f allocates the same offset of f, puts it into the layout context, and continues from step b);

 d) Otherwise, mark the domain displacement change and place it in the layout context from step b) to continue.

 The detailed steps of adjusting the above steps 1丄 4) and determining whether the method offset has changed are:

 a) First determine the offset of the new and old instance methods in the method table, and get the method table mtable and mtable';

 b) Take a method m in the old class method table mtable. If the method m' offset in the new class is the same, ie m.offset==m'.offset, continue to take the next method in the old class;

c) Otherwise, if the offset m.off _Se t of the old class method in the old class method table belongs to the range of the parent class inheritance method in the new class method table, mark the method offset change;

d) Otherwise, if the offset m.off _Se t of the old class method in the old class method table belongs to the scope of the class definition method, then the method offset in the new class and the offset method in the new class method table Offset swap, ie

 Mtable' [m' .offset] = mtable' [m.offset];

 Mtable' [m' .offset] .offset = m, .offset;

 Mtable' [m.offset] = m,;

 m, .offset=m.offset;

 e) Otherwise, if the offset of the old class method in the old class method table exceeds the length of the new class method table, the new class method table size is expanded, and the new class method table is offset by the same method.

 Extend(mtable');

 Mtable' [m' .offset] = null;

 Mtable' [m.offset] = m,;

 M'.offset=m. offset

 In the above step 1.2), the specific step of the method of marking the class existence class is to take a method in the class existence class to determine whether it has the same method in the new version class:

1.2.1) If it does not exist, mark the method as a reduced method; 1.2.2) If there is a method, the method for judging the method existence method to further determine whether the method body is changed is:

 a) The instruction to take the old and new version methods insnOld, insnNew;

 b) if insnOld and insnNew are not the same instruction, it is determined that the method body has changed;

 c) If insnOlde and insnNew are the same instruction, if they do not refer to Constant Pool, compare the fields of the instruction according to the instruction format; if the Constant Pool is referenced, take the contents of the Constant Pool for comparison; if the comparison result is different, Then it is determined that the method body changes.

 In step 1.3), the specific steps of the tag reference structure change class are:

 1.3.1) Take an instruction,

 1.3.2) If the instruction is new/anewarray/multianewarray, if the instance field size of the target class it accesses is determined to change in step 1.1.3 above, then the method of marking the method as a structural change class is used. ;

 1.3.3) If the instruction is instanceof/checkcast, if the inheritance structure of the target class it accesses is determined to change in the above step 1), then the method is marked as a method of using the structural change class;

 1.3.4) If the instruction is one of getfield/putfield/getstatic/putstatic/invokevirtual/invokespecial, if the target method of the access, the offset of the target domain is determined to occur in steps 1.1.2) and 1.1.4) above To change, mark the method as a method of referencing a structure change class.

 The detailed process of step 3.1) is:

 3.1.1) Remove the old version of the class;

 3.1.2) Modify the path of the ClassLoader to find the bytecode data stream corresponding to the new class;

 3.1.3) Load the new class through the dynamic class loading mechanism. After loading the class, replace the VM metadata attribute of the old class with the new class, modify the type Id in the type system, and assign the old class Id to the new class.

 3.1.4) Modify the offset of the static field and method in the static data table, and replace the data at the corresponding position with the pointer corresponding to the new class; the offset of the instance field and the instance method adjust the bytecode binary file through analysis;

 3.1.5) After the adjustment, perform subsequent operations such as initialization and instantiation.

 The invention combines and analyzes a binary file in a dynamic update of a Java program, and combines a new VM dynamic loading class mechanism to reduce the overhead of recompilation of machine code by dynamic update at runtime. , improve the efficiency of the update. DRAWINGS

 Figure 1 shows the different update operations based on the updates that occur in the Java class.

 Figure 2 shows the class-level tagging process of the analysis tool of the present invention,

 FIG. 3 shows a flow of analyzing and marking the class structure of the analysis tool of the present invention,

Figure 4 shows the flow of analysis and labeling of the method body by the analysis tool of the present invention, Figure 5 shows the flow of the analysis tool of the present invention to find a reference update method.

 6 is a content of a class in a different version of a program before and after a Java program update in an embodiment.

 Figure 7 is a table of the methods of classes A and B in Figure 6,

 Figure 8 is a bytecode of the class B method m3 of Figure 6. detailed description

 The invention is further described below in conjunction with specific embodiments.

 Figure 6 shows the contents of the classes in different versions of the program before and after a Java program update. The figure on the left is the class in the version 1 program, and the one on the right is the class in the version 2 program.

 In the analysis stage, the flow shown in Figure 2 is used. First, the version one class is topologically sorted according to the inheritance relationship, and then the order of comparison is A B C D E.

 According to the flow of Fig. 2, the A is judged to have the same class A' in the version 2, and then it is judged whether the class structure of the class A has changed. According to the flow of Figure 3, before and after the parent class update of A, the Object class has not changed, the class A domain has not changed, and the object size has not changed. The class A has added methods, so the class structure changes. By further analysis and comparison, it can be judged that the offset of the class A method ml can be kept unchanged. Class A is marked as a class with a class but a structural change. Then, according to the flow of Figure 4, the method implementation in A is compared. However, since the implementation of the method in A is not given here, the flow of the comparison method of Figure 4 will be compared in class C. Time to elaborate.

 According to the flow of Fig. 2, after comparing A, the class B is taken out for analysis, and it is found that the same class B' exists in the second version, and then it is judged whether the structure of the class B changes. According to the flow of Figure 3, the parent class A of class B is marked as a class but the structure changes, and B is inevitably marked as a class but the structure changes. According to the flow of Figure 3, the details of B change are continuously determined. It is found that the new instance domain j is added to B, and the static domain s is reduced. The instance domain i is unchanged before and after the update, and the offset can be kept unchanged by the adjustment. After that, the size of the class instance is determined. Since the static domain is reduced and the instance domain is increased, the size of the class B instance is increased. According to the flow of Figure 3, the method offset of class B is continuously determined. It is found that class B removes the implementation of method m2 and overwrites the method ml in parent class A, and the method m3 remains unchanged. The offset of the method in class B can be maintained, as shown in Figure 7. Next, the implementation of the method in A will be compared according to the flow of FIG. 4, and the bytecode of the method m3 in B is as shown in FIG. 8. According to the flow of Figure 4, the first two instructions of m3 can be judged as the same instruction by opcode. The third instruction has operands and the same, but since the operand of the instruction refers to the symbol in ConstantPool, it is necessary to take out the symbol for comparison and find that it is different. Therefore, it can be judged that the method m3 has changed before and after the update.

 According to the flow of Figure 2, after class B is compared, the class C is extracted for analysis, and it is found that the same class C' exists in version 2, and then it is determined whether the structure of class C has changed. According to the flow of Figure 3, the parent class, domain, and method of class C have not changed, so class C is marked as a class and the structure has not changed. Continue to compare the method of class C. According to the flow of Figure 4, it can also be judged that the ml method has changed, and the method ml in class C is marked as a restricted method.

According to the flow of Figure 2, after class C is compared, class D is taken for analysis, and it is found that the same class D' exists in version two, and then the class is judged. Whether the structure of C has changed. Analysis can be found that the D class is completely consistent before and after the update, but the D class refers to the class B. In the subsequent second traversal, according to the flow of Figure 5, find out the method that needs to be recompiled. According to the flow of Figure 2, after class D is compared, class E is taken for analysis, and it is found that class E does not exist in version two, so all methods in class E are marked as restricted methods.

 According to the flow of Figure 2, the first traversal is completed after class E is compared. Then a second traversal to find out which method needs to be recompiled. According to the flow of Fig. 5, when analyzing the method m in D, it is found that the method has no change in the displacements of the methods ml, m2, and m3 in B. However, when analyzing the initialization method of D, it is found that the size of the object of B changes, so the constructor needs to be recompiled, and the initialization method is marked as a constrained method of 0SR.

 After all the analysis is complete, we need to reconstruct the binary bytecode files of class A and class B to ensure that the displacement of the corresponding method and domain is unchanged when loading at runtime.

 When the runtime is updated, the program is loaded and updated according to the flow of Figure 1. A and class B need to reload the class and generate new VM metadata, and compile new methods; class C needs to recompile method ml; class D needs to recompile its initialization method; class E removes it, This can be done with the JVM's garbage collection management mechanism.

 Our static analysis tools determine the method tables for all classes during the analysis phase, and try to ensure that the method displacements that exist before and after the update are unchanged through a series of adjustments, and the results are fed back into the original binary. With our static analysis tools, you can effectively reduce the number of methods that change bytecode but machine code needs to be recompiled.

Claims

Rights request 1. A Java class online update method based on bytecode file reconstruction, which is characterized by the following steps:  1) Analyze the binary bytecode file in the old and new versions of the program, and find out the class for which the class update, the class method body update, and the class reference method update are targeted;  2) Find suitable update points at runtime according to the analysis results;  3) At runtime, according to the update of the class, the corresponding update operation is performed, specifically:  3.1) For classes that are updated by the class, replace the old class with the old one;  3.2) For the class updated by the class method body, re-read and initialize the methods defined by it;  3.3) For classes whose class reference method is updated, recompile the method with the corresponding reference;  4) Convert the class instance of the type change in the heap to a new type instance.  2. The Java class online update method based on bytecode file reconstruction according to claim 1, wherein the process of the above step 1) is:  The JVM-based update analysis tool traverses all classes twice,  First traversal:  1.1) According to the situation of the class in the old version in the new version, classify the old version class, mark the classes that exist in the reduced class and the old and new versions; further mark the class definition changes for the classes that exist in the new and old versions. And classes whose class definitions do not change;  1.2) According to the situation in the new version of the methods in the class in the old version, the methods of reducing, the methods that exist but the method body changes, and the methods that exist and the method body are unchanged are marked;  The second traversal:  1.3) For those methods that exist in the old and new versions and whose method body remains the same, mark the method that references the class whose class definition changed; Mark the output:  1.4) Reduce the class in step 11), all methods in the class whose class definition changes, the method in which the method body in the class definition is changed, the method body whose method body is unchanged but the class whose class definition is changed is marked as Restricted method  1.5) In the restricted method marked in step 14), the bytecode invariant method is marked as a restricted method with the bytecode unchanged.  3. The Java class online update method based on bytecode file reconstruction according to claim 2, wherein the step 1.1) the specific step of marking the class is: taking a class C in the old version, and determining that it is in the new version. Whether the same C class exists in the middle, if it does not exist, the mark C is a reduced class; if it exists, further compare whether the definition of the class changes, and the step of determining whether the class definition changes: 1.1.1) Determine whether the basic information of the class in the binary file has changed. If the parent class is a reduced class, mark the class inheritance relationship change; 1.1.2) Determining and marking whether the domain existence domain can remain unchanged at runtime, and calculating the offset of the instance domain in the heap object instance space according to the layout of the instance domain of the corresponding VM platform, the offset and the domain Appear the order in the binary file, reconstruct the binary bytecode file generated by the compiler, and adjust the order in which the fields appear;  1.1.3) After adjusting the layout in 1.1.2), judge and mark if the instance size of the class has changed if necessary;  1.1.4) Judging and Marking Method Existence Method Whether the offset can be kept unchanged at runtime, the offset is calculated according to the corresponding VM platform instance method offset algorithm, and the offset is related to the order in which the method appears in the binary file. Refactoring the binary bytecode file generated by the compiler, and adjusting the order in which the methods appear;  1.1.5) If one of the above changes, it is determined that the definition of the class changes; otherwise, the class definition is unchanged, and the adjusted bytecode is written into the file.  4. The Java class online update method based on bytecode file reconstruction according to claim 3, wherein the detailed steps of adjusting 1.1.2) and determining whether the domain offset has changed are:  Prerequisites: If the gap is found to increase the size of the object instance during the adjustment process, the adjustment is stopped.  a) Layout the old class domain and sort the fields by offset;  b) sequentially fetch a field f from the sorted list, and continue to fetch the next domain if there is no identical domain f' in the new class; c) otherwise, if it can destroy the premise without destroying the premise Domain f allocates the same offset of f, puts it into the layout context, and continues from step b);  d) Otherwise, mark the domain displacement change and place it in the layout context from step b) to continue.  5. The Java class online update method based on bytecode file reconstruction according to claim 3, wherein the detailed steps of adjusting the above step 1.1.4 and determining whether the method offset is changed are:  a) First determine the offset of the new and old instance methods in the method table, and get the method table mtable and mtable';  b) Take a method m in the old class method table mtable. If the method m' offset in the new class is the same, ie m.offset==m'.offset, continue to take the next method in the old class; c) Otherwise, if the offset m.off _Se t of the old class method in the old class method table belongs to the range of the parent class inheritance method in the new class method table, mark the method offset change; d) Otherwise, if the offset m.off _Se t of the old class method in the old class method table belongs to the scope of the class definition method, then the method offset in the new class and the offset method in the new class method table Offset swap, ie  Mtable' [m' .offset] = mtable' [m. offset];  Mtable' [m' .offset] .offset = m, .offset;  Mtable, [m. offset] = m,;  m, .offset=m.offset; e) Otherwise, if the offset of the old class method in the old class method table exceeds the length of the new class method table, then expand the new class method table size, Offset the new class method table to the same method  Extend(mtable');  Ratable' [m' .offset] = null;  Ratable' [m.offset] = m,;  M'.offset=m. offset o  The method for updating a Java class online based on bytecode file reconstruction according to claim 2, wherein in the step 1.2), the specific step of the method for marking the class existence class is to take a method in the class existence class. , to determine whether it exists in the new version of the class: 1.2.1) If it does not exist, mark the method as a reduced method;  1.2.2) If there is a method, the method for judging the method existence method to further determine whether the method body is changed is:  a) The instruction to take the old and new version methods insnOld, insnNew;  b) If insnOld and insnNew are not the same instruction, it is determined that the method body has changed;  c) If insnOlde and insnNew are the same instruction, if they do not refer to Constant Pool, compare the fields of the instruction according to the instruction format; if the Constant Pool is referenced, take the contents of the Constant Pool for comparison; if the comparison result is different, Then it is determined that the method body changes.  7. The Java class online update method based on bytecode file reconstruction according to claim 2, wherein in step 1.3), the specific steps of the tag reference structure change class are:  1.3.1) Take an instruction,  1.3.2) If the instruction is new/anewarray/multianewarray, if the instance field size of the target class it accesses is determined to change in step 1.1.3 above, then the method of marking the method as a structural change class for the bow I is marked. ;  1.3.3) If the instruction is instanceof/checkcast, if the inheritance structure of the target class it accesses is determined to change in step 1.1.1) above, then the method is marked as a method of using the structural change class;  1.3.4) If the instruction is one of getfield/putfield/getstatic/putstatic/invokevirtual/invokespecial, if the target method of accessing, the offset of the target domain is determined to occur in steps 1丄2) and 1.1.4) above To change, mark the method as a method of referencing a structure change class.  8. The Java class online update method based on bytecode file reconstruction according to claim 1, wherein the detailed process of step 3.1) is:  3.1.1) Remove the old version of the class;  3.1.2) Modify the path of the ClassLoader to find the bytecode data stream corresponding to the new class;  3.1.3) Load the new class through the dynamic class loading mechanism. After loading the class, replace the VM metadata attribute of the old class with the new class, modify the type Id in the type system, and assign the old class Id to the new class. 3.1.4) Modify the offset of the static domain and method in the static data table, and replace the data at the corresponding location with the pointer corresponding to the new class; The offset of the instance method is adjusted by adjusting the bytecode binary file; 3.1.5) After the adjustment, the initialization, instantiation and other subsequent operations of the class are performed.