CN111459799B - Software defect detection model establishing and detecting method and system based on Github - Google Patents

Abstract

The invention discloses a software defect detection model establishment, detection method and system based on Github, wherein the establishment of the detection model includes: firstly, preprocessing the data set in the Github platform to obtain the change record and the corresponding Bug that meet the requirements Fix file pairs; then process the change records that meet the requirements to generate slice vectors and labels; finally input the slice vectors and labels into the bidirectional LSTM model for training and learning, and obtain a trained detection model. For the target file to be detected, the vector of the target file is processed and input into the detection model to obtain the detection result. The method of the invention solves the problems of unbalanced data, insufficient data diversity and poor model generalization ability faced by the current defect detection based on source code learning because the data set is too small, and can achieve higher detection accuracy.

Description

A Github-based software defect detection model establishment, detection method and system

technical field

The invention belongs to the technical field of code auditing, and in particular relates to a Github-based software defect detection model establishment, detection method and system.

Background technique

A variety of defect analysis tools already exist in the field of code auditing, which attempt to detect common defects in software. Static detection tools such as Clang can do this without executing the program. Dynamic inspection tools detect defects by repeatedly executing many test cases on real or virtual processors. Both static and dynamic detection tools are based on manual definition of defect rules, so they are limited to manual design rules and cannot guarantee complete testing of the code base. Symbolic execution replaces input data with symbolic values, and performs analysis and diagnosis on the control flow diagram of the program. Although it can probe all feasible program paths, symbolic execution is expensive and does not scale well to large programs. In addition to these traditional tools, there has been a considerable amount of recent work on program analysis using machine learning. Numerous open source platforms such as Github provide opportunities to learn patterns of software defects directly from mined data.

At present, there are many technologies that use machine learning and deep learning to realize defect detection in the field of defect detection. After Hovsepyan et al. represented the Java source code through the bag of words model, they used the support vector machine (SVM) to predict the label of the source code. However, their work is limited to training and evaluation on a single software repository. Mou et al. implemented defect detection by embedding words into nodes in the abstract syntax tree of source code and training a tree-based convolutional neural network. This work explored the potential of deep learning in program analysis. Kapur et al. used 8 machine learning methods to learn the characteristics of defects and give the possibility of defects in source code files. Li et al. utilized a recurrent neural network (RNN) to train code snippets related to library/API function calls to detect two types of defects related to these API calls inappropriately.

The limited datasets (both limited in size and variety) used by most of the previous work limited the utility of the results and prevented them from exploiting the full power of deep learning, such as the related work mentioned above in PROMISE, DEFECT4J, NVD, SARD These data sets contain less than 8 open source warehouses at most, and are updated slowly, with few types of defects and low defect coverage, which makes the existing models unable to detect complex and diverse defects. . At present, there is no related work that directly mines data in large-scale open source code bases, and learns defect features from them to achieve detection purposes. The main reason is that collecting data on large open source code bases such as Github faces a very high false positive rate. affect the effectiveness of the model.

Contents of the invention

In order to solve the deficiencies in the prior art, the present invention provides a Github-based software defect detection model establishment, detection method and system, which solves the problem of slow update of the vulnerability database in the existing method, few types of defects, and low defect coverage, which lead to defects Static detection technology must face the problems of unbalanced samples, models that cannot be used to detect complex and diverse defects, and models that fail quickly.

In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve:

A method for building a software defect detection model based on Github, comprising the following steps:

Step 1, data preprocessing:

Sort the warehouses in Github, select the top-ranked warehouses as the source data set, filter and deduplicate the change records of each warehouse in the source data set, and obtain the required change records and their corresponding Bug-Fix files right;

Step 2, process the change records that meet the requirements obtained in step 1 and their corresponding Bug-Fix file pairs, and generate slice vectors and labels:

Step 2.1, analyze and compare the Bug-Fix file pairs of the change records that meet the requirements obtained in step 1, and obtain the added and deleted code line information corresponding to each change record;

Step 2.2, according to the added and deleted code line information obtained in step 2.1 and the data flow of the Bug-Fix file pair, determine the defective code line that causes the defect for each change record;

Step 2.3, according to the data flow and control flow information of the Bug-Fix file pair of the change record that meets the requirements obtained in step 1, generate slices of the Bug-Fix file pair;

Step 2.4, according to the defect code line determined in step 2.2, add labels to the slices of the Bug-Fix file pair obtained in step 2.3;

Step 2.5, replace the variable names one by one with the slices of the Bug-Fix file pairs obtained in step 2.3, and replace them with a unified token;

In step 2.6, word segmentation is performed on the slices whose variable names have been replaced in step 2.5, and a token sequence is obtained for each slice;

Step 2.7, converting each token sequence obtained in step 2.6 into a vector;

Step 3, model training:

Input the slice vector and label obtained in step 2 into the bidirectional LSTM model for training and learning, and obtain a trained detection model.

Specifically, the step 1 specifically includes:

Step 1.1, sort the warehouses in Github according to the impact factor fork of the warehouses from large to small, and select the top 30% to 35% warehouses as the source data set;

Step 1.2, use the keyword search method to filter the change records of each warehouse in the source data set, and obtain the change records with different defect types;

Step 1.3, filter out the change records that do not meet the verb/direct object pattern among the change records with different defect types obtained in step 1.2;

Step 1.4, use TextRank technology to re-screen the change records obtained in step 1.3, and eliminate mixed change records;

In step 1.5, deduplicate the change records obtained in step 1.4 to obtain the change records that meet the requirements and their corresponding Bug-Fix file pairs.

Specifically, in step 2.4, in the labeling process, the slice containing the defective code line is marked as 0, and the slice not containing the defective code line is marked as 1.

Specifically, in the step 2.7, the word2vec tool is used to convert each token sequence into a vector.

The invention also discloses a Github-based software defect detection model building system, including the following modules:

The data preprocessing module is used to sort the warehouses in Github, select the top-ranked warehouses as the source data set, filter and deduplicate the change records of each warehouse in the source data set, and obtain the change records that meet the requirements and Its corresponding Bug-Fix file pair;

Data extraction modules, including:

The added and deleted code line information acquisition module is used to analyze and compare the Bug-Fix file pairs of the change records that meet the requirements obtained by the data preprocessing module, and obtain the added and deleted code line information corresponding to each change record ;

The defective code line acquisition module is used to determine the defective code line that each change record causes a defect according to the added and deleted code line information obtained by the added and deleted code line information acquisition module and the data flow of the Bug-Fix file pair;

The slice generation module is used to generate slices of the Bug-Fix file pair according to the data flow and control flow information of the Bug-Fix file pair of the change record that meets the requirements obtained by the data preprocessing module;

Add labeling module, be used for according to the defective code row determined by defect code row obtaining module, add label to the slice of the Bug-Fix file pair that slice generation module obtains;

The variable name replacement module is used to replace the variable names one by one for the slices of the Bug-Fix file pairs obtained by the slice generation module, and replace them with a unified token;

The word segmentation processing module is used to perform word segmentation processing on the slices whose variable names have been replaced by the step variable name replacement module, and each slice obtains a token sequence;

The vector conversion module is used to convert each token sequence obtained by the word segmentation processing module into a vector;

The model training module is used to input the vector and label of the slice obtained by the data extraction module into the bidirectional LSTM model for training and learning, and obtain a trained detection model.

Specifically, the data preprocessing includes:

The source data set acquisition module is used to sort the warehouses in Github according to the impact factor fork of the warehouses from large to small, and select the warehouses with the top 30% to 35% as the source data sets;

A screening module, which is used to filter the change records of each warehouse in the source data set by using a keyword search method to obtain change records with different defect types;

The secondary screening module is used to filter out the change records that do not conform to the verb/direct object mode among the change records with different defect types obtained by the primary screening module;

The third screening module is used to re-screen the change records obtained by the secondary screening module by using TextRank technology, and eliminate the mixed change records;

The deduplication module is used to deduplicate the change records obtained by the three-time screening module to obtain the required change records and their corresponding Bug-Fix file pairs.

Specifically, in the labeling module, during the labeling process, the slice containing the defective code line is marked as 0, and the slice not containing the defective code line is marked as 1.

Specifically, in the vector conversion module, a word2vec tool is used to convert each token sequence into a vector.

The present invention also discloses a Github-based software defect detection method, comprising the following steps:

Step 1, for the target file to be detected, process according to the above steps 2.3, 2.6 and step 2.7 to obtain the vector of the target file;

Step 2, input the vector obtained in step 1 into the detection model to obtain the detection result.

The present invention also discloses a software defect detection system based on Github, comprising:

The data processing module is used to process the target file to be detected according to the above steps 2.3, 2.6 and 2.7 to obtain the vector of the target file;

The detection module is used to input the vector obtained by the data processing module into the detection model to obtain the detection result.

Compared with prior art, the beneficial effect of the present invention is:

(1) The present invention proposes a method that can directly obtain data sets on Github, and the false positive rate is low, and the finally obtained data sets are far larger than the current related work data sets. It solves the problems that the current defect detection based on source code learning is too small and must face data imbalance, insufficient data diversity, and poor model generalization ability.

(2) The present invention utilizes the slicing method to realize defect location with a smaller granularity, and solves the problem that most of the related work detection defects can only give an approximate location, such as a package or a file.

(3) When slicing data, the present invention integrates the data flow and control flow features of the source code, and better captures the execution order information of the source code, so that the model can learn more code features, thereby achieving higher Detection accuracy.

Description of drawings

Fig. 1 is the flow chart of model establishment and detection described in the embodiment.

Figure 2 is the commit obtained in step 2.1 of the embodiment and the corresponding Bug-Fix file.

Fig. 3 is the slice extracted from Bug File in step 2.3 of the embodiment.

Fig. 4 is the slice extracted from Fixed File in step 2.3 of the embodiment.

Fig. 5 is the variable name replacement of slice in step 2.5 of the embodiment.

Fig. 6 is the slice word segmentation process in step 2.6 of the embodiment.

Fig. 7 is a schematic structural diagram of the bidirectional LSTM neural network in the embodiment.

Fig. 8 is a flow chart of data preprocessing described in step 1 in the embodiment.

Detailed ways

"Slice" refers to a multi-line code fragment obtained by cutting the code according to certain rules. The cutting method can follow data flow or control flow and other custom methods. In the present invention, the segmentation is based on two parts of information, data flow and control flow, with the purpose of extracting multi-line codes with semantic relevance and highlighting defect information.

"Change record" refers to every time the warehouse manager on Github changes, corrects and adds codes in the warehouse, it is called the change record of the warehouse, that is, commit.

"Mixed change records" means that a commit contains many reasons for modification, and it is impossible to intuitively see which part of the code is modified for what reason.

"Verb/direct object mode" means that the change record description in the change record conforms to the verb + direct object rule, such as the change record description "fix a bug", which has a subordinate relationship. In Stanford CoreNLP, "bug" is called "dobj " is the direct object, among which "fix" is called governor, that is, a verb, subordinate to "bug".

"Token" refers to the characters generated after cutting the source code, and is used to represent variables, keywords, special characters, etc. in the source code. For example, the cut tokens of "if(str==null)" are "if", "(", "str", "==", "null", ")".

"Token sequence" means a series of token collections in a specific order. For example, the sequence of tokens formed by "if (str==null)" according to the order of appearance of the source code is ["if", "(", "str", "==", "null", ")"].

Each qualified change record obtained after data preprocessing in the present invention includes a Bug-Fix file pair.

The present invention does not depend on a specific programming language. For the convenience of explanation, the present invention uses the open source code library Github and the Java language as examples to introduce the specific details of the present invention in detail. The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

A method for establishing a software defect detection model based on Github disclosed in this embodiment comprises the following steps:

Step 1, data acquisition and its preprocessing

Github is currently the largest and most influential open source project hosting platform. Each warehouse on Github has its warehouse name, such as repository_1, and every time the warehouse manager makes changes, corrections, and additions to the code in the warehouse, it is called the change record of the warehouse, that is, commit. Among them, for each commit, the platform will require the submitter to add an explanatory description of this commit, the purpose is to introduce the modification intention this time, and facilitate the iterative development and management of the software, such as "fix a bug". The description is called a commit message. Therefore, we can filter the commit message to find the commit that meets the requirements. For each commit, the platform also has a file modification record corresponding to this commit, that is, a Bug-Fix file pair, which are respectively the Bugfile before modification and the Fixed file after modification.

At present, there are many research works using historical modification information on Github. For example, Zhou et al. directly filter commits according to keyword search or definition rules, but related work points out that this method has a high false positive rate, and Commits have a lot of low-quality problems. The low-quality problems mainly include the following three kinds of commits that do not meet the research intent, but are included in the search set. (1) "how" type commits do not clearly indicate that this change is for What is the change, that is, "why", but simply pointing out how and where the change was changed, that is, "how"; (2) Hybrid commit, that is, a commit contains many reasons for modification, which cannot be intuitive You can see which part of the code is modified for what reason. (3) ID-type commit, that is, a commit does not have any description, only the ID of the event.

In order to solve this problem, we propose a general method to obtain data from Github, that is, a commit screening method based on V-DO (verb/direct object) filtering and TextRank keyword extraction. First, rank the warehouses to obtain the commits in warehouses with relatively high quality; then search the commits according to the defect type according to the traditional method. In this step, ID-type commits will not appear in the data set because they do not match the keyword search; In the part of commit screening, we did the following work (1) Screening the commit message that conforms to the verb/direct-object (V-DO) rule, the purpose is to remove the "how" type commit[9]; (2) use TextRank The technology screens the commits again to remove mixed commits; finally, deduplication is performed to remove the commit duplication caused by fork. The detailed process is shown in Figure 8 Data Acquisition Flowchart.

The entire processing flow of Step 1 is shown in Figure 8, specifically including:

Step 1.1, sort the warehouses in Github, and select the top 30% to 35% warehouses as the source data set.

In this embodiment, it is preferable to sort the fork from large to small according to the impact factor of the warehouse; fork refers to copying a copy from someone else's code base to your own code base. Unlike ordinary copying, fork includes All commit records in the original repository. At present, most of the work is to sort the warehouses according to the star first, but in the experiment, it was found that the quality of the commit message in the 100 warehouses ranked according to the star was far lower than that of the 100 warehouses ranked according to the fork.

Step 1.2, use the keyword search method to filter the change records of each warehouse in the source data set, and obtain the change records of different defect types;

In this embodiment, the API provided by Github is used to conduct preliminary screening according to the traditional method of adding defect type keywords to "fix". In this embodiment, five defect types are selected, which are null pointer exception, file operator exception, and illegal parameters. Abnormal, SQL injection, XSS injection, the corresponding keywords are as follows:

NullPointerException: nullpointerexception, nullpointer, npe;

File addressing exception (FileNotFoundException): filenotfoundexception, filenotfound;

Illegal ArgumentException (IllegalArgumentException): illegalargumentException, illegalargument;

SQL injection (SQL injection): sql injection;

XSS injection (XSS injection): xss injection.

Step 1.3, re-screening the change records obtained in step 1.2 with five defect types, filtering out the change records that do not meet the verb/direct object pattern;

In this step, the verb/direct object mode is introduced, and by filtering out messages that do not conform to the verb/direct object mode, a set of data with a very similar commit message format can be obtained. To find patterns, this example uses the natural language processing (NLP) tool Stanford CoreNLP to annotate sentences with grammatical dependencies. A grammatical dependency is a set of dependencies between parts of a sentence. For example, the phrase "fix a bug", which has a subordination relationship, is called "dobj" in Stanford CoreNLP, where the governor is "fix" and the subordination is "bug". For the V-DO filter layer, look for "dobj" dependencies, which represent verb/direct object patterns. For each sentence, check whether the sentence starts with a "dobj" dependency. If a sentence starts with "dobj", mark that sentence as a "dobj" sentence. That is, it is considered to be a commit that meets the requirements.

In step 1.4, use TextRank technology to re-screen the change records obtained in step 1.3, and eliminate mixed change records. This embodiment uses python to implement the TextRank algorithm, specifically: input the commit message into TextRank, and if the extracted keywords top5 contain the keywords involved in step 1.2, then this commit is deemed to meet the requirements.

In step 1.5, because Github has a fork function, the commits of the warehouses may appear repeatedly in the two warehouses. In order to prevent the repetition of commits from affecting the accuracy of later model training, this step deletes the change records obtained in step 1.4. Repeat, delete duplicate change records, and get the change records that meet the requirements and their corresponding Bug-Fix file pairs.

Through the above step 1, the required commit and the Bug-Fix file pair involved in the commit are obtained. However, since a file contains not only defective parts, but also other parts that are not related to defects, the code that is only related to defects is extracted through the following step 2, and this part of code is used to strengthen defect features for better training models. This extraction of codes that are only related to defects is called slicing. With the slicing method, smaller-grained defect positioning can be achieved. Most of the current related work detects defects that can only give an approximate location, such as a package. Or a file problem.

Step 2: Process the change records that meet the requirements obtained in Step 1, and generate vectors and labels of slices.

Step 2.1: Analyze and compare the Bug-Fix file pairs of the change records that meet the requirements obtained in step 1, and obtain the added and deleted code line information corresponding to each change record. As shown in Figure 2, a commit that fixes a null pointer exception corresponds to a file containing a defect and a file after repair. This commit fixes the null pointer exception of the file by deleting line 5 and adding lines 5, 6, 7, and 8.

Step 2.2, according to the added and deleted code line information obtained in step 2.1 and the data flow of the Bug-Fix file pair, determine the defective code line that causes the defect in each change record.

In this embodiment, according to the change information determined in step 2.1 (ie delete 5, add 5, 6, 7, 8) and the type information of the defect (null pointer exception), it is necessary to determine the line of code that caused the defect. For the null pointer, according to the data flow information, it is determined that the variable msg2 in line 6 of the fixed file causes the risk, and then according to the data flow information, the data source of this variable is ex.getMessage(), so these two elements are both Identify it as a Buggy Item, and then determine whether there is a line of code in the Bug file, including the method call of the Buggy item. If it is included, it is considered a defective line of code. According to the above method, it can be determined that line 5 in the bug file is a code line containing a defect.

Step 2.3, according to the data flow and control flow information of the Bug-Fix file pair of the change records that meet the requirements obtained in Step 1, generate slices of the Bug-Fix file pair. Each Bug-Fix file pair corresponds to multiple slices, that is, the respective slices of the Bug file and the Fixed file.

As shown in Figure 3, in this embodiment, for the bug file extraction slice, because the fourth line involves an if conditional statement, so the code execution here will be divided into two branches, that is, the slice will be divided into two, one is slice1, that is, execute the 4th line, execute to the 5th line, and then execute until the end of the method body. One is slice2, that is, the condition in line 4 is not passed, the statement in line 5 is not executed, and then it is executed until the end of the method body. The same is true for fixed files, as shown in Figure 4, three slices, slice4, slice5, and slice6 will be generated.

Step 2.4, according to the defective code line determined in step 2.2, add labels to the slices of the Bug-Fix file pair obtained in step 2.3.

In this embodiment, the slice containing the defective code line is marked as 0, and the slice not containing the defective code line is marked as 1. It should be noted here that in order to enhance the defect information, all code lines executed from the beginning of the method body to the defective code line are marked as new slices and recorded as negative samples, marked as 0. The main reason is that there is a data imbalance problem after the slice is extracted in step 2.3. As shown in Figure 4, the positive samples are slice1, slice3, slice4, slice5, slice6, and the negative sample is slice2. This will lead to the model not being able to learn the characteristics of negative samples very well, but will focus on learning the characteristics of positive samples. Therefore, in order to strengthen the effect of learning defect information, for each defective code line, the code lines executed from the beginning of the method body to the defective code line are regarded as a set, and then a slice is generated. For example, Slice3 is a collection of all code lines executed from line 1 to line 5 of the defective code line, and marks it as a negative sample, thereby expanding the amount of negative samples. And remove the repeated samples in the positive samples and negative samples.

In step 2.5, in order to extract the features of the slice, the slices of the Bug-Fix file pair obtained in step 2.3 are replaced with variable names one by one, and replaced with a unified token. In this embodiment, it is replaced by "var" + the order in which the variable appears in the file, such as var1, var2, as shown in Figure 5. In this way, the influence on the extracted features due to different variable names can be avoided.

In step 2.6, word segmentation is performed on the slices whose variable names have been replaced in step 2.5, and a token sequence is obtained for each slice. Specifically: Divide the slices into words one by one, and separate them with spaces, as shown in Figure 6.

In step 2.7, convert each token sequence obtained in step 2.6 into a vector. This embodiment uses the word2vec tool for vector conversion, which is based on the idea of distributed representation, which maps a token to an integer, and then converts it into a fixed-length vector. , Generate a corresponding 50-dimensional vector for each token, and splicing these vectors together to form a vector for each slice.

Step 3, model training:

Input the slice vector and label obtained in step 2 into the bidirectional LSTM model for training and learning, and obtain a trained detection model.

This embodiment adopts a bidirectional LSTM neural network, which can accurately capture the data flow characteristics of defect codes and improve the recognition accuracy of defects.

The bidirectional LSTM neural network structure is shown in Figure 7, including:

Bidirectional LSTM layer, including two LSTM neural networks, the input of one network is the sequence of vectors from front to back, using the above information to predict the following information to capture the context, and the input of the other network is the sequence of vectors from back to front, Use the following information to predict the above information, and capture the context relationship from another perspective; finally, splicing the output of the hidden layer units of the two networks as the output of the bidirectional LSTM layer;

The fully connected layer is used to map the features learned by the two LSTM neural networks into the label space of the sample;

The activation layer is used to map the multidimensional vector output by the hidden layer to the label space of the sample. It can be output to get the model's predicted label value 0 or 1 for a sample.

Another embodiment of the present invention discloses a Github-based software defect detection model building system, including the following modules:

Data preprocessing module, including:

The source data set acquisition module is used to sort the warehouses in Github, and select the top 30% to 35% warehouses as the source data set. The principle of sorting warehouses in this embodiment is the same as the above embodiment of the model building method.

The primary screening module is used to filter the change records of each warehouse in the source data set by using a keyword search method to obtain change records with different defect types. This embodiment also selects five types of defects, which are null pointer exception, file operator exception, illegal parameter exception, SQL injection, and XSS injection.

The secondary screening module is used to filter out the change records that do not conform to the verb/direct object mode among the change records with different defect types obtained by the primary screening module;

The third screening module is used to re-screen the change records obtained by the secondary screening module by using TextRank technology, and eliminate the mixed change records;

The de-duplication module is used to de-duplicate the change records obtained by the three-time screening module, and obtain the required change records and their corresponding Bug-Fix file pairs;

Data extraction modules, including:

The added and deleted code line information acquisition module is used to analyze and compare the Bug-Fix file pairs of the change records that meet the requirements obtained by the data preprocessing module, and obtain the added and deleted code line information corresponding to each change record ;

The defective code line acquisition module is used to determine the defective code line that each change record causes a defect according to the added and deleted code line information obtained by the added and deleted code line information acquisition module and the data flow of the Bug-Fix file pair;

The slice generation module is configured to generate slices of the Bug-Fix file pair according to the data flow and control flow information of the Bug-Fix file pair of the change records that meet the requirements obtained by the data preprocessing module. Each Bug-Fix file pair corresponds to multiple slices, that is, the respective slices of Bug file and Fixed file.

Adding a labeling module, configured to add labels to the slices of the Bug-Fix file pair obtained by the slice generating module according to the defective code lines determined by the defective code line obtaining module. In this embodiment, the slice containing the defective code line is marked as 0, and the slice not containing the defective code line is marked as 1. It should be noted here that, in order to enhance the defect information, all lines of code executed from the beginning of the method body to the line of defective code are marked as a new slice and recorded as a negative sample, marked as 0.

The variable name replacement module is used to replace the variable names one by one for the slices of the Bug-Fix file pairs obtained by the slice generation module, and replace them with a unified token; such as var1, var2, as shown in Figure 5.

The word segmentation processing module is used to perform word segmentation processing on the slices whose variable names have been replaced by the step variable name replacement module, and each slice obtains a token sequence;

The vector conversion module is used to convert each token sequence obtained by the word segmentation processing module into a vector. In this embodiment, the word2vec tool is used for vector conversion.

The model training module is used to input the vector and label of the slice obtained by the data extraction module into the bidirectional LSTM model for training and learning, and obtain a trained detection model. In this embodiment, a bidirectional LSTM neural network is used, and the structure of the bidirectional LSTM neural network is shown in FIG. 7 .

Through the above detection model establishment method and system, a detection model for detecting the target file to be detected is obtained. Based on the detection model, another embodiment of the present invention also discloses a software defect detection method based on Github. The detection The method includes the following steps:

Step 1, for the target file to be detected, process according to the steps 2.3, 2.6 and 2.7 in the embodiment of the above-mentioned model building method to obtain the vector of the target file;

Step 2: Input the vector in step 1 into the detection model obtained in the above embodiment to obtain the detection result.

For each slice, the detection model can generate a label for it, 0 means defective, 1 means no defect, that is, the purpose of prediction is achieved, and according to the prediction result and the predicted slice, it can also locate which code lines of the target file have defects , to achieve fine-grained location of defects.

A kind of software defect detection system based on Github is also disclosed in another embodiment of the present invention, and this system comprises:

The data processing module is used to process the target file to be detected according to steps 2.3, 2.6 and step 2.7 to obtain the vector of the target file;

The detection module is configured to input the vector obtained by the data processing module into the detection model described in the above embodiment to obtain a detection result.

Simulation

Step 1: The inventor crawled the list of top-ranked warehouses created in all years from 2009 to 2018 from the Github platform, and obtained as much as possible a list of top-ranked warehouses, a total of 378,631 warehouses, accounting for 6.4% of the total. Follow the above steps to crawl the commits that meet the requirements from Github, and then manually mark 10,000 null pointer exceptions, 10,000 file addressing exceptions, 10,000 illegal parameter exceptions, 1,000 SQL injections, and 1,000 XSS injections. Then use the V-DO filtering technology and the keyword extraction technology of TextRank to screen the commits, and the accuracy rate can reach more than 98%, so this method is applied to the entire data set. Finally, the data set was randomly shuffled, and the sampling inspection was performed manually, and all five types of defects were found to be wrongly marked. A total of 153,140 commits were finally screened. The results are shown in Table 1.

Table 1 The number of commits crawled by each defect type and the number of commits after screening

Step 2: By extracting slices for each type of defect, a total of 155,070 slices, 54,000 null pointer exceptions, 52,000 file addressing exceptions, 37,000 illegal parameter exceptions, 5,070 SQL injections, and 5,070 XSS injections were obtained. . Details are shown in Table 1. The data obtained by the present invention is far larger than the data set of related work at present.

Step 3: The hardware platforms for bidirectional LSTM neural network training are: NVIDIA GeForce GTX 1080 GPU, Intel Xeon E5-1620 CPU. When tuning the parameters of the bidirectional LSTM network, leave the defaults of the model unchanged and set the parameters to values widely used in the deep learning community. At the time of input, the vector dimension word_dim of each word is 200 dimensions, and the longest max_len of a sample is limited to 200 words, and zeros are filled if it is not enough. The sample batch_size of one input is 8, and the learning rate learning_rate is 0.01. The node of BLSTM in BLSTM model is 300.

Experimental results:

Divide the dataset into 8:1:1 for training, verification, and testing respectively. After model training, our results are shown in the table below. Because we mark the code blocks containing defect information as 0, and the code blocks without defect information as 1, so the samples containing defect information are regarded as negative samples, and the samples without defect information are regarded as positive samples. We detect each type of defect and the results are shown in Table 2.

It can be seen from Table 2 that the feasibility of the present invention has achieved very high accuracy rates on five types of defects. And it can be seen from Table 1 that the amount of data in the present invention is very large, and the data sets of existing related work are all operated on data sets such as PROMISE and DEFECT4J, and the maximum number of warehouses contained in these data sets is less than 8. Moreover, the present invention utilizes a slicing method for data cutting, and then uses slices for training and detection, and finally realizes small-grained defect location.

Table 2 F1-measure value of test results

It should be noted that the present invention is not limited to the above-mentioned embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some technical changes without creative work according to the disclosed technical content. Some replacements and modifications are made to the features, and these replacements and modifications are all within the protection scope of the present invention.

Claims (10)

1. A software defect detection model building method based on Github is characterized by comprising the following steps:

step 1, data preprocessing:

sorting the warehouses in the Github, selecting the warehouse with the highest rank as a source data set, and screening and de-weighting change records of each warehouse in the source data set to obtain the change records meeting the requirements and the corresponding Bug-Fix file pairs;

the Bug-Fix file pair refers to a file Bug file before modification and a modified file Fixed file;

step 2, processing the change records meeting the requirements obtained in the step 1 and the corresponding Bug-Fix file pairs thereof to generate sliced vectors and labels:

step 2.1, analyzing and comparing the Bug-Fix file pairs of the change records meeting the requirements obtained in the step 1 to obtain added and deleted code line information corresponding to each change record;

step 2.2, determining a defect code line causing defects of each change record according to the added and deleted code line information obtained in the step 2.1 and the data stream of the Bug-Fix file pair;

step 2.3, generating slices of the Bug-Fix file pairs according to the data flow and control flow information of the Bug-Fix file pairs which meet the requirement and are obtained in the step 1;

step 2.4, according to the defect code line determined in the step 2.2, adding a label to the slice of the Bug-Fix file pair obtained in the step 2.3;

step 2.5, replacing the variable names of the slices of the Bug-Fix file pair obtained in the step 2.3 one by one, and replacing the slices with a unified token;

step 2.6, the slices with the variable names replaced in the step 2.5 are subjected to word segmentation one by one, and each slice obtains a token sequence;

step 2.7, converting each token sequence obtained in the step 2.6 into a vector;

step 3, model training:

and (3) inputting the vector and the label of the slice obtained in the step (2) into a bidirectional LSTM model for training and learning to obtain a trained detection model.

2. The method for building a Github-based software defect detection model according to claim 1, wherein the step 1 specifically comprises:

step 1.1, sorting the warehouses in Github from large to small according to the influence factors fork of the warehouses, and selecting the warehouses with the top 30% -35% of ranks as a source data set;

step 1.2, screening change records of each warehouse in the source data set by adopting a keyword search method to obtain change records with different defect types;

step 1.3, filtering change records which do not accord with verb/direct object modes in the change records with different defect types obtained in the step 1.2;

step 1.4, re-screening the change records obtained in the step 1.3 by adopting a TextRank technology, and removing mixed change records;

the TextRank technology is a TextRank algorithm, and the specific method for screening the change record obtained in the step 1.3 by adopting the TextRank technology is as follows: inputting the change record obtained in the step 1.3 into a TextRank algorithm, if the first five extracted keywords comprise the keywords related in the step 1.2, considering that the change record is in accordance with the requirement, entering the step 1.5, and if not, deleting the change record;

and step 1.5, removing the duplicate of the change record obtained in the step 1.4 to obtain the change record meeting the requirement and a corresponding Bug-Fix file pair thereof.

3. The method according to claim 1, wherein in the step 2.4, during the labeling process, the slice including the defect code line is marked as 0, and the slice not including the defect code line is marked as 1.

4. The method of claim 1, wherein in step 2.7, each token sequence is converted into a vector using a word2vec tool.

5. A software defect detection model building system based on Github is characterized by comprising the following modules:

the data preprocessing module is used for sequencing the warehouses in the Github, selecting the warehouse with the top rank as a source data set, and screening and removing the weight of the change record of each warehouse in the source data set to obtain the change record meeting the requirement and the corresponding Bug-Fix file pair; the Bug-Fix file pair refers to a file Bug file before modification and a modified file Fixed file;

a data extraction module comprising:

the added and deleted code line information acquisition module is used for analyzing and comparing the Bug-Fix file pairs of the change records meeting the requirements, which are obtained by the data preprocessing module, so as to obtain the added and deleted code line information corresponding to each change record;

the defect code line acquisition module is used for determining the defect code line causing the defect of each change record according to the added and deleted code line information acquired by the added and deleted code line information acquisition module and the data stream of the Bug-Fix file pair;

the slice generating module is used for generating slices of the Bug-Fix file pairs according to the data flow and the control flow information of the Bug-Fix file pairs which meet the required change records and are obtained by the data preprocessing module;

the tag adding module is used for adding tags to the slices of the Bug-Fix file pair obtained by the slice generating module according to the defect code line determined by the defect code line obtaining module;

the variable name replacing module is used for replacing the variable names of the slices of the Bug-Fix file pair obtained by the slice generating module one by one into a uniform token;

the word segmentation processing module is used for carrying out word segmentation processing on the slices with the variable names replaced by the variable name replacing module one by one, and each slice obtains a token sequence;

the vector conversion module is used for converting each token sequence obtained by the word segmentation processing module into a vector;

and the model training module is used for inputting the vector and the label of the slice obtained by the data extraction module into the bidirectional LSTM model for training and learning to obtain a trained detection model.

6. The Github-based software fault detection modeling system of claim 5, wherein said pre-processing of data includes:

the source data set acquisition module is used for sorting the warehouses in the Github from large to small according to the influence factors fork of the warehouses, and selecting the warehouses with the ranks 30% -35% as the source data set;

the primary screening module is used for screening the change records of each warehouse in the source data set by adopting a keyword searching method to obtain the change records with different defect types;

the secondary screening module is used for filtering change records which do not accord with verb/direct object modes in the change records with different defect types obtained by the primary screening module;

the third screening module is used for re-screening the change records obtained by the secondary screening module by adopting a TextRank technology and rejecting mixed change records;

the TextRank technology refers to a TextRank algorithm, and the concrete method for screening the change records obtained in the step 1.3 by adopting the TextRank technology is as follows: inputting the change records obtained by the secondary screening module into a TextRank algorithm, if the first five extracted keywords contain the keywords related to the primary screening module, keeping the change records, entering a duplicate removal module, and otherwise, deleting the change records;

and the duplication removing module is used for removing duplication from the change records obtained by the third screening module to obtain the change records meeting the requirements and the corresponding Bug-Fix file pairs.

7. The Github-based software defect detection modeling system of claim 5, wherein in the tag-adding module, slices containing lines of defect code are marked as 0 and slices not containing lines of defect code are marked as 1 during tag addition.

8. The Github-based software bug detection model building system of claim 5, wherein the vector conversion module converts each token sequence into a vector using a word2vec tool.

9. A software defect detection method based on Github is characterized by comprising the following steps:

step 1, processing a target file to be detected according to steps 2.3 and 2.6 and step 2.7 of any one of claims 1 to 4 to obtain a vector of the target file;

and 2, inputting the vector obtained in the step 1 into the detection model in the claim 1 to obtain a detection result.

10. A system for detecting software defects based on gitubs, comprising:

a data processing module, configured to process the target file to be detected according to steps 2.3 and 2.6 and step 2.7 of any one of claims 1 to 4, to obtain a vector of the target file;

and the detection module is used for inputting the vector obtained by the data processing module into the detection model of claim 1 to obtain a detection result.

Images