KR20180111126A - A method for big data processing for fusion technology extraction - Google Patents

Abstract

The present invention relates to a big data processing method for extracting a fusion technique for expanding key features related to a technology of interest using extracted big data, and more particularly, to a big data processing method for extracting fusion data from a set of two or more data into one or more new clusters Setting a new cluster; A cluster comparison step of comparing one or more preset clusters with one or more of the set new clusters; Determining each representative cluster path of one or more clusters based on the comparison result; And setting representative cluster path information including a representative cluster path of the determined one or more clusters.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a method and apparatus for extracting fusion technology,

The present invention relates to a big data processing method for extracting a fusion technique, and more particularly, to a big data processing method for extracting a fusion technique for extending major features related to a technology of interest using extracted big data.

The importance of data mining has been highlighted by the advent of big data, which has a wide variety of data types and ranges. In particular, various methodologies for analyzing unstructured data such as web page contents and web log, social network service text information and images generated at an incredibly fast speed on the web have appeared, The coverage of mining is expanding.

Logistic regression, pricipal ananlysis, discriminat analysis, and clustering analysis are the methodologies that are used in data mining.

The prior art document discloses an invention relating to query ranking based on query clustering and categorization. In the prior art literature, search queries are selected and grouped into one or more clusters. A representative query is selected for each cluster, and each cluster is associated with a corresponding representative category. The rank is assigned to each cluster based on the cluster popularity of the cluster and the category popularity of the representative category of the cluster, and the selected representative query is presented in order according to the rank of each cluster.

In the prior art, a representative query is selected for each cluster, each cluster is associated with the corresponding representative category, and the query result of the keyword is compared with the existing cluster to update the similarity. However, It does not disclose.

US registered patent US8145623 B1

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and its object is to update a basic hierarchical structure using data similar to a basic hierarchical structure extracted from big data, to extract data different from a basic hierarchical structure extracted from big data To create and update a new hierarchical structure to expand key features related to the technology of interest using the extracted big data.

A big data processing method for extracting a fusion technique according to an embodiment of the present invention includes: a new cluster setting step of setting a set of two or more data into one or more new clusters; A cluster comparison step of comparing one or more preset clusters with one or more of the set new clusters; Determining each representative cluster path of one or more clusters based on the comparison result; And setting representative cluster path information including a representative cluster path of the determined one or more clusters.

The new cluster setting step may include setting a path including the two or more data with reference to a relation of data having a predetermined connection relationship.

And setting a cluster according to a degree of similarity of a plurality of links constituting the set paths.

The degree of similarity may be a similar degree of links commonly included in the set paths.

The step of setting a new cluster may include setting a representative cluster path in one or more new clusters.

The step of setting a new cluster may include a step of setting a representative cluster path of each new cluster when there are a plurality of the set new clusters.

The cluster comparison step may determine whether each representative cluster path of the preset clusters and each representative cluster path of the set new clusters are similar.

The step of determining similarity may include comparing a standard deviation of each representative cluster path of the preset clusters with a standard deviation of each representative cluster path of the set new clusters.

Wherein if it is determined that the pair of at least one predetermined cluster and the new cluster are similar in the cluster comparison step, determining whether the new cluster constituting the pair determined to be similar to the predetermined cluster is a cluster having the same hierarchical structure .

And connecting a new cluster to a predetermined cluster constituting the pair judged to be similar if it is determined that the hierarchical structure of the predetermined cluster and the new cluster forming the pair determined to be similar are not the same.

The step of determining the representative cluster path may be a step of determining each representative cluster path of the one or more connected clusters to which the new cluster is connected.

And designating one object in each representative cluster path of the one or more connected clusters as a category of a representative cluster path including the object if each representative cluster path of the one or more connected clusters is determined.

The cluster comparison step may include generating a set including one or more clusters of the new clusters if it is determined that at least one of the new clusters is not similar to all the predetermined clusters .

Wherein the one or more clusters included in the set include a corresponding object corresponding to a target object included in a hierarchical structure of one or more predetermined clusters, The path including the corresponding object may be a cluster judged not similar to the path including the target object in the one or more predetermined cluster.

Generating a hierarchy of one or more clusters included in the set; And generating a set of hierarchical structures of the one or more clusters.

The step of determining the representative cluster path may be a step of determining each representative cluster path of the one or more clusters included in the cluster.

And designating one object in each representative cluster path of one or more clusters included in the set as a category of a path including the object.

Wherein the step of setting the representative cluster path information comprises the step of setting the representative cluster path information by adding information on the representative cluster path of one or more clusters determined in the step of determining the representative cluster path to the representative cluster path information .

According to the big data processing method for extracting the fusion technique according to an embodiment of the present invention, main features related to the technology of interest are expanded through data extracted from big data similar to the existing hierarchical structure.

In addition, it is similar to the objects constituting the existing hierarchical structure, but the connection relation between the objects has the effect of expanding the main features related to the interested technology through the data extracted from the big data which is different from the existing hierarchical structure.

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the technical features of the invention.
FIG. 1 is a flowchart briefly showing a method of processing big data for extracting a fusion technique according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a path in a cluster in a big data processing method for extracting a fusion technique according to an embodiment of the present invention.
3 is a diagram illustrating an example of a connection process between clusters in a big data processing method for extracting a fusion technique according to an embodiment of the present invention.

The terms first and / or second in this specification are used only for the purpose of distinguishing one element from another. That is, the components are not intended to be limited by the terms.

The components, features, and steps referred to in the specification as " comprising " in this specification are intended to mean that there are corresponding components, features, and steps, and do not preclude the presence of one or more other components, features, steps, and the like Is not.

Includes plural forms as long as it is not specified and specified in the singular form herein. That is, the components and the like referred to in this specification may mean the presence or addition of one or more other components or the like.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs to be.

That is, terms such as those defined in commonly used dictionaries should be construed as meaning consistent with meaning in the context of the related art, and unless otherwise expressly defined herein, interpreted in an ideal or overly formal sense It does not.

Hereinafter, a method of processing big data for extracting a fusion technique according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Here, the subject of each step of the method is a central processing unit and a processor.

FIG. 1 is a flowchart briefly showing a method of processing big data for extracting a fusion technique according to an embodiment of the present invention.

Referring to FIG. 1, a method of processing big data for extracting a fusion technique according to an embodiment of the present invention includes steps (S101) of constructing a path between data by analyzing big data and referring to a data relation, (S103); comparing the preset cluster with the new cluster (S105); determining a data set (Set) of the new cluster having a hierarchical structure different from the predetermined cluster; (S111) of creating a new hierarchical structure of new clusters, determining a representative cluster path, designating a category of a path (S113 and S115), and connecting representative cluster path information (Step S117).

In step S101, the main keyword of the big data is analyzed. Referring to the relation of the data having the predetermined connection relationship, and setting one or more paths including the data.

Step S103 is a step of setting clusters by comparing routes between the data configured in step S101 and setting clusters according to the similarities (link similarities) of a plurality of links constituting each path. Link similarity refers to a similar degree of links that each path commonly includes. And sets similar paths that include links in common among a plurality of paths into one cluster. The links (partial paths) commonly included in the paths included in one cluster are constituted by representative cluster paths of the corresponding clusters. That is, a cluster is set according to the similarity of a plurality of links constituting the set routes, and the similarity is a similar degree of links commonly included in the set routes. Also, a representative cluster path is set in one or more new clusters.

Step S105 is a step of comparing new clusters, which are clusters that have been set through the steps S101 and S103, with the representative cluster path of each of the predetermined clusters and the representative cluster path of each of the new clusters, And determining the degree of similarity between the clusters. Each link has statistical values, attributes, etc., and each path has a standard deviation of the value of the included link. A method for determining the similarity between each representative cluster path of the predetermined clusters and each representative cluster path of the new clusters is to compare the similarity of the standard deviations of the representative cluster paths. That is, it is determined whether each representative cluster path of the predetermined clusters is similar to each representative cluster path of the new clusters. Also, the standard deviation of each representative cluster path of the predetermined clusters is compared with the standard deviation of each representative cluster path of the new clusters to determine whether they are similar.

In step S107, if it is determined that the pair of at least one of the predetermined clusters and the new clusters compared in step S105 is similar, it is a step for determining a new cluster having a different hierarchical structure from the predetermined cluster. In step S107, since a new cluster constituting a pair of clusters determined to be similar may be the same as the predetermined cluster, it is similar when judging the similarity between the preset cluster and the representative cluster path through step S105, Clusters. ≪ / RTI >

As a result of the determination, if it is determined that the predetermined cluster and the new cluster have different hierarchical structures, a set of data having different hierarchical structures (data included in the new cluster but not included in the predetermined cluster) .

That is, the step S107 compares the preset cluster with the new cluster. If it is determined that the pair of at least one preset cluster and the new cluster are similar, the new cluster and the predetermined cluster, Or the like.

In operation S109, the set of data having different hierarchical structures set in operation S107 is linked to a set of existing data. If the new cluster that is judged to be similar includes data that is not included in the predetermined cluster, this step is to include this data in the data set of the predetermined cluster. A connection cluster is created by connecting a new cluster to a preset cluster.

That is, in step S109, when it is determined that the hierarchical structure of the predetermined cluster and the new cluster forming the pair determined to be similar are not the same, the step of connecting the new cluster to the predetermined cluster.

If it is determined in step S105 that the one or more new clusters are not similar to all preset clusters, the step S111 may determine that the new hierarchical structure of one or more new clusters Respectively. In step S111, the data relation in step S101 may be referenced in creating the hierarchical structure. The new clusters, which are judged not to be similar to each other in step S105, generate respective hierarchical structures and form a hierarchical structure. However, rather than all the new clusters judged not to be similar to the predetermined cluster to create a hierarchical structure, at least one object of the new cluster should be similar to an object having a plurality of hierarchical structures of one or more clusters of predetermined clusters And the path including the object of the new cluster should not be similar to the path included in the predetermined cluster.

That is, in step S111, when it is determined that at least one of the new clusters is not similar to all the predetermined clusters, a set including new clusters determined to be not similar is generated. It also creates a hierarchy of clusters contained in the set and generates a set of generated hierarchies. However, the clusters included in the set must include a corresponding object corresponding to the target object included in the hierarchical structure of one or more predetermined clusters, and the path including the corresponding object in the one or more clusters included in the set may include one or more predetermined It should be judged not to be similar to the path containing the target object in the cluster.

For example, in step S111, there are a predetermined cluster A and a new cluster B determined to be not similar in step S105, and the representative cluster path of A and B includes a common object C . That is, B includes a corresponding object C corresponding to at least one target object C of A. In this case, if it is determined that the path including the object C in A and the path including the corresponding object C in B are not similar, a new hierarchical structure for B is generated.

Since the step S113 can be performed by the clusters that have undergone the step S109 or S111, the step S113 is followed by the step S113 (S113-S109) and the step S111 is followed by the step S113 (S113-S111).

In step S113-S109, since the new clusters satisfying the similarity in the predetermined cluster but having different hierarchical structures are connected in step S109, the step of determining the representative cluster path of the connected cluster in which one or more new clusters are connected to the predetermined cluster to be. That is, it is a step of determining each representative cluster path of one or more connection clusters to which a new cluster is connected.

In operation S113-S111, the representative cluster path of each cluster having the hierarchical structure generated in operation S111 is determined. That is, it is a step of determining each representative cluster path of one or more clusters included in the set generated in step S111.

The step S115 is a step of designating one object included in each representative cluster path of the clusters after the step S113 (S113-S109 or S113-S111) as a category of the representative cluster path including the object. Here, since the object designated as the category represents the representative cluster path of the cluster to which the object belongs, the object having the highest similarity with other objects in the cluster is designated as a category. That is, the step S115 is a step of designating one object having the highest mutual similarity among the objects in each representative cluster path of the one or more clusters in which the representative cluster path is determined by going through the step S113 as a category of the representative cluster path.

The mutual similarity is determined by comparing one or more objects belonging to the representative cluster path with other objects in the cluster to determine the object having the highest similarity with other objects in the cluster. That is, the object having the highest mutual similarity is judged through a relative comparison between the objects belonging to the representative cluster path of one cluster.

The step S117 is a step of updating the representative cluster path information when the representative cluster path of each cluster is changed through the steps of S101 to S115. In updating the representative cluster path information, step S105 may be referred to. (Through steps S113-S119), information on the representative cluster paths newly created (through steps S113-S111) is added to the information of the existing representative cluster paths and set. That is, step S117 is a step of setting the representative cluster path information by adding information on the representative cluster path of at least one cluster in which the representative cluster path has been determined to the representative cluster path information through step S113.

FIG. 2 is a diagram illustrating an example of a path in a cluster in a big data processing method for extracting a fusion technique according to an embodiment of the present invention.

FIG. 2 includes a cluster 100, an A path 101, a B path 102, a C path 103, and an A path object 104A to 109A that are present in the cluster 100.

Referring to Figures 1 and 2,

In step S101, the A path 101, the B path 102, and the C path 103 can be set with reference to relations of data having a predetermined connection relationship.

In step S103, the A path 101, the B path 102, and the C path 103 include common paths 106A-> 107A. Accordingly, the A path 101, the B path 102, and the C path 103 can be set as the cluster 100, and the paths 106A-> 107A can be set as the representative cluster path of the cluster 100. [

In step S105, since the path 106A-> 107A is the representative cluster path of the cluster 100, it can be compared with the representative cluster path of the predetermined cluster.

As a result of step S105, if it is determined that the cluster 100 is similar to the predetermined cluster, the hierarchical structure of the cluster 100 is compared with the hierarchical structure of the predetermined cluster in step S107, It can be determined whether or not they are the same. As a result of step S107, if it is determined that the cluster 100 and the predetermined cluster are not the same, the cluster 100 may be connected to the predetermined cluster in step S109 to create a connected cluster. In step S113, the representative cluster path of the connected cluster may be determined. In step S115, one of the representative cluster paths of the connected cluster may be designated as a category of the representative cluster path. In step S117, the representative cluster path information of the connected cluster may be added to the representative cluster path information.

If it is determined in step S105 that the cluster 100 and the preset cluster are not similar, a hierarchical structure for the cluster 100 may be generated in step S111. In step S113, the representative cluster path of the cluster 100 may be determined. In step S115, an object having higher mutual similarity with the objects in the cluster 100 among the two objects 106A and 107A belonging to the representative cluster path of the cluster 100 is selected as the representative cluster path 106A-> 107A of the cluster 100, As shown in FIG. In step S117, the representative cluster path (106A- > 107A) of the cluster 100 may be added to the representative cluster path information.

3 is a diagram illustrating an example of a connection process between clusters in a big data processing method for extracting a fusion technique according to an embodiment of the present invention.

Cluster path-A 209 shown in FIG. 3 is a path including objects 1, 2, and 3 included in DOC 3 205 and DOC 4 207. However, since one cluster of the DOC2 203 includes objects of 1, 2, and 3, the cluster paths have the connection order, and therefore the connection order of the cluster path-A 209 and the DOC2 203, The DOC2 203 does not include the cluster including the cluster path-A.

However, since DOC1 201, DOC2 203 and DOC3 205 all contain the same object connection sequence (4-5-6) as objects 4, 5, and 6, 0.0 > 211 < / RTI >

The cluster path-A 209 includes objects of 1, 2, and 3, and the cluster path-B 211 includes objects of 4, 5, and 6, A 209 and the cluster path-B 211, it is possible to connect the cluster path-A 209 and the cluster path-B 211 using the 1 to 6 data connection information of the DOC 3 205 . Referring to FIG. 1, in step S109, a new cluster may be connected to a predetermined cluster.

Although the description herein has been made in some illustrative aspects, various modifications and variations can be made from the categories defined by the following claims, and the technical scope of the invention is defined in the following claims It should be decided by.

100: Cluster
101: Path A
102: Path B
103: C path
104A to 109A: Objects included in path A
201: DOC 1
203: DOC 2
205: DOC 3
207: DOC 4
209: Cluster path -A
211: Cluster path-B

Claims (18)

Setting a set of two or more pieces of data as one or more new clusters;
A cluster comparison step of comparing one or more preset clusters with one or more of the set new clusters;
Determining each representative cluster path of one or more clusters based on the comparison result; And
And setting representative cluster path information including a representative cluster path of the determined one or more clusters.
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
The new cluster setting step includes:
And setting a path including the two or more data with reference to a relation of data having a predetermined connection relationship.
Big Data Processing Method for Fusion Technology Extraction. 3. The method of claim 2,
And establishing a cluster according to a degree of similarity of a plurality of links constituting the set paths.
Big Data Processing Method for Fusion Technology Extraction. The method of claim 3,
Wherein the similarity degree is a degree of similarity of links commonly included in the set paths,
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
The new cluster setting step includes:
And setting a representative cluster path in one or more new clusters.
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
The new cluster setting step includes:
When there are a plurality of the new clusters set,
And setting a representative cluster path of each new cluster.
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
The cluster comparison step may include:
Determining whether each representative cluster path of the preset clusters is similar to each representative cluster path of the set new clusters,
Big Data Processing Method for Fusion Technology Extraction. 8. The method of claim 7,
The step of determining whether or not the similarity is similar,
Comparing the standard deviation of each representative cluster path of the predetermined clusters with the standard deviation of each representative cluster path of the set new clusters,
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
In the cluster comparison step,
If it is determined that the pair of at least one predetermined cluster and the new cluster is similar,
Further comprising the step of determining whether a new cluster and a predetermined cluster constituting a pair determined to be similar are clusters having the same hierarchical structure,
Big Data Processing Method for Fusion Technology Extraction. 10. The method of claim 9,
If it is determined that the hierarchical structure of the predetermined cluster and the new cluster forming the pair determined to be similar are not the same,
And connecting a new cluster to a predetermined cluster constituting a pair determined to be similar,
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
Wherein determining the representative cluster path comprises:
Determining a representative cluster path of one or more connection clusters connecting the new clusters,
Big Data Processing Method for Fusion Technology Extraction. 12. The method of claim 11,
If each representative cluster path of the one or more connected clusters is determined,
Designating one object in each representative cluster path of the one or more connected clusters as a category of a representative cluster path including the object;
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
The cluster comparison step may include:
If it is determined that at least one of the new clusters is not similar to all the predetermined clusters,
Generating a set containing one or more clusters of the new clusters.
Big Data Processing Method for Fusion Technology Extraction. 14. The method of claim 13,
In the step of generating the set,
The one or more clusters included in the set may include:
A corresponding object corresponding to a target object included in a hierarchical structure of at least one predetermined cluster,
Wherein the path including the corresponding object in one or more clusters included in the cluster is a cluster judged not similar to a path including the target object in the one or more predetermined cluster,
Big Data Processing Method for Fusion Technology Extraction. 14. The method of claim 13,
Generating a hierarchy of one or more clusters included in the set; And
And generating a set of hierarchical structures of said one or more clusters.
Big Data Processing Method for Fusion Technology Extraction. 14. The method of claim 13,
Wherein determining the representative cluster path comprises:
Determining each representative cluster path of one or more clusters included in the set,
Big Data Processing Method for Fusion Technology Extraction. 17. The method of claim 16,
Designating one object in each representative cluster path of one or more clusters included in the set as a category of a representative cluster path including the object;
Big Data Processing Method for Fusion Technology Extraction. The method according to claim 1,
Wherein the step of setting the representative cluster path information comprises:
And setting the representative cluster path information by adding information on a representative cluster path of one or more clusters determined in the step of determining the representative cluster path to the representative cluster path information,
Big Data Processing Method for Fusion Technology Extraction.

Images