KR20230016537A - Method, computer device, and computer program to classify data using unsupervised contrastive learing - Google Patents

Abstract

A method, computer device, and computer program for classifying data through unsupervised contrastive learning are disclosed. The data classification method comprises the steps of: generating a first similarity matrix for a contextual representation of given data; generating a second similarity matrix for an auxiliary representation that complements the context representation; generating a third similarity matrix by combining the first similarity matrix and the second similarity matrix; and performing model learning for data classification through contrastive learning using the third similarity matrix. Accordingly, the problem of distorting data meaning can be solved.

Description

Method, computer device, and computer program capable of classifying data through unsupervised contrast learning

The description below relates to an unsupervised contrastive clustering technique for data classification.

With the recent development of deep learning, NLP (natural language processing) task performance is also greatly improved.

Most approaches to building NLP models rely on supervised learning methods and require at least thousands of labeled datasets.

In order to build a dialogue system for a new domain, huge design costs are required from intent design to data collection and model training.

While log data is sufficient in most situations, information about labeling what intents exist and what data has specific intents is often lacking.

Intent discovery is about finding new categories for intent classification in unlabeled data.

The intent discovery problem can be approached by an unsupervised method that does not use labeling data and a semi-supervised method that uses a small amount of labeling data.

At this time, in the case of a method based on representation learning, it is difficult to construct an appropriate representation for clustering because it does not include a clustering promotion target.

In addition, since the offline clustering method learns the model alternately between the parameter update step and the cluster assignment step, there is a risk of exposure to learning instability and expression irregularity.

In addition, contrast clustering methods vary greatly depending on data augmentation techniques. Unlike the vision domain, data augmentation in the text domain distorts the meaning of sentences and can also change labels, which can degrade model performance.

It provides techniques to learn from unlabeled data.

Provides a technique to classify data through similarity-based unsupervised contrast clustering.

A data classification method executed in a computer device, the computer device comprising at least one processor configured to execute computer readable instructions contained in a memory, the data classification method comprising: by the at least one processor, a given generating a first similarity matrix for a contextual representation of data; generating, by the at least one processor, a second similarity matrix for an auxiliary representation complementing the context representation; generating, by the at least one processor, a third similarity matrix by combining the first similarity matrix and the second similarity matrix; and performing, by the at least one processor, model learning for data classification through contrastive learning using the third similarity matrix.

According to one aspect, the performing of the model learning may include performing the control learning without data augmentation by extracting a positive sample based on similarity between data instances for unlabeled data classification. can

According to another aspect, the performing of model learning may include selecting positive samples and negative samples row-wise for the third similarity matrix; and performing the collational learning by applying cross entropy loss to the positive samples and the negative samples.

According to another aspect, generating the first similarity matrix may include extracting a dense contextual representation by encoding the data through a pretrained language model (PLM).

According to another aspect, generating the first similarity matrix may include generating the first similarity matrix using a dot product of vectors with respect to the context expression.

According to another aspect, generating the second similarity matrix may include generating the auxiliary expression as a sparse expression indicating existence of data regardless of an order of data.

According to another aspect, generating the second similarity matrix may include generating the auxiliary expression using term frequency-inverse document frequency (TF-IDF) when text type data is given. .

According to another aspect, the generating of the second similarity matrix may include generating the auxiliary expression using a depth map or feature map of an image when data of an image type is given can include

According to another aspect, the generating of the third similarity matrix may include adjusting the scale between the first similarity matrix and the second similarity matrix using a standard deviation of the first similarity matrix and an average of the second similarity matrix. It may include determining hyperparameters for .

According to another aspect, generating the third similarity matrix may include setting a decay coefficient for the second similarity matrix according to learning progress.

A computer program stored in a computer readable recording medium is provided to execute the data classification method according to any one of claims 1 to 10 in the computer device.

A computer device comprising: at least one processor configured to execute computer readable instructions contained in a memory, wherein the at least one processor performs the steps of: generating a first similarity matrix for a given contextual representation of data; generating a second similarity matrix for an auxiliary expression supplemented with the context expression; generating a third similarity matrix by combining the first similarity matrix and the second similarity matrix; and a computer device processing a process of performing model learning for data classification through collational learning using the third similarity matrix.

According to embodiments of the present invention, the problem of data semantic distortion can be solved by applying unsupervised collation clustering based on pairwise similarity in a mini-batch.

According to embodiments of the present invention, by using an auxiliary representation that is a sparse representation of data given as an input, it is possible to alleviate the problem of overlapping noise by contrastive learning without using data augmentation.

1 is a block diagram for explaining an example of an internal configuration of a computer device according to an embodiment of the present invention.
2 is a flowchart illustrating an example of a data classification method that can be performed by a computer device according to an embodiment of the present invention.
3 illustrates an example of a process of generating a similarity matrix for a contextual expression according to an embodiment of the present invention.
4 illustrates an example of a process of generating a similarity matrix for an auxiliary expression according to an embodiment of the present invention.
5 illustrates an example of a process of generating a final similarity matrix for collation learning in one embodiment of the present invention.
6 illustrates an example of a collation learning process using a similarity matrix in an embodiment of the present invention.
7 illustrates an example of a clustering process using PLM learned by unsupervised collegial learning in one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to similarity-based unsupervised collation clustering techniques. In this specification, a sample means data sampled for learning and evaluation from all data, and an instance may be used to refer to data classified as separate labels without forming a cluster in contrast learning. However, this is a term that eventually refers to data according to processing steps, and is not strictly classified, and can be collectively referred to as data.

Embodiments, including those specifically disclosed herein, may utilize pairwise similarity in mini-batches to avoid semantic distortion problems due to data augmentation, and may use auxiliary representations to avoid learning instability due to pseudo labels. can

1 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention. For example, a data classification system according to embodiments of the present invention may be implemented by the computer device 100 shown in FIG. 1 .

As shown in FIG. 1, a computer device 100 is a component for executing a data classification method according to embodiments of the present invention, and includes a memory 110, a processor 120, a communication interface 130, and an input/output interface. (140).

The memory 110 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, a non-perishable mass storage device such as a ROM and a disk drive may be included in the computer device 100 as a separate permanent storage device distinct from the memory 110. Also, an operating system and at least one program code may be stored in the memory 110 . These software components may be loaded into the memory 110 from a recording medium readable by a separate computer from the memory 110 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memory 110 through the communication interface 130 rather than a computer-readable recording medium. For example, software components may be loaded into memory 110 of computer device 100 based on a computer program installed by files received over network 160 .

The processor 120 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processor 120 by memory 110 or communication interface 130 . For example, processor 120 may be configured to execute received instructions according to program codes stored in a recording device such as memory 110 .

The communication interface 130 may provide functions for the computer device 100 to communicate with other devices through the network 160 . For example, a request, command, data, file, etc. generated according to a program code stored in a recording device such as the memory 110 by the processor 120 of the computer device 100 is transmitted to the network ( 160) to other devices. Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device 100 via the communication interface 130 of the computer device 100 via the network 160 . Signals, commands, data, etc. received through the communication interface 130 may be transmitted to the processor 120 or the memory 110, and files, etc. may be stored as storage media that the computer device 100 may further include (described above). permanent storage).

The communication method is not limited, and may include not only a communication method utilizing a communication network (eg, a mobile communication network, wired Internet, wireless Internet, and broadcasting network) that the network 160 may include, but also short-distance wired/wireless communication between devices. there is. For example, the network 160 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , one or more arbitrary networks such as the Internet. In addition, the network 160 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, and the like. Not limited.

The input/output interface 140 may be a means for interface with the input/output device 150 . For example, the input device may include devices such as a microphone, keyboard, camera, or mouse, and the output device may include devices such as a display and a speaker. As another example, the input/output interface 140 may be a means for interface with a device in which functions for input and output are integrated into one, such as a touch screen. The input/output device 150 and the computer device 100 may be configured as one device.

Also, in other embodiments, computer device 100 may include fewer or more elements than those of FIG. 1 . However, there is no need to clearly show most of the prior art components. For example, the computer device 100 may be implemented to include at least a portion of the above-described input/output device 150 or may further include other components such as a transceiver, a camera, various sensors, and a database.

In this embodiment, we propose a new clustering framework for data classification.

The data classification method according to the present invention is a novel end-to-end (E2E) unsupervised clustering method, and can perform comparative learning without data augmentation by extracting positive samples by utilizing similarity between data instances.

In addition, in the present embodiment, by using an auxiliary representation that guides the model to extract an appropriate positive sample, instability in the initial learning stage that occurs when only a dense contextual representation is used can be alleviated.

2 is a flowchart illustrating an example of a data classification method that can be performed by a computer device according to an embodiment of the present invention.

The data classification method according to the present embodiment may be performed by the computer device 100 described above. In this case, the processor 120 of the computer device 100 may be implemented to execute a control instruction according to an operating system code or at least one program code included in the memory 110 . Here, the processor 120 causes the computer device 100 to perform steps S210 to S230 included in the data classification method of FIG. 2 according to a control command provided by a code stored in the computer device 100 ( 100) can be controlled.

Referring to FIG. 2 , in step S210, the processor 120 may encode the given input data into a dense context representation and construct an auxiliary representation to compensate for learning instability. Using similarity-based pseudo-labels to extract positive samples when the model is not sufficiently trained is very unstable because pseudo-labels may not be selected correctly and noise due to incorrect label selection accumulates during the training process. In order to alleviate this problem, in the present embodiment, as an expression that can supplement a wide category, an auxiliary expression indicating the existence of data regardless of the order can be used.

In step S220, the processor 120 may generate a similarity matrix for the context expression and the auxiliary expression. The processor 120 may generate a similarity matrix indicating whether a pair of instances in the context representation and the auxiliary representation belong to the same cluster. Processor 120 may use the batch of data instances to build a similarity matrix.

In step S230, the processor 120 may select a positive sample from each row of the similarity matrix generated in step S220 and learn a model for the dialog system with a contrast loss. The processor 120 applies cross entropy loss by selecting the data having the highest value row-wise for the similarity matrix as positive samples and selecting the remaining data as negative samples. to learn a pretrained language model (PLM).

In this embodiment, by applying contrast learning without data augmentation, the problem of semantic distortion due to data augmentation can be avoided, and by using an auxiliary expression that guides the model to extract appropriate positive samples, contrast learning is performed without data augmentation. It is possible to mitigate the noise overlapping problem that may occur according to the above method.

After learning, the processor 120 extracts an expression for given input data from the learned PLM and applies various clustering algorithms to perform classification (labeling) on the corresponding expression. The processor 120 may perform clustering (data classification) by encoding input data using only the PLM in the learned model and applying K-mean clustering or agglomerative clustering.

An example of a task requiring data classification is to classify the intent of specific data (for example, a query in a customer center). Unsupervised contrast learning for data classification according to the present invention can be appropriately used to solve this task, and the detailed process is as follows.

3 illustrates an example of a process of generating a similarity matrix for a contextual expression according to an embodiment of the present invention.

과 특수 토큰(예를 들어, [CLS], [SEP] 등)을 이용하여 PLM(30)의 입력을 구성한다.Referring to FIG. 3 , processor 120 uses PLM 30 to extract high-level semantic features of data. Processor 120 has N samples

and special tokens (for example, [CLS], [SEP], etc.) to configure the input of the PLM (30).

를 출력한다.The PLM 30 has features such as Equation 1

outputs

[Equation 1]

은

의 토큰 집합을 의미하고, M은 토큰의 수를 의미한다. 그리고,

는 [CLS] 토큰에 대응하는 마지막 은닉 상태 벡터(last hidden state vector)를 의미한다.Here, [CLS] represents the entire sentence, and [SEP] is a special token that distinguishes each sentence.

silver

means a set of tokens, and M means the number of tokens. and,

Means the last hidden state vector corresponding to the [CLS] token.

The clustering method according to the present embodiment aims to apply contrast learning without using data augmentation that may cause semantic distortion. The processor 120 may encode an input sentence using the PLM 30 and generate a similarity matrix 310 using a vector dot product.

을

의 조밀한 문맥 표현이라 하자. 프로세서(120)는 인스턴스 쌍이 동일한 클러스터(예를 들면, 동일한 의도)에 속하는지 여부를 나타내는 유사도 행렬(310)을 계산할 수 있다. 문맥 표현에 대한 유사도 행렬(310)은 수학식 2와 같이 정의될 수 있다.

second

Let be a dense contextual representation of The processor 120 may calculate a similarity matrix 310 indicating whether a pair of instances belong to the same cluster (eg, the same intent). The similarity matrix 310 for the context expression may be defined as in Equation 2.

[Equation 2]

는 동일한 인스턴스가 포지티브 쌍으로 선택되는 것을 막기 위한 무한수(infinite number)를 의미한다.

는 N×N 차원을 가진 유사도 행렬을 나타내고,

은

와

사이의 유사성을 나타낸다. 본 실시예에서는 유사도 함수로 정규화 및 차원성 감소가 없는 표현의 내적을 사용한다.here,

means an infinite number to prevent the same instance from being selected as a positive pair.

denotes a similarity matrix with dimensions N×N,

silver

and

indicate similarities between them. In this embodiment, the dot product of an expression without normalization and dimensionality reduction is used as the similarity function.

와 가장 유사한 샘플은 포지티브 샘플로 나타내고 나머지 샘플은 네거티브 샘플로 나타낸다. 본 실시예에서는 수학식 3과 같이 정의된 교차 엔트로피 손실을 사용한다.except for myself

The sample most similar to is designated as a positive sample and the remaining samples are designated as negative samples. In this embodiment, the cross entropy loss defined as in Equation 3 is used.

[Equation 3]

은

인 경우 1을 출력하고 그 외 0을 출력하는 지시 함수(indicator function)를 의미하고,

는 모델이 예측하기 어려운 하드 네거티브들에서 학습하는 데에 도움이 되는 확률 조절 파라미터(temperature parameter)를 의미한다.here,

silver

means an indicator function that outputs 1 in case of , and outputs 0 otherwise,

denotes a temperature parameter that helps the model learn from hard negatives that are difficult to predict.

Unlike instance-level collational learning, the present invention can learn features suitable for clustering by explicitly grouping data instances with the same intent as a result of Equations 2 and 3.

4 illustrates an example of a process of generating a similarity matrix for an auxiliary expression according to an embodiment of the present invention.

Unsupervised collated learning according to the present invention is more advantageous than collated learning based on data augmentation. Contrastive learning based on data augmentation pushes out other instances regardless of semantic similarity, whereas unsupervised collegial learning according to the present invention can group different instances together in consideration of semantic similarity.

There is a limit to extracting positive samples from unlabeled data using only similarity between data representations without fine tuning. The PLM 30 may output a vector that does not sufficiently represent an instance at the beginning of learning when sufficient learning is not performed on the target domain. This can lead to incorrect sample selection as well as similarity calculations. Incorrect selection of positive samples in the early stages can include noise in training, and as the training progresses, noise accumulates continuously, which can degrade model performance.

In order to alleviate this problem, in the present embodiment, an auxiliary expression capable of supplementing the context expression encoded through the PLM 30 may be used. As an example, the processor 120 may generate an auxiliary representation by applying a sparse word representation that ignores the order of words and focuses on the presence or absence of a word and the importance of the word in the dataset. For auxiliary representation, in the case of text type data, BoW (bag of words), TF-IDF (Term Frequency - Inverse Document Frequency), etc. may be used. For example, unigram TF-IDF may be used.

In this embodiment, the data classification target is described as text data, but is not limited thereto. For example, in the case of an image, the auxiliary expression is a depth map or a feature map between images instead of a deep learning methodology ), etc. can be applied to various techniques that can extract features necessary to construct a similarity matrix from an image. In other words, it can be applied to contrast learning for the purpose of image clustering by combining a deep learning-based similarity matrix and a new similarity matrix composed of a methodology specific to images in addition to deep learning.

Auxiliary representations can explicitly indicate similarity between instances, regardless of semantic meaning, by comparing word frequencies. Similarity based on word frequency can guide the model to select appropriate positive samples at an early stage of training, so that auxiliary representations can reduce noise at an early stage of learning.

An auxiliary expression can be defined as in Equation 4.

[Equation 4]

는 단어 크기를 의미한다.

을

의 보조 표현이라 한다.here,

stands for word size.

second

is called the auxiliary expression of

The similarity matrix 420 for the auxiliary expression can be defined as Equation 5.

[Equation 5]

는 동일한 인스턴스가 포지티브 쌍으로 선택하는 것을 막기 위한 무한수를 의미한다.

는 N×N 차원을 가진 유사도 행렬을 나타내고,

은

와

사이의 유사성을 나타낸다.here,

means an infinite number to prevent the same instance from choosing as a positive pair.

denotes a similarity matrix with dimensions N×N,

silver

and

indicate similarities between them.

5 illustrates an example of a process of generating a final similarity matrix for collation learning in one embodiment of the present invention.

Referring to FIG. 5 , the processor 120 may generate a final similarity matrix 530 for collation learning by combining a similarity matrix 310 for a context expression and a similarity matrix 420 for an auxiliary expression. For example, the processor 120 multiplies and scales at least one of the similarity matrix 310 for the context expression and the similarity matrix 420 for the auxiliary expression by an appropriate value, and then adds the two similarity matrices 310 and 420 A final similarity matrix 530 may be constructed.

As an example, the final similarity matrix 530 may be defined as Equation 6.

[Equation 6]

은 문맥 표현에 대한 유사도 행렬(310)

와 보조 표현에 대한 유사도 행렬(420)

간의 스케일 조정을 위한 하이퍼 파라미터를 의미한다.here,

is the similarity matrix 310 for the context representation

and the similarity matrix 420 for auxiliary representations

It means a hyperparameter for adjusting the scale of the liver.

를

에 곱하여

의 스케일을 조정한 후

와 더하는 방식으로 최종 유사도 행렬(530)을 생성할 수 있다.Processor 120

cast

multiplied by

After scaling the

The final similarity matrix 530 may be generated by adding .

를 계산할 수 있으며, 이때

는 수학식 7과 같이 정의될 수 있다.Processor 120 every learning

can be calculated, where

Can be defined as in Equation 7.

[Equation 7]

는

의 표준 편차(standard deviation)를 의미하고,

는

의 평균 값을 의미한다.here,

Is

means the standard deviation of

Is

mean the average value of

에 곱한 후 보조 표현에 대한 유사도 행렬(420)

에 반영할 수 있다.In order to reduce the influence of the auxiliary expression, the processor 120 uses a decay coefficient for reducing the learning rate for the auxiliary expression as a scaling parameter.

After multiplying by , the similarity matrix for the secondary representation (420)

can be reflected in

In other words, the attenuation coefficient aims to learn the PLM 30 itself while gradually reducing the influence of the auxiliary expression as learning progresses. For example, the initial value of the attenuation coefficient may be set to 1, and may be reduced by 0.9 when 1 epoch learning is performed on all data sets.

6 illustrates an example of a collation learning process using a similarity matrix in an embodiment of the present invention.

Referring to FIG. 6 , the processor 120 selects the data having the highest value row by row for the final similarity matrix 530 as positive samples and selects the remaining data as negative samples, and then performs collational learning on the model. . In other words, processor 120 may extract positive samples for each data instance and then apply a contrast loss. By learning to get closer to positive samples and farther away from negative samples, if you bring them closer to other instances in the batch selected as positive pairs, you can consider not only the distance within the cluster but also the distance between clusters. Cross-entropy loss may be used for contrast learning, but is not limited thereto.

7 illustrates an example of a clustering process using PLM learned by unsupervised collegial learning in one embodiment of the present invention.

Referring to FIG. 7 , the processor 120 may perform clustering in which instances having the same label (eg, intention) are grouped by learning characteristics suitable for clustering. The processor 120 extracts an expression from the PLM 30 learned for given input data and applies various clustering algorithms to classify the corresponding expression (eg, what kind of intention the corresponding expression corresponds to).

For example, the K-mean clustering algorithm can be one of the algorithms for optimizing the cost function of Equation 8.

[Equation 8]

는 0이 아닌 요소가 하나만 있는 할당 벡터(assignment vector)를 의미한다.

는

의 j번째 요소를 나타내며, W의 k번째 열은 k번째 클러스터의 센트로이드(centroid)를 나타낸다.Here, K denotes a predefined number of clusters,

denotes an assignment vector with only one non-zero element.

Is

represents the jth element of W, and the kth column of W represents the centroid of the kth cluster.

As described above, according to embodiments of the present invention, unsupervised collation learning can be performed without using data augmentation by extracting positive samples by utilizing similarity between data instances.

And, according to embodiments of the present invention, sparse representations (auxiliary representations) of a broader category that guide the model to extract appropriate positive samples are used as input representations together with dense contextual representations, resulting in Instability in the early learning stage can be alleviated.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), and a programmable PLU (programmable logic unit). logic unit), microprocessor, or any other device capable of executing and responding to instructions, one or more general purpose or special purpose computers. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. The software and/or data may be embodied in any tangible machine, component, physical device, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. there is. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. In this case, the medium may continuously store a program executable by a computer or temporarily store the program for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, and ROM, RAM, flash memory, etc. configured to store program instructions. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (14)

A data classification method executed on a computer device,
The computer device includes at least one processor configured to execute computer readable instructions contained in a memory;
The data classification method,
generating, by the at least one processor, a first similarity matrix for a contextual representation of given data;
generating, by the at least one processor, a second similarity matrix for an auxiliary representation complementing the context representation;
generating, by the at least one processor, a third similarity matrix by combining the first similarity matrix and the second similarity matrix; and
Performing, by the at least one processor, model learning for data classification through contrastive learning using the third similarity matrix.
Data classification method comprising a. According to claim 1,
The step of performing the model learning,
Performing the contrast learning by extracting positive samples based on similarity between data for label-free data classification
Characterized by a data classification method. According to claim 1,
The step of performing the model learning,
selecting positive samples and negative samples row-wise for the third similarity matrix; and
Performing the contrast learning by applying cross entropy loss to the positive samples and the negative samples.
Data classification method comprising a. According to claim 1,
Generating the first similarity matrix,
Extracting a dense contextual representation by encoding the data through a pretrained language model (PLM)
Data classification method comprising a. According to claim 1,
Generating the first similarity matrix,
generating the first similarity matrix using a dot product of vectors with respect to the context representation;
Data classification method comprising a. According to claim 1,
Generating the second similarity matrix,
generating the auxiliary representation as a sparse representation representing the presence of data regardless of the order of the data;
Data classification method comprising a. According to claim 1,
Generating the second similarity matrix,
Generating the auxiliary expression using TF-IDF (Term Frequency-Inverse Document Frequency) when text type data is given
Data classification method comprising a. According to claim 1,
Generating the second similarity matrix,
Generating the auxiliary expression using a depth map or feature map for an image when data of an image type is given
Data classification method comprising a. According to claim 1,
Generating the third similarity matrix,
Determining a hyperparameter for scaling between the first similarity matrix and the second similarity matrix using the standard deviation of the first similarity matrix and the average of the second similarity matrix
Data classification method comprising a. According to claim 9,
Generating the third similarity matrix,
Setting a decay coefficient for the second similarity matrix according to learning progress
Data classification method comprising a. A computer program stored in a computer readable recording medium in order to execute the data classification method of any one of claims 1 to 10 in the computer device. In a computer device,
at least one processor configured to execute computer readable instructions contained in memory;
including,
The at least one processor,
generating a first similarity matrix for a contextual representation of given data;
generating a second similarity matrix for an auxiliary expression supplemented with the context expression;
generating a third similarity matrix by combining the first similarity matrix and the second similarity matrix; and
A process of performing model learning for data classification through contrast learning using the third similarity matrix
A computer device that processes According to claim 12,
The at least one processor,
Performing the contrast learning by extracting positive samples based on similarity between data for classification of unlabeled data
Characterized by a computer device. According to claim 12,
The at least one processor,
selecting positive samples and negative samples in row units from the third similarity matrix; and
Performing the contrast learning by applying cross entropy loss to the positive samples and the negative samples.
Characterized by a computer device.

Images