CN113571177A - On-line forum user depression detection model based on gated convolution network - Google Patents

Abstract

The invention discloses a method for detecting depression of online forum users based on a gated convolutional network. A model for detecting depression of online paper users based on a gated convolutional network is used to realize the detection. The detection of the model is realized through post-level operations and user-level operations. The post-level representation is obtained through post-level operations; the user state representation is obtained through user-level operations, and then the probability distribution on the label is output by the softmax layer to achieve detection. The method of the present invention can use gating weights to effectively identify language associated with negative emotions in user posts, thereby enabling depression detection.

Description

A Depression Detection Model for Online Forum Users Based on Gated Convolutional Networks

technical field

The invention relates to the technical field of depression detection, in particular to an online forum user depression detection model based on a gated convolutional network.

Background technique

Depression is one of the most common mental health-related disorders and a leading cause of self-harm and suicide worldwide and affects millions of people. Early detection and treatment of depression can reduce damage caused by the disorder. But services for early detection and treatment of depression and other mental illnesses are very limited. And many patients are reluctant to proactively seek help from a healthcare provider. These problems keep patients from getting timely treatment, which can lead to further deterioration of the condition.

More and more people with depression are using online resources (Twitter, website, Reddit, etc.) to express their psychological problems and seek help. In particular, some online forums that can choose to be anonymous or anonymous are more popular. The early detection of depression task using social media data has emerged as an effective means. At the same time, the massive amount of social media data makes it difficult to manually identify users with depression or suicidal tendencies, making the development of automated depression detection technology even more critical.

In the forum, there are often a large number of posts with sensitive content, indicating that users are at risk of suicide and self-harm. Early depression detection using appropriate deep learning models and social media data prevents potential self-harm. But the existing depression detection models are not powerful enough to capture key emotional information from the large number of posts made by each user, which makes the performance of these models unsatisfactory.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an online forum user depression detection model based on gated convolution network for the technical defects existing in the prior art, and to provide an online forum user depression detection model based on gated convolution network for depression detection method of detection.

The technical scheme adopted for realizing the purpose of the present invention is:

A method for detecting depression in online forum users based on gated convolutional networks, including:

A model for depression detection of online paper users based on gated convolutional network is used to achieve detection. The detection steps of this model are as follows:

S1. Post-level actions

The input of the model is passed through a multi-layer convolutional neural network with gated units, and the convolutional neural network utilizes limited contextual information to obtain the key features of the post representation;

Among them, the processing layers of post-level operations consist of two convolutional layers and a global average pooling; the first convolutional layer uses two convolutional kernels of different sizes to obtain abstract feature maps, and then, the second convolutional layer with gated units The product layer uses convolution kernels to obtain different gating weights, and applies the gating weights to the element-wise product of the feature map generated by the first convolution layer to obtain post-level representations;

Each word is represented by a word embedding matrix L _ω ∈ R ^d×|V| , |V| is the number of words in the vocabulary, d is the dimension of the word vector, and each post is represented by n words, respectively {ω ₁ , ω ₂ , ... ω _i ... ω _n }, let x _i ∈ R ^d be the d-dimensional word vector corresponding to the ith word in the post, the post embedding of length n is expressed as

X _{1 : n} = [x ₁ , x ₂ , . . . , x _n ]

In the first convolutional layer, a CNN and multiple convolutional filters of different widths are used to generate a representation of the post; convolutional filters with different widths are treated as multi-granular local information obtained by the extractor; different window sizes are used of multiple convolutional filters to obtain multiple feature maps;

Let K∈R ^s×d be a convolutional filter with stride 1 applied to a window of s words to produce new features, [x _i+1 , x _i+2 , ..., x _{i+s -1} ] represents the word embedding in a window with a fixed length of s, concatenate the two vectors, namely X _{i: i+s-1} , and generate a new feature a _i from X _{i: i+s-1}

a _i =f(K*X _i:i+s+1 +b)

where b∈R is the bias term, * denotes the convolution operation, and f is the activation function LeakyReLU, the filter applied to the posts [X1 _:s , X2 _:s+1 ,...,Xn _{-s+ 1:n} ] s words for each possible window to generate feature map A,

A=[a ₁ , a ₂ , ..., an _-s+1 ]

Among them, A∈R ^(n-s+1)×1 means that all feature maps are obtained through filters of different sizes, and then each feature map A is passed as an input to the second convolutional layer;

The second convolutional layer includes convolutional layers and gating units, which are used to generate different gating weights, and a convolution operation with a kernel F∈R ^{h × 1} , which is used to extract the feature map A, and the convolution kernel F is a window h slides on feature a ₁ to generate gating weights;

Among them, g _l ∈ R, l=1,2,...,n-s+1, all gating weight elements are generated by the feature map A and the convolution kernel F, and form the gating weight matrix G:

G=[g ₁ , g ₂ , ..., g _n-s+1 ]

Among them, G∈R ^(n-s+1)×1 , m is the number of convolution kernels in the second convolutional layer, and the gating unit is used to extract different gating weight matrices G ₁ , G ₂ ,... , G _m , and then obtain the output feature map O by gating the weight matrix G:

是矩阵之间的按元素乘积，O∈R^m×(n-s+1)；in,

is the element-wise product between matrices, O∈R ^m×(n-s+1) ;

The output feature map O of the first convolutional layer is controlled by the gating weight matrix G;

Then input the output feature map O of the second convolutional layer to the global average pooling layer and concatenate all outputs to obtain the representation of the post;

S2. User-level operations

Send the obtained post representation input to the user-level operation processing, use the same method as the post-level operation to obtain the user state representation, and then pass the obtained user state representation to the fully connected softmax layer, which outputs the labels The probability distribution on , to achieve detection; the model loss function uses the classification cross entropy, the target sentiment distribution of each document is recorded as p ^T , and p is the predicted document sentiment distribution;

Among them, T is the training data, C is the number of categories, i is the index of the document, j is the category index, and the purpose of training is to minimize the error of the cross-entropy between p ^T and p of all training documents.

The difference between the inventive model and the benchmark was statistically significant (McNemar test, p<0.05). The inventive model was compared to multiple benchmarks using MNB and SVM classifiers with two sets of functions. The proposed MGL-CNN model for depressed users outperforms the baseline in precision, recall, and F1 (increases by 6.8%, 6.7%, and 5.9%, respectively). It turns out that the method can use gating weights to effectively identify language associated with negative sentiment in user posts.

Description of drawings

Fig. 1 is the overall structure diagram of the present invention's depression detection model (MG-CNN) for online paper users based on gated convolutional network;

FIG. 2 is a structural diagram of a post-level operation model of the present invention.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

As shown in Figures 1-2, the method for detecting depression of online forum users based on gated convolutional network of the present invention utilizes a model for detecting depression of online paper users based on gated convolutional network to realize detection. The detection steps of the model are as follows :

Step 1: Post-level actions

The input to the model is passed through multiple layers of convolutional neural networks with gated units, which utilize limited contextual information to obtain key features of post representations.

Post-level operations in MG-CNN, consisting of two convolutional layers and a global average pooling. In the demonstration model of post-level operations in MG-CNN, the first convolutional layer uses two convolutional kernels of different sizes to obtain abstract feature maps. Then, the second convolutional layer with gating units uses convolution kernels to obtain different gating weights. These gating weights are applied to perform element-wise multiplication with the feature map generated by the first convolutional layer to obtain a post-level representation.

Each word is represented by a word embedding matrix L _ω ∈ R ^d×|V| , where |V| is the number of words in the vocabulary and d is the dimension of the word vector. Each post of the user is represented as n words, respectively {ω ₁ , ω ₂ , ... ω _i ... ω _n }, let x _i ∈ R ^d be the same as the ith word in the post (English word) The corresponding d-dimensional word vector, such as Iam depress, should become [0, 0, 1], [0, 1, 0][0, 1, 1], and the dimension is 3. Posts of length n are embedded as

X _{1 : n} = [x ₁ , x ₂ , . . . , x _n ] (1)

In the first convolutional layer, a CNN and multiple convolutional filters of different widths are used to generate the representation of the post. Convolutional filters with different widths can be considered as extractors to obtain multi-granularity local information such as N-Grams. Likewise, a convolutional filter of width 2 essentially captures the semantics of bigrams in user posts. Use multiple convolutional filters with different window sizes to obtain multiple feature maps. Let K ∈ R ^{s × d} be a convolutional filter with stride 1 applied to a window of s words to produce new features. [x _i+1 , x _i+2 , . . . , x _i+s-1 ] represents the word embedding in a window of fixed length s, concatenating the two vectors, namely X _{i: i+s-1} . Generate a new feature a _i by X _{i: i+s-1}

a _i =f(K*X _i:i+s+1 +b) (2)

where b∈R is the bias term, * denotes the convolution operation, and f is the activation function (LeakyReLU). This filter is applied to the s words of each possible window in posts [X1 _:s , X2 _:s+1 ,...,Xn _-s+1:n ] to generate feature map A,

A=[a ₁ , a ₂ , ..., an _-s+1 ] (3)

where A∈R ^(n-s+1)×1 . All feature maps are obtained through filters of different sizes, and then each feature map A is passed as input to the second convolutional layer.

The second convolutional layer consists of convolutional layers and gating units. This layer is used to generate different gating weights. It contains a convolution operation with a kernel F ∈ R ^h×1 , which is mainly used to extract the contextual feature A. The convolution kernel F slides on the feature a ₁ with a window h to generate the gating weights.

where g _l ∈ R, l=1,2,...,n-s+1. All gating weight elements are generated by feature map A and convolution kernel F, and form a gating weight matrix:

G=[g ₁ , g ₂ , ..., g _n-s+1 ] (5)

where G∈R ^(n-s+1)×1 . Let m be the number of convolution kernels used in the second convolutional layer. Different gating weight matrices are extracted using the gating unit of MGL-CNN: G ₁ , G ₂ , ..., G _m . Then, the output feature map O is obtained by gating the weight matrix G:

是矩阵之间的按元素乘积。在MG-CNN中，O∈R^m×(n-s+1)。in

is the element-wise product between matrices. In MG-CNN, O∈Rm ^×(n-s+1) .

The output O of the first convolutional layer is controlled by the gating weight G. These gating weights are multiplied by the feature map A and control what information should be propagated in the layers. To capture the global information of the post, the output of the second convolutional layer is then fed into the global average pooling layer, and all outputs are concatenated to obtain the representation of the post.

Step 2: User-level operations

The obtained post representation input is sent to the user-level operation, and the user's state representation is obtained using the same method as the post-level operation. Then, the obtained user features are passed to a fully connected softmax layer whose output is a probability distribution over the labels. The loss function of the model uses categorical cross entropy. The target sentiment distribution of each document is denoted as p ^T , where p is the predicted document sentiment distribution.

where T is the training data, C is the number of categories, i is the index of the document, and j is the category index. The goal of training is to minimize the error of the cross-entropy between p ^T and p of all training documents.

Experimental verification:

The experiments are based on the Reddit Self-Reported Depression Diagnosis (RSDD) dataset and the Early Detection of Depression dataset (eRisk 2017). The performance of the proposed model is evaluated by comparison with other strong baseline models, and the performance of the model is analyzed.

The RSDD dataset consists of training, validation and testing datasets, each containing approximately 3,000 diagnosed users and 35,000 control users. Use the validation set to tune the hyperparameters of the model and baseline. eRisk 2017 consists of training and testing datasets. The training set of eRisk 2017 contains 83 depressed users and 403 control users, while the test set contains 52 depressed users and 349 control users.

The RSDD validation set was used to select hyperparameters for the depression detection model, and the test set was used to report the results. Embedding layers are initialized without pre-trained (like publicly available Glove) embeddings. The raw encoding of the input to the depression model consists of one-hot vectors, and then the input layer is used to learn 50- and 100-dimensional embeddings. The learning rate (lr) is set to 0.001. For RSDD, eRisk2017, set the batch size to 64 and 128, respectively. Define Maxm as the maximum number of posts per user and Maxn as the maximum number of features per post. When a user's document exceeds the maximum number of posts, the posts will be randomly scrambled, then the posts will be randomly selected and the length of Maxm will be intercepted. For example, set the model to receive up to 600 posts (Maxm=600) in the RSDD dataset, where each post contains up to 100 words (Maxn=100). The depression detection model proposed by the present invention is implemented with Keras framework.

For MG-CNN post-level operations, the window sizes of the convolution kernel(s) of the first convolutional layer are set to 2, 3, 4, 5, and 6, respectively, and each window size has 30 different convolution kernels. In the second convolutional layer, the convolution kernels (h) have window sizes of 1, 3, 5, and 7, and each window size has 30 different convolution kernels. A convolution kernel with a window size of 3 is used in the second layer of SG-CNN. For the user-level operations of the model, the same parameters are set as for the post-level operations, without any specific adjustments to the dataset.

The model of the present invention uses the softmax function and the classification cross entropy as its loss function, and handles the class balance problem by means of "classification cross". All models are trained by stochastic gradient descent using the Adam optimizer.

Results for identifying depressed users from methods and other benchmarks in RSDD are shown in the table above. The difference between the model of the present invention and the benchmark was statistically significant (McNemar test, p<0.05). The inventive model was compared to multiple benchmarks using MNB and SVM classifiers with two sets of functions.

The MGL-CNN model proposed by the present invention for depressed users outperforms the baseline in terms of precision, recall and F1 (increases by 6.8%, 6.7% and 5.9%, respectively). It turns out that the method can use gating weights to effectively identify language associated with negative sentiment in user posts.

The results of the model and the current best-performing methods on the depression early detection dataset are shown in the table above. Absolute values of metrics from baseline suggest that early detection of depression tasks is difficult.

As far as F1 is concerned, the performance is low, with a top F1 of 0.64. Some methods (like FHDO-BCSGB) choose to optimize for precision but lower recall, while others (like UNSLA) choose to optimize for recall but lower precision. This may be related to the size and creation of the dataset. The performance of the proposed model (MGL-CNN) in terms of precision, recall and F1 on depressed users is close to several state-of-the-art methods.

Furthermore, the inventive model is not designed to improve one metric like the baseline model, but performs well on all three metrics (Precision, Recall and F1). The results of the present model and the state-of-the-art method show that the proposed general neural network architecture can also be used for early detection of depression in different online forums.

The above are only the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. These improvements and Retouching should also be considered within the scope of protection of the present invention.

Claims (4)

1. The method for detecting the depression of the online forum user based on the gated convolutional network is characterized in that the model for detecting the depression of the online thesis user based on the gated convolutional network is used for realizing detection, and the detection steps of the model are as follows:

s1, post level operations

Inputting the model through a multilayer convolutional neural network with a gate control unit, wherein the convolutional neural network acquires key features represented by posts by using limited context information;

the processing layer of the post level operation comprises two convolution layers and a global average pool; the first convolutional layer obtains an abstract feature map by using two convolution kernels with different sizes, then a second convolutional layer with a gating unit obtains different gating weights by using the convolution kernels, and the gating weights and the feature map generated by the first convolutional layer are subjected to element-by-element multiplication to obtain a post-level representation;

each word is embedded by a word into a matrix L_ω∈R^d×|V|Where | V | is the number of words in the vocabulary, d is the dimension of the word vector, and each post is represented as n words, each being { ω [ ]₁，ω₂，...ω_i...ω_nLet x_i∈R^dFor the d-dimensional word vector corresponding to the ith word in the post, the post of length n is shown embedded as

X_1：n＝[x₁，x₂，...，x_n]

In the first convolutional layer, generating a representation form of the post by using a CNN and a plurality of convolutional filters with different widths; treating convolution filters with different widths as extractors to obtain multi-granularity local information; using a plurality of convolution filters of different window sizes to obtain a plurality of feature maps;

let K be an element of R^s×dIs a convolution filter of step size 1 applied to a window of s words to generate a new feature, [ x ]_i+1，x_i+2，...，x_i+s-1]Meaning word embedding in a window of fixed length s, concatenating two vectors, i.e. X_i：i+s-1From X_i：i+s-1Generating a new feature a_i

a_i＝f(K*X_i：i+s+1+b)

Where b ∈ R is a bias term, representing a convolution operation, and f is the activation function LeakyReLU, the filter is applied to the post [ X ∈ R_1：s，X_2：s+1，...，X_n-s+1：n]The s words in each possible window in the feature map a are generated,

A＝[a₁，a₂，...，a_n-s+1]

wherein A ∈ R^(n-s+1)×1Representing all feature maps obtained by filters of different sizes, and then transmitting each feature map A as input to the second convolutional layer;

the second convolutional layer comprises convolutional layer and gating unit for generating different gating weights, including a core F e R^h×1Is used for extracting the feature map A, and the convolution kernel F is arranged at the feature a by a window h₁Sliding up to generate gating weights;

wherein, g_lE R, l 1, 2.., n-s +1, all gating weight elements are generated by the feature map a and the convolution kernel F, and constitute a gating weight matrix G:

G＝[g₁，g₂，…，g_n-s+1]

wherein G ∈ R^(n-s+1)×1M is the number of convolution kernels in the second convolution layer, and different gating weight matrixes G are extracted by using the gating units₁，G₂，...，G_mThen, obtaining an output characteristic diagram O through a gating weight matrix G:

wherein,

is the product by element between matrices，O∈R^m×(n-s+1)；

The output characteristic diagram O of the first convolution layer is controlled by a gating weight matrix G;

then inputting the output feature graph O of the second convolutional layer into the global average pooling layer and connecting all the outputs to obtain the representation of the post;

s2, user-level operation

The obtained post representation input is sent to user level operation processing, a user state representation is obtained by using the same method as the post level operation, then the obtained user state representation is transmitted to a fully connected softmax layer, and the probability distribution on a label is output by the softmax layer, so that detection is realized; the model loss function uses the classification cross entropy, and the target emotion distribution of each document is marked as p^TP is the predicted document emotion distribution;

wherein T is training data, C is classification number, i is document index, j is class index, and training is to make p of all training documents^TAnd p minimizes the error in cross entropy.

2. The gated convolutional network-based online forum user depression detection method as claimed in claim 1, wherein said model is implemented with a Keras framework.

3. The gated convolutional network-based online forum user depression detection method as claimed in claim 2, wherein for MG-CNN post level operation, the window sizes of the convolutional kernels in the first convolutional layer are set to 2, 3, 4, 5, 6, respectively, each window size having 30 different convolutional kernels; the window sizes of the convolution kernels in the second convolutional layer are 1, 3, 5, 7, and each window size has 30 different convolution kernels; a convolution kernel with a window size of 3 is used in the second layer.

4. The gated convolutional network-based online forum user depression detection method as claimed in claim 3, wherein for user-level operations, the same parameters as for post-level operations are set.

Images