CN114818786B - Channel screening method, emotion recognition system and storage medium - Google Patents

Abstract

The application discloses a channel screening method, a mood recognition system and a storage medium. The method comprises the steps of obtaining multichannel electroencephalogram signals and preprocessing; obtaining the frequency domain characteristics of each channel and the brain connection characteristics between every two channels; thereby obtaining a connection-channel characteristic; inputting each connection-channel characteristic and emotion label into a width learning system to obtain a recognition result and verifying the recognition result to obtain average test accuracy; and sorting and screening all the connection-channel characteristics according to the average test accuracy to obtain a key connection-channel characteristic set, thereby obtaining an important channel. Extracting important frequency domain features and important brain connection features from the important channels, mapping the important frequency domain features and the important brain connection features through an AEKM algorithm to obtain AEKM frequency domain features and AEKM brain connection features, and splicing the AEKM frequency domain features and the AEKM brain connection features to obtain fusion features; and then inputting the emotion labels and the emotion labels into a width learning system for cross-test emotion recognition, so as to obtain the classification accuracy.

Description

Channel screening method, emotion recognition method, system and storage medium

Technical field

This application relates to but is not limited to the technical field of emotion recognition, and in particular relates to a channel screening method, emotion recognition method, system and storage medium.

Background technique

As one of the most basic psychological processes of human beings, emotional changes play a vital role in daily life. With the rapid development of human-computer interaction technology, researchers are paying more and more attention to enhancing the multiple capabilities of computers in identifying, interpreting, processing, and simulating human emotions.

The emotion recognition EEG data and emotion recognition methods in related technologies mainly use all the electrode channels of the EEG cap to collect data and use fusion features to build a model within a single subject. When collecting data, the subject needs to wear the EEG cap. EEG caps containing dozens or even hundreds of electrodes collect signal data, which requires a lot of time and manpower to collect data.

Contents of the invention

This application aims to solve at least one of the technical problems existing in the prior art. To this end, this application proposes a channel screening method, emotion recognition method, system and storage medium, which can screen the channels of EEG signals and reduce the number of channels, thereby reducing the time and cost spent in collecting data.

The first embodiment of the present application provides a channel screening method, including:

Acquire multi-channel EEG signals and preprocess the EEG signals;

Obtain the frequency domain characteristics of each channel according to the preprocessed EEG signal;

Obtain the brain connection characteristics between each two channels according to the preprocessed EEG signals;

According to the frequency domain features and the brain connection features, connection-channel features are obtained; wherein each connection-channel feature includes one of the brain connection features and two of the frequency domain features, and the brain connection features correspond to The two channels respectively correspond to the two frequency domain features;

Input each connection-channel feature and emotion label into the width learning system, obtain the recognition result and verify the recognition result, and obtain the average test accuracy corresponding to each connection-channel feature;

Sort all the connection-channel features according to the average test accuracy, and filter according to the preset proportion threshold to obtain a key connection-channel feature set;

According to the key connection-channel feature set, important channels are obtained.

The channel screening method according to the embodiment of the first aspect of the present application has the following technical effects: the channel screening method of the embodiment of the present application acquires multi-channel EEG signals and preprocesses the EEG signals; and then based on the preprocessed The frequency domain characteristics of each channel are obtained from the EEG signal; the brain connection characteristics between each two channels are obtained based on the preprocessed EEG signal; the connection-channel characteristics are obtained based on the frequency domain characteristics and brain connection characteristics; among them, each A connection-channel feature includes a brain connection feature and two frequency domain features. The two channels corresponding to the brain connection feature correspond to two frequency domain features respectively; input each connection-channel feature and emotion label into the width learning system, Obtain the recognition results and verify the recognition results, and obtain the average test accuracy corresponding to each connection-channel feature; according to the average test accuracy, sort all connection-channel features, filter according to the preset proportion threshold, and obtain the key connections -Channel feature set; obtain important channels based on key connections - channel feature set. In this way, the screening of EEG signal channels is completed and the number of channels is reduced, thereby reducing the time and cost spent in collecting data.

According to some embodiments of the first aspect of the present application, preprocessing the EEG signal includes:

The EEG signal is filtered through 5 different frequency bands.

According to some embodiments of the first aspect of the present application, obtaining the frequency domain characteristics of each channel based on the preprocessed EEG signal includes:

Features are extracted from the EEG signals of each frequency band of each channel through the first formula to obtain corresponding frequency band differential entropy features. Each frequency domain feature includes the frequency band differential entropy corresponding to 5 different frequency bands. Characteristics; wherein, the first formula is specifically:

Wherein, the DE represents the differential entropy characteristics of the frequency band, σ ² is the variance of the corresponding EEG signal, π is a constant, and e is Euler’s constant.

According to some embodiments of the first aspect of the present application, obtaining the brain connection characteristics between each two channels based on the preprocessed EEG signals includes:

The instantaneous phase of the EEG signal of each channel is calculated through the second formula; the second formula is:

Wherein, the total number of channels is C, i=1,2,...,C, φ _i (t) represents the instantaneous phase of the EEG signal of the i-th channel, t=1,2,3,..., T, T is the sampling point, s _i (t) represents the time series of the i-th channel, is the Hilbert transform of s _i (t);

Through the third formula, according to the instantaneous phase of the EEG signals of each two channels, the phase lag index characteristics between the EEG signals of each two channels are obtained, and the phase lag index characteristics are used as the brain connection characteristics. ;The third formula is:

Wherein, PLI _i,k represents the phase lag index characteristic between the EEG signal of the i-th channel and the EEG signal of the k-th channel, and φ _i (t) represents the EEG signal of the i-th channel. The instantaneous phase of the signal, φ _k (t) represents the instantaneous phase of the EEG signal of the k-th channel, T is the sampling point, t=1,2,3,...,t, i=1,2 ,3,…,C, k=1,2,3,…,C.

According to some embodiments of the first aspect of the present application, each connection-channel feature and emotion label are input into the width learning system, a recognition result is obtained and the recognition result is verified, and each connection-channel feature and emotion label are input to the width learning system, and the recognition result is verified, and each connection-channel feature and emotion label are input to the width learning system The average test accuracy corresponding to channel features includes:

Through the fourth formula, mapping characteristics are obtained according to the connection-channel characteristics; the fourth formula is:

X ^p =[Z ₁ ,…,Z _p ]

Among them, Z ^p represents the mapping feature set, Z _i represents the i-th group of mapping features, Characterization mapping function, p is the number of feature maps, /> is the weight,/> is the bias term,/> Characterizing connection-channel characteristics;

Through the fifth formula, the mapping feature set is sent to the enhancement node to obtain the enhanced features; the fifth formula is:

H ^q =[H ₁ ,…,H _q ]

Among them, H ^q represents the enhanced feature set, H _j represents the jth group of enhanced features, ξ _j represents the nonlinear activation function, q is the number of feature enhancements, is the weight,/> is the bias term, Z ^p represents the mapping feature set;

The enhanced feature set and the mapping feature set are concatenated to obtain a conversion feature set A = [Z ^p , H ^q ]; where A represents the conversion feature set, Z ^p represents the mapping feature set, H ^q Characterize the set of enhanced features;

According to the conversion feature set and the emotion label, the objective function of the width learning system is minimized to obtain a label space; the objective function is:

Among them, F represents the F-norm of the matrix, Approximation used for measurement accuracy,/> Characterization/> Regularization term, W represents the weight of the width learning system, Y represents the emotion label, λ represents the penalty factor, I represents the unit matrix, A represents the conversion feature set, A = [Z ^p , H ^q ], Z ^p represents The mapping feature set, H ^q represents the enhanced feature set; by solving the objective function, it is obtained:

W＝(λI+A ^T A) ^-1 A ^T Y

Verify according to the label space to obtain the average test accuracy corresponding to each connection-channel feature.

The second embodiment of the application provides an emotion recognition method, including the channel screening method described in any one of the embodiments of the first aspect of the application;

The emotion recognition method also includes:

Extract important frequency domain features and important brain connection features according to the important channels, map the important frequency domain features and the important brain connection features through the AEKM algorithm respectively, and obtain AEKM frequency domain features and AEKM brain connection features respectively;

The AEKM frequency domain features and the AEKM brain connection features are spliced to obtain fusion features. The fusion features and the emotion labels are input to the width learning system to perform cross-subject emotion recognition to obtain classification accuracy.

According to some embodiments of the second aspect of the present application, the important frequency domain features and the important brain connection features are mapped through the AEKM algorithm, respectively, to obtain the AEKM frequency domain features and the AEKM brain connection features, including:

According to the important channels, important features are obtained, specifically:

Among them, m∈{FD, BC}, X ^m is an important feature, when m = FD, x ^m is the important frequency domain feature; when m = BC, x ^m is the important brain connection feature, X ^m Each row of is composed of (x _i ^m ) ^T , is the i-th sample of m, n is the number of samples;

The important frequency domain features and the brain connection features are mapped to obtain AEKM features; where,

g _m (x ^m )∈R ^l×1

Among them, m∈{FD,BC}, g _m (x ^m ) represents the AEKM feature; l represents the dimension; when m=FD, g _m (x ^m ) is the AEKM frequency domain feature; when m=BC, g _m (x ^m ) is the AEKM brain connection feature; when m = FD, x ^m is the important frequency domain feature; when m = BC, x ^m is the important brain connection feature;

The important frequency domain features and the brain connection features are mapped to obtain AEKM features, including:

For the important features mentioned Pass/> Algorithm, obtain the set of marked points, specifically:

Among them, V ^m represents the set of marked points, It represents the jth marker point, l represents the dimension, j=1,2,…,l;

The first kernel matrix is constructed through the sixth formula; the sixth formula is:

Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the first kernel matrix, σ _m is the kernel parameter, and κ (.,.,.) is the kernel function;

The second kernel matrix is constructed through the seventh formula; the seventh formula is:

Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the second kernel matrix, σ _m is the kernel parameter, and κ (.,.,.) is the kernel function;

Perform eigenvalue decomposition on the second kernel matrix, specifically:

Among them, l represents the dimension, is a diagonal matrix, the diagonal elements are/> eigenvalues,/> The column vector of is the eigenvector corresponding to the eigenvalue;

Through the eighth formula, according to the first kernel matrix and the second kernel matrix, the AEKM feature matrix is obtained The eighth formula is:

Among them, n is the number of samples, l represents the dimension, G ^m represents the AEKM feature matrix, when m = FD, G ^m is the AEKM frequency domain feature matrix; when m = BC, G ^m is the AEKM brain connection feature matrix, Among them,/> is a diagonal matrix, the diagonal elements are/> eigenvalues,/> The column vector of is the eigenvector corresponding to the eigenvalue, and M ^m serves as the eigenvalue mapping matrix;

According to the AEKM frequency domain feature matrix and the AEKM brain connection feature matrix, a fusion feature matrix G ^F = [G ^BC , G ^FD ] is obtained, where G ^F represents the fusion feature matrix, and G ^FD represents the AEKM frequency domain feature matrix. Domain feature matrix, G ^BC represents the AEKM brain connection feature matrix.

The third embodiment of the present application provides a channel screening system, including: at least one first memory;

at least one first processor;

at least one program;

The program is stored in the first memory, and the first processor executes at least one program to implement:

The channel screening method as described in any one of the embodiments of the first aspect of this application.

The fourth embodiment of the present application provides an emotion recognition system, including: at least one second memory;

at least one second processor;

at least one program;

The program is stored in the second memory, and the second processor executes at least one program to implement:

The emotion recognition method as described in any one of the embodiments of the second aspect of this application.

The fifth aspect embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable signals. The computer-executable signals are used to execute:

The channel screening method as described in any one of the embodiments of the first aspect of this application; or,

The emotion recognition method as described in any one of the embodiments of the second aspect of this application.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of drawings

The present application will be further described below in conjunction with the accompanying drawings and examples, wherein:

Figure 1 is a step flow chart of a channel screening method according to some embodiments of the present application;

Figure 2 is a flow chart of steps for obtaining brain connection features according to some embodiments of the present application;

Figure 3 is a flow chart of steps for obtaining the average test accuracy corresponding to each connection-channel feature according to some embodiments of the present application;

Figure 4 is a step flow chart of an emotion recognition method according to some embodiments of the present application;

Figure 5 is a flow chart of steps for obtaining a fusion feature matrix according to some embodiments of the present application.

Detailed ways

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application and cannot be understood as limiting the present application.

In the description of this application, it should be understood that the orientation descriptions involved, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc. are based on the orientation or positional relationship shown in the drawings, and are only In order to facilitate the description of the present application and simplify the description, it is not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of the present application.

In the description of this application, several means one or more, plural means two or more, greater than, less than, exceeding, etc. are understood to exclude the original number, and above, below, within, etc. are understood to include the original number. If there is a description of first and second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the order of indicated technical features. relation.

In the description of this application, unless otherwise explicitly limited, words such as setting, installation, and connection should be understood in a broad sense. Those skilled in the art can reasonably determine the specific meaning of the above words in this application in conjunction with the specific content of the technical solution.

In the description of this application, reference to the description of the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" is intended to be in conjunction with the description of the embodiment. or examples describe specific features, structures, materials, or characteristics that are included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

As one of the most basic psychological processes of human beings, emotional changes play a vital role in daily life. With the rapid development of human-computer interaction technology, researchers are paying more and more attention to enhancing the multiple capabilities of computers to identify, interpret, process and simulate human emotions. Emotion recognition plays a vital role in human-computer interaction. There is an increasing number of studies on different aspects of emotion, and it has been found that there are many types of features related to emotions, such as frequency domain features and connection features of EEG signals. Multi-feature input can reflect different aspects of emotion and provide complementary information. For example, frequency domain features provide information within a channel, while brain connection features represent the relationship between channels, which is more conducive to establishing an accurate and robust emotion recognition model. Fusion of data from multiple features can improve the accuracy of emotion recognition. In research on emotion recognition at the subject level, more and more researchers are beginning to pay attention to research on emotion recognition across subjects. In the cross-subject experiment, in all sample data sets of all subjects, a cross-subject emotion recognition model was constructed through the "one-subject-leave cross-validation method". The "one-subject-leave cross-validation method" means to select one of the All samples of the subjects are used as the test set, and all samples of the remaining subjects are used as the training set. Compared with the model in which a single subject corresponds to a single recognition model, the cross-subject emotion recognition model has better generalization and higher robustness. After collecting new subject data, emotion recognition can be directly performed without having to re- The time to train the model for this subject can greatly reduce the time cost.

Emotional EEG data and emotion recognition methods in related technologies mainly use all the electrode channels of the EEG cap to collect data and use fusion features to build a model within a single subject. When collecting data, the subject needs to wear a EEG caps containing dozens or even hundreds of electrodes collect signal data, which requires a lot of time and manpower to collect data.

Based on this, this application proposes a channel screening method, emotion recognition method, system and storage medium, which can screen features, that is, screen the channels of EEG signals to reduce the number of channels, thereby reducing the time and time spent in collecting data. cost. The reduction in the number of channels means the reduction of electrodes. Compared with the experimental collection cost of all electrodes, using fewer and more effective electrodes to conduct experiments can save a lot of manpower and material resources, making the experiment more efficient.

In the first aspect, referring to FIG. 1 , an embodiment of the present application provides a channel screening method, including but not limited to step S100, step S200, step S300, step S400, step S500, step S600 and step S700.

Step S100, obtain multi-channel EEG signals and preprocess the EEG signals;

Step S200, obtain the frequency domain characteristics of each channel based on the preprocessed EEG signal;

Step S300, obtain the brain connection characteristics between each two channels based on the preprocessed EEG signals;

Step S400, obtain connection-channel features based on frequency domain features and brain connection features; wherein each connection-channel feature includes one brain connection feature and two frequency domain features, and the two channels corresponding to the brain connection feature are respectively associated with two Frequency domain feature correspondence;

Step S500, input each connection-channel feature and emotion label into the width learning system, obtain the recognition result and verify the recognition result, and obtain the average test accuracy corresponding to each connection-channel feature;

Step S600: Sort all connection-channel features according to the average test accuracy, filter according to the preset proportion threshold, and obtain a key connection-channel feature set;

Step S700: Obtain important channels based on key connection-channel feature sets.

The channel screening method in the embodiment of the present application acquires multi-channel EEG signals and pre-processes the EEG signals; then obtains the frequency domain characteristics of each channel based on the pre-processed EEG signals; based on the pre-processed EEG signals The electrical signal obtains the brain connection features between each two channels; based on the frequency domain features and brain connection features, the connection-channel features are obtained; among them, each connection-channel feature includes a brain connection feature and two frequency domain features. The two channels corresponding to the connection feature correspond to two frequency domain features respectively; input each connection-channel feature and emotion label into the width learning system, obtain the recognition result and verify the recognition result, and obtain the correspondence between each connection-channel feature The average test accuracy; according to the average test accuracy, all connection-channel features are sorted, and filtered according to the preset proportion threshold to obtain the key connection-channel feature set; based on the key connection-channel feature set, important channels are obtained. In this way, the screening of EEG signal channels is completed and the number of channels is reduced, thereby reducing the time and cost spent in collecting data. Moreover, according to the important channels, the corresponding important frequency domain features and important brain connection features can be obtained. The important frequency domain features and important brain connection features can be used for emotion recognition, and compared with the channels without filtering, after channel filtering With the obtained important frequency domain features and important brain connection features, the accuracy of emotion recognition results is higher.

It can be understood that the channel screening method in the embodiment of the present application uses an EEG cap to collect EEG signal data of a subject when watching movie clips with different emotional labels, and the EEG signal has multiple channels. In other embodiments, the existing public emotional EEG data set SEED is used as experimental data, in which the EEG cap used in this data set is a 62-channel EEG cap of the International 10-20 system, so the acquired EEG The signal consists of 62 channels.

It can be understood that in step S100, the EEG signal is preprocessed, including:

EEG signals are filtered through 5 different frequency bands.

Specifically, the sampling frequency of the original data is reduced to 200Hz. To remove irrelevant artifacts, all raw EEG signals were filtered with a bandpass filter ranging from 1 to 50 Hz. Then use 5 frequency bands to filter the EEG signal. The 5 frequency bands are delta(δ)(1-4Hz), theta(θ)(4-8Hz), alpha(α)(8-14Hz), beta(β) )(14-31Hz) and gamma(γ)(31-50Hz).

It is understandable that frequency domain features reflect energy differences in different brain regions under different emotions. Common frequency domain features include Power Spectral Density (PSD) features and Differential Entropy (DifferentiaEntropy, DE) features. This application uses differential entropy features as frequency domain features. Step S200 includes: performing feature extraction on the EEG signals of each frequency band of each channel through the first formula to obtain corresponding frequency band differential entropy features. Each frequency domain feature includes frequency band differential entropy features corresponding to 5 different frequency bands; Among them, the first formula is specifically:

Among them, DE represents the frequency band differential entropy characteristics, σ ² is the variance of the corresponding EEG signal, π is a constant, and e is Euler's constant. Since the EEG signal includes 62 channels, and the EEG signal of each channel is divided into 5 frequency bands, the DE feature dimension corresponding to one sample is 310 (62 channels × 5 frequency bands).

Understandably, emotional responses in the brain involve the coordination of multiple cortical areas. Therefore, the brain can be viewed as a complex network with connectivity to explore the interactions between multiple brain regions. Common brain connection features include Phase Lag Index (PLI) features, Phase Locking Value (PLV) features, Partial Directed Coherence (PDC) features, etc. This application uses the phase delay index feature as the brain connection feature. Referring to Figure 2, step S300 includes but is not limited to step S310 and step S320.

Step S310, calculate the instantaneous phase of the EEG signal of each channel through the second formula; the second formula is:

Among them, the total number of channels is C, i=1,2,…,C, φ _i (t) represents the instantaneous phase of the EEG signal of the i-th channel, t=1,2,3,…,T, T is The sampling point, s _o (t) represents the time series of the i-th channel, is the Hilbert transform of s _o (t); the analysis signal corresponding to the time series s _i (t) is:/> In the expression that analyzes the signal, It can be calculated by the following formula/>

exist In the calculation formula, PV represents the Cauchy principal value, and τ represents the integral variable. It should be noted that in SEED, the EEG signal includes 62 channels, so C=62. This application does not specifically limit the number of channels. Those skilled in the art can set the number of EEG signal channels according to actual needs.

Step S320, through the third formula, according to the instantaneous phase of the EEG signals of each two channels, the phase lag index feature between the EEG signals of each two channels is obtained, and the phase lag index feature is used as the brain connection feature; the third formula is :

Among them, PLI _i,k represents the phase lag index characteristics between the EEG signal of the i-th channel and the EEG signal of the k-th channel, φ _i (t) represents the instantaneous phase of the EEG signal of the i-th channel, φ _k ( t) represents the instantaneous phase of the k-th channel EEG signal, T is the sampling point, t=1,2,3,…,T, i=1,2,3,…,C, k=1,2,3 ,…,C. Since the EEG signal of each frequency band contains 62 channels, the connection matrix dimension of a PLI feature is 62×62. Due to the symmetry of the connection matrix, this application only uses the values in the triangles on the matrix and removes the self-connected values on the diagonal (the value is 1). Therefore, the PLI feature dimension of the sample corresponding to each frequency band is 62 × (62-1) ÷ 2 = 1891. Therefore, the PLI feature dimension of a sample is 9455 (1891 connections × 5 frequency bands).

It can be understood that in step S400, connection-channel features are obtained based on frequency domain features and brain connection features; where each connection-channel feature corresponds to one brain connection feature and two frequency domain features. specific, Represents the brain connection features extracted from channels k ₁ and k ₂ ,/> and/> Represent the frequency domain features extracted from channels k ₁ and k ₂ respectively, then connect - channel features/> in/> In, k=1,2,3,…,1891, k ₁ =1,2,3,…,C, k ₂ =1,2,3,…,C.

It can be understood that, referring to FIG. 3 , step S500 may include but is not limited to step S510, step S520, step S530, step S540 and step S550.

Step S510: Obtain mapping features according to the connection-channel features through the fourth formula; the fourth formula is:

Z ^p =[Z ₁ ,…,Z _p ]

Among them, Z ^p represents the mapping feature set, Z _i represents the i-th group of mapping features, Characterization mapping function, p is the number of feature maps, /> is the weight,/> is the bias term,/> Characterizes connection-channel characteristics; under the premise of appropriate dimensions, it can be randomly generated/> and/> and/> Fine-tuning can be performed by a sparse autoencoder to obtain a more appropriate mapping feature set Z ^p .

Step S520: Send the mapping feature set to the enhancement node through the fifth formula to obtain enhanced features; the fifth formula is:

H ^q =[H ₁ ,…,H _q ]

Among them, H ^q represents the enhanced feature set, H _j represents the jth group of enhanced features, ξ _j represents the nonlinear activation function, q is the number of feature enhancements, is the weight,/> is the bias term, Z ^p represents the mapping feature set;

Step S530, concatenate the enhanced feature set and the mapping feature set to obtain the conversion feature set A = [Z ^p , H ^q ]; where a represents the conversion feature set, Z ^p represents the mapping feature set, and H ^q represents The enhanced feature set;

Step S540, according to the conversion feature set and the emotion label, minimize the objective function of the width learning system to obtain the label space; the objective function is:

Among them, F represents the F-norm of the matrix, Approximation used for measurement accuracy,/> Characterization/> Regularization term, W represents the weight of the width learning system, Y represents the emotion label, λ represents the penalty factor, I represents the unit matrix, A represents the transformation feature set, A = [Z ^p , H ^q ], Z ^p represents the mapping feature set, H ^q represents the enhanced feature set;/> Used to smooth the distribution of W and avoid overfitting;/> y _i represents the i-th emotion label. By solving the objective function we get:

W＝(λI+A ^T A) ^-1 A ^T Y

Step S550: Verify according to the label space to obtain the average test accuracy corresponding to each connection-channel feature. Specifically, the average test accuracy BLS representation width learning system,/> Represents connection-channel features, k=1,2,3,…,1891, Y represents emotion label.

It can be understood that after obtaining the average test accuracy corresponding to each connection-channel feature, the average test accuracy corresponding to each connection-channel feature is stored in the array Acc _set , specifically:

Acc _set =[Acc _set ,Acc _k ]

Acc _set represents the array, Acc _k represents the average test accuracy, k=1,2,3,…,1891. Sorting the average accuracy in the array from high to low gives:

The corresponding index that represents the accuracy of each connection-channel feature is arranged from large to small, and can be Filter according to the preset proportion threshold to obtain the filtered connection-channel features, that is, obtain the key connection-channel feature set,

Characterizing the key connection-channel feature set, C is the number of channels, C=62, t is the preset proportion threshold, and those skilled in the art can set t according to actual needs. Since each connection-channel feature corresponds to a brain connection feature and two frequency domain features, it can be based on the key connection-channel feature set/> The important channels are obtained, and then the corresponding filtered brain connection features are obtained based on the important channels, that is, the important brain connection features X ^BC are obtained, and the corresponding filtered frequency domain features are obtained based on the important channels, that is, the important frequency domain features X ^FD are obtained. It should be noted that since different brain connection features in the key connection-channel feature set may be generated by the same channel, duplicate channels will appear during screening. When screening, duplicate channels need to be screened out.

In a second aspect, embodiments of the present application provide an emotion recognition method, including the channel screening method of any one of the embodiments of the first aspect of the present application.

Referring to Figure 4, the emotion recognition method in this embodiment of the present application also includes but is not limited to step S900 and step S1000.

Step S900, extract important frequency domain features and important brain connection features according to important channels, map the important frequency domain features and important brain connection features through the AEKM algorithm respectively, and obtain AEKM frequency domain features and AEKM brain connection features respectively;

Step S1000, splice AEKM frequency domain features and AEKM brain connection features to obtain fusion features, input the fusion features and emotion labels to the width learning system, perform cross-subject emotion recognition, and obtain classification accuracy.

The emotion recognition method of the second embodiment of the present application uses the channel screening method of the first embodiment to screen the channels of the EEG signals to obtain important channels, and extract important frequency domain features and important brain connection features based on the important channels. Important frequency domain features and important brain connection features are mapped through the AEKM algorithm to obtain AEKM frequency domain features and AEKM brain connection features respectively; AEKM frequency domain features and AEKM brain connection features are spliced to obtain fusion features, and the fusion features are combined with emotions The labels are input into the width learning system to perform cross-subject emotion recognition and obtain the classification accuracy. By filtering the channels of EEG signals, the number of channels is reduced, thereby reducing the time and cost spent in collecting data. The reduction in the number of channels means the reduction of electrodes. Compared with the experimental collection cost of all electrodes, using fewer and more effective electrodes to conduct experiments can save a lot of manpower and material resources, and the features are screened and the data processed is reduced. , making the experiment more efficient and the accuracy of the recognition results higher.

It should be noted that (AEKM, Approximate Empirical Kernel Map) is a mapping algorithm that can extract sufficient discriminant information with a small dimension l, and can provide fast calculation for subsequent classifiers.

It can be understood that step S900 may include the following steps:

According to the important channels, important features are obtained, specifically:

Among them, X ^m is an important feature. When m ^{= FD} , x ^m is an important frequency domain feature; when m = BC, _x ^m is an important brain connection feature. Each row ^of i=1,2,…,n, is the i-th sample of m, n is the number of samples;

Important frequency domain features and brain connection features are mapped to obtain AEKM features; where,

g _m (v ^m )∈R ^l ×1 (m=FD, BC)

Among them, g _m (x ^m ) represents the AEKM features; l represents the dimension; when m = FD, g _m (x ^m ) is the AEKM frequency domain feature; when m = BC, g _m (x ^m ) is the AEKM brain connection Features; when m=FD, x ^m is an important frequency domain feature; when m=BC, x ^m is an important brain connection feature.

It can be understood that, referring to Figure 5, in step S900, important frequency domain features and brain connection features are mapped to obtain AEKM features, which may specifically include but are not limited to steps S910, step S920, step S930, step S940, and step S950. and step S960.

Step S910, for important features Pass/> Algorithm, obtain the set of marked points, specifically:

Among them, V ^m represents the set of marked points, Represents the j-th marker point, l represents the dimension, j=1,2,…,l; it should be noted that,/> It can be obtained using a fast approximate k-means sampling algorithm; the ^l mark points are the cluster centers of .

Step S920, construct the first kernel matrix through the sixth formula; the sixth formula is:

Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the first kernel matrix, σ _m is the kernel parameter, and κ(.,.,.) is the kernel function;

Step S930: Construct the second kernel matrix through the seventh formula; the seventh formula is:

Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the second kernel matrix, σ _m is the kernel parameter, and κ(.,.,.) is the kernel function;

Step S940: Perform eigenvalue decomposition on the second kernel matrix, specifically as follows:

Among them, l represents the dimension, is a diagonal matrix, the diagonal elements are/> eigenvalues of The column vector of is the eigenvector corresponding to the eigenvalue;

Step S950: Obtain the AEKM feature matrix according to the first kernel matrix and the second kernel matrix through the eighth formula. The eighth formula is:

Among them, n is the number of samples, l represents the dimension, G ^m represents the AEKM feature matrix, when m = FD, G ^m is the AEKM frequency domain feature matrix; when m = BC, G ^m is the AEKM brain connection feature matrix, Among them,/> is a diagonal matrix, the diagonal elements are/> eigenvalues,/> The column vector of is the eigenvector corresponding to the eigenvalue, and M ^m serves as the eigenvalue mapping matrix;

Step S960: Obtain the fusion feature matrix based on the AEKM frequency domain feature matrix and the AEKM brain connection feature matrix.

^GF = [G ^BC , G ^FD ], where ^GF represents the fusion feature matrix, G ^FD represents the AEKM frequency domain feature matrix, and G ^BC represents the AEKM brain connection feature matrix. because Therefore, the AEKM frequency domain feature matrix G ^FD and the AEKM brain connection feature matrix G ^BC can be obtained from the AEKM feature matrix G ^m . Specifically, the fusion feature matrix G ^F =[G ^BC , G ^FD ].

After obtaining the fusion feature matrix, input the fusion feature matrix and emotion labels into the width learning system to perform cross-subject emotion recognition and obtain the classification accuracy, specifically:

Acc=BLS(G ^F ,Y)

Among them, Acc represents the classification accuracy, BLS represents the width learning system, G ^F represents the fusion feature matrix, and Y represents the emotion label.

It can be understood that this application uses the existing public emotional EEG data set SEED as experimental data. The EEG cap used in this data set is the 62-channel EEG cap of the International 10-20 system. Therefore, the EEG signals obtained Includes 62 channels. The differential entropy feature is used as the frequency domain feature, and the phase lag index feature between the EEG signals of each two channels is used as the brain connection feature. After processing by the emotion recognition method in the above embodiment, in the channel screening method, the preset proportion threshold is t=3%, 62 channels are screened, and 37 channels are obtained after screening. The frequency domain feature matrix and the brain connection feature matrix were extracted based on the 37 channels. The frequency domain feature matrix and the brain connection feature matrix were respectively mapped by the AEKM algorithm to obtain the AEKM frequency domain feature matrix G ^FD and the AEKM brain connection feature matrix G ^BC . The AEKM The frequency domain feature matrix and the AEKM brain connection feature matrix are spliced to obtain a fusion feature matrix. The fusion feature matrix and emotion labels are input to the width learning system to perform cross-subject emotion recognition and obtain the classification accuracy. As shown in Table 1, Table 1 shows the classification accuracy obtained by inputting different features into the width learning system.

Table 1 Accuracy obtained by the "leave one subject" cross-validation method

Referring to Table 1, from the results of Table 1, the accuracy of the fused feature matrix is improved compared to the accuracy of a single feature. This can show that the characteristics of frequency domain features and brain connection features are complementary. After fusion, different Emotions have better recognition performance; from the results in Table 1, it can be seen that the number of channel electrodes in the selected 37-channel feature is 40.32% less than that in the 62-channel feature. The accuracy of frequency domain features, brain connection features, and fusion feature matrix Compared with 62-channel, the accuracy has increased, which shows that the channels selected by the method of this application are effective.

The third embodiment of the present application provides a channel screening system, including:

at least one first memory;

at least one first processor;

at least one program;

The program is stored in the first memory, and the first processor executes at least one program to implement:

The channel screening method described in the embodiment of the first aspect of this application.

The first processor and the first memory may be connected through a bus or other means.

As a non-transitory readable storage medium, memory can be used to store non-transitory software instructions and non-transitory instructions. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. It will be appreciated that the memory may optionally include memory located remotely relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

The first processor executes non-transient software instructions, instructions and signals stored in the first memory to implement various functional applications and data processing, that is, to implement the channel screening method of the first embodiment.

The non-transient software instructions and instructions required to implement the channel screening method of the first embodiment or the emotion recognition method of the second embodiment are stored in the memory. When executed by the processor, the first embodiment of the present application is executed. The channel screening method, for example, performs the above-described method steps S100 to S700 in Figure 1, method steps S310 to S320 in Figure 2, and method steps S510 to S550 in Figure 3.

It should be noted that the channel screening system in the above-mentioned embodiment is based on the same inventive concept as the channel screening method in the above-mentioned embodiment. Therefore, in the above-mentioned embodiment The corresponding content of the channel screening method is also applicable to the channel screening system in the above-mentioned embodiments, and has the same implementation principles and technical effects. In order to avoid redundant description content, it will not be described in detail here.

The fourth embodiment of the present application provides an emotion recognition system, including:

at least one second memory;

at least one second processor;

at least one program;

The program is stored in the second memory, and the second processor executes at least one program to implement:

The emotion recognition method as described in the embodiment of the second aspect of this application.

The second processor and the second memory may be connected through a bus or other means.

The second processor executes the non-transient software instructions, instructions and signals stored in the second memory to perform various functional applications and data processing, and implements the emotion recognition method of the embodiment of the second aspect.

The non-transient software instructions and instructions required to implement the emotion recognition method of the embodiment of the second aspect are stored in the memory. When executed by the processor, the channel screening method or the embodiment of the second aspect of the application is executed. The emotion recognition method, for example, performs the above-described method steps S100 to S700 in Figure 1, method steps S310 to S320 in Figure 2, method steps S510 to S550 in Figure 3, and method steps S900 to S1100 in Figure 4 , method steps S910 to S960 in Figure 5.

It should be noted that the emotion recognition system in the above-mentioned embodiments is based on the same inventive concept as the emotion recognition method in the above-mentioned embodiments. Therefore, in the above-mentioned embodiments The corresponding content of the emotion recognition method is also applicable to the emotion recognition system in the above-mentioned embodiments, and has the same implementation principles and technical effects. To avoid redundancy of description content, it will not be described in detail here.

The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place. , or it can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The fifth aspect embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable signals. The computer-executable signals are used to execute:

The channel screening method as described in the embodiment of the first aspect of this application; or,

The emotion recognition method as described in the embodiment of the second aspect of this application.

The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor or a controller, for example, by a processor of the multi-data source processing system in the above embodiment, so that the processor Perform the channel screening method according to the first embodiment of the present application or the emotion recognition method according to the second embodiment, for example, perform the above-described method steps S100 to S700 in Figure 1, method steps S310 to S320 in Figure 2, Method steps S510 to S550 in Figure 3, method steps S900 to S1000 in Figure 4, and method steps S910 to S960 in Figure 5.

Through the description of the above embodiments, those of ordinary skill in the art can understand that all or some steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit . Such software may be distributed on readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable signals, data structures, modules of instructions, or other data. removable, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or may Any other medium used to store the desired information and that can be accessed by a computer. Additionally, it is known to those of ordinary skill in the art that communication media typically embodies a computer-readable signal, data structure, instruction module, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

The embodiments of the present application have been described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the above-mentioned embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various embodiments can be made without departing from the purpose of the present application. Variety. In addition, the embodiments of the present application and the features in the embodiments may be combined with each other without conflict.

Claims (9)

1. A channel screening method, characterized by comprising: Acquire multi-channel EEG signals and preprocess the EEG signals; Obtain the frequency domain characteristics of each channel according to the preprocessed EEG signal; Obtain the brain connection characteristics between each two channels according to the preprocessed EEG signals; According to the frequency domain features and the brain connection features, connection-channel features are obtained; wherein each connection-channel feature includes one of the brain connection features and two of the frequency domain features, and the brain connection features correspond to The two channels respectively correspond to the two frequency domain features; Input each connection-channel feature and emotion label into the width learning system, obtain the recognition result and verify the recognition result, and obtain the average test accuracy corresponding to each connection-channel feature; Sort all the connection-channel features according to the average test accuracy, and filter according to the preset proportion threshold to obtain a key connection-channel feature set; According to the key connection-channel feature set, important channels are obtained; Input each connection-channel feature and emotion label into the width learning system, obtain the recognition result and verify the recognition result, and obtain the average test accuracy corresponding to each connection-channel feature, including: Through the fourth formula, mapping characteristics are obtained according to the connection-channel characteristics; the fourth formula is: z ^p =[Z ₁ ,…,Z _p ] Among them, Z ^p represents the mapping feature set, Z _i represents the i-th group of mapping features, Characterization mapping function, p is the number of feature maps, /> is the weight,/> is the bias term,/> Characterizing connection-channel characteristics; Through the fifth formula, the mapping feature set is sent to the enhancement node to obtain the enhanced features; the fifth formula is: H ^q =[H ₁ ,…,H _q ] Among them, H ^q represents the enhanced feature set, H _j represents the jth group of enhanced features, ξ _j represents the nonlinear activation function, q is the number of feature enhancements, is the weight,/> is the bias term, Z ^p represents the mapping feature set; The enhanced feature set and the mapping feature set are concatenated to obtain a conversion feature set A = [Z ^p , H ^q ]; where A represents the conversion feature set, Z ^p represents the mapping feature set, H ^q Characterize the set of enhanced features; According to the conversion feature set and the emotion label, the objective function of the width learning system is minimized to obtain a label space; the objective function is: Among them, F represents the F-norm of the matrix, Approximation used for measurement accuracy,/> represents the l ₂ regularization term, W represents the weight of the width learning system, Y represents the emotion label, λ represents the penalty factor, I represents the unit matrix, A represents the conversion feature set, A=[Z ^p , H ^q ], Z ^p represents the mapping feature set, and H ^q represents the enhanced feature set; by solving the objective function, it is obtained: W＝(λI+A ^T A) ^-1 A ^T Y Verify according to the label space to obtain the average test accuracy corresponding to each connection-channel feature. 2. The channel screening method according to claim 1, wherein the preprocessing of the EEG signal includes: The EEG signal is filtered through 5 different frequency bands. 3. The channel screening method according to claim 2, wherein the frequency domain characteristics of each channel are obtained according to the preprocessed EEG signal, including: Features are extracted from the EEG signals of each frequency band of each channel through the first formula to obtain corresponding frequency band differential entropy features. Each frequency domain feature includes the frequency band differential entropy corresponding to 5 different frequency bands. Characteristics; wherein, the first formula is specifically: Wherein, the DE represents the differential entropy characteristics of the frequency band, σ ² is the variance of the corresponding EEG signal, π is a constant, and e is Euler’s constant. 4. The channel screening method according to claim 3, wherein the brain connection characteristics between each two channels are obtained according to the preprocessed EEG signals, including: The instantaneous phase of the EEG signal of each channel is calculated through the second formula; the second formula is: Wherein, the total number of channels is C, i=1,2,...,C, φ _i (t) represents the instantaneous phase of the EEG signal of the i-th channel, t=1,2,3,..., T, T is the sampling point, s _i (t) represents the time series of the i-th channel, is the Hilbert transform of s _i (t); Through the third formula, according to the instantaneous phase of the EEG signals of each two channels, the phase lag index characteristics between the EEG signals of each two channels are obtained, and the phase lag index characteristics are used as the brain connection characteristics. ;The third formula is: Wherein, PLI _i,k represents the phase lag index characteristic between the EEG signal of the i-th channel and the EEG signal of the k-th channel, and φ _i (t) represents the EEG signal of the i-th channel. The instantaneous phase of the signal, φ _k (t) represents the instantaneous phase of the EEG signal of the k-th channel, T is the sampling point, t=1,2,3,...,T, i=1,2 ,3,…,C, k=1,2,3,…,C. 5. An emotion recognition method, characterized in that it includes the channel screening method according to any one of claims 1 to 4; The emotion recognition method also includes: Extract important frequency domain features and important brain connection features according to the important channels, map the important frequency domain features and the important brain connection features through the AEKM algorithm respectively, and obtain AEKM frequency domain features and AEKM brain connection features respectively; The AEKM frequency domain features and the AEKM brain connection features are spliced to obtain fusion features. The fusion features and the emotion labels are input to the width learning system to perform cross-subject emotion recognition to obtain classification accuracy. 6. The emotion recognition method according to claim 5, characterized in that the important frequency domain features and the important brain connection features are mapped respectively through the AEKM algorithm to obtain the AEKM frequency domain features and the AEKM brain connection respectively. Features, including: According to the important channels, important features are obtained, specifically: Among them, m∈{FD, BC}, X ^m is an important feature, when m = FD, x ^m is the important frequency domain feature; when m = BC, x ^m is the important brain connection feature, X ^m Each row of is composed of (x _i ^m ) ^T , i=1,2,…,n, is the i-th sample of m, n is the number of samples; The important frequency domain features and the important brain connection features are mapped to obtain AEKM features; where, g _m (x ^m )∈R ^l×1 Among them, m∈{FD,BC}, g _m (x ^m ) represents the AEKM feature; l represents the dimension; when m=FD, g _m (x ^m ) is the AEKM frequency domain feature; when m=BC, g _m (x ^m ) is the AEKM brain connection feature; when m = FD, x ^m is the important frequency domain feature; when m = BC, x ^m is the important brain connection feature; The important frequency domain features and the brain connection features are mapped to obtain AEKM features, including: For the important features mentioned Pass/> Algorithm, obtain the set of marked points, specifically: Among them, m∈{FD,BC}, V ^m represents the set of marked points, It represents the jth marker point, l represents the dimension, j=1,2,…,l; The first kernel matrix is constructed through the sixth formula; the sixth formula is: Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the first kernel matrix, σ _m is the kernel parameter, and κ (.,.,.) is the kernel function; The second kernel matrix is constructed through the seventh formula; the seventh formula is: Among them, k=1,…,n, j=1,…,l, n is the number of samples, l represents the dimension,/> Characterizing the second kernel matrix, σ _m is the kernel parameter, and κ (.,.,.) is the kernel function; Perform eigenvalue decomposition on the second kernel matrix, specifically: Among them, l represents the dimension, is a diagonal matrix, the diagonal elements are/> eigenvalues,/> The column vector of is the eigenvector corresponding to the eigenvalue; Through the eighth formula, according to the first kernel matrix and the second kernel matrix, the AEKM feature matrix is obtained The eighth formula is: Among them, n is the number of samples, l represents the dimension, G ^m represents the AEKM feature matrix, when m = FD, G ^m is the AEKM frequency domain feature matrix; when m = BC, G ^m is the AEKM brain connection feature matrix, Among them,/> is a diagonal matrix, the diagonal elements are/> eigenvalues,/> The column vector of is the eigenvector corresponding to the eigenvalue, and M ^m serves as the eigenvalue mapping matrix; According to the AEKM frequency domain feature matrix and the AEKM brain connection feature matrix, a fusion feature matrix is obtained ^GF = [G ^BC , G ^FD ], where ^GF represents the fusion feature matrix, G ^FD represents the AEKM frequency domain feature matrix, and G ^BC represents the AEKM brain connection feature matrix. 7. A channel screening system, characterized in that it includes: at least one first memory; at least one first processor; at least one program; The program is stored in the first memory, and the first processor executes at least one program to implement: The channel screening method according to any one of claims 1 to 4. 8. An emotion recognition system, characterized by comprising: at least one second memory; at least one second processor; at least one program; The program is stored in the second memory, and the second processor executes at least one program to implement: The emotion recognition method according to any one of claims 5 to 6. 9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable signals, and the computer-executable signals are used to execute: The channel screening method according to any one of claims 1 to 4; or, The emotion recognition method according to any one of claims 5 to 6.