CN107610099A - A kind of emotion state of mind detecting system and method based on pupil information - Google Patents

Abstract

The implementation of the present invention discloses an emotional and mental state detection system and method based on pupil information. The system includes: an emotional induction module for emotional induction of testers; an information collection module for collecting images captured by cameras in each frame The information processing module is used for image processing, calculating the pupil information and judging whether the information belongs to neutral information; the neutral information processing module is used for integrating and calculating the average value of the pupil diameter in the neutral information; the information feedback module is used for Transform the non-neutral information into test emotional information and feed it back to the remote server, determine and diagnose the emotional state of the tester and send it to the tester. The implementation of the embodiment of the present invention not only overcomes the influence of strong light on pupils with different aging degrees in pupil detection, but also can effectively detect emotional and mental states based on pupil information, and at the same time, this detection method can be applied to mobile devices. It is of great significance in mobile medicine.

Description

An emotional and mental state detection system and method based on pupil information

technical field

The invention relates to the technical fields of cognitive psychology, biological feature extraction, image processing, and mobile medical care, and in particular to an emotional and mental state detection system and method based on pupil information.

Background technique

With the rapid development of the market economy, fast-paced life has brought a lot of pressure on people's mental psychology, and the incidence of various emotional disorders is increasing day by day.

Affective disorders are mainly manifested in emotions, and changes in emotions directly affect the size of the pupils. The dynamic change of pupil diameter has the characteristics of non-adaptability and not being controlled by consciousness. Positive emotional information will make the pupil diameter enlarged, and negative emotional information will make the pupil diameter narrow, so it can accurately reflect the mental activity of the brain. In addition, the emotional reaction rate and concentration of the pupils can also reflect the mental state of the tester. The emotional response rate of the pupils of autistic patients is slower than that of ordinary people; the emotional bandwidth of the pupils of patients with schizophrenia is narrow and the eye tracking function is abnormal; patients with depression have impaired selective attention and increased attention to negative information. Reduced attention to positive information and greater sensitivity to negative information.

These phenomena are most obviously concentrated among the elderly and young people, and have become the focus of people's attention. How to accurately, quickly and conveniently reflect the patient's psychological state, and how to prevent and treat the patient's psychological problems has become a difficult problem. At present, most of the emotional diagnoses are made in the form of scales, and these diagnostic results are often affected by patients' subjective concealment, which makes the judgment results inaccurate.

In recent years, some methods have emerged to identify emotional disorders by collecting information on emotional bandwidth and focus points. However, these methods all use multiple infrared cameras or infrared cameras that special mobile devices do not have, which is not conducive to people's quick and convenient grasp of their own mental state; and in order to simplify the calculation complexity, these methods also use the bright pupil The detection method, but ignores the secondary variable brought by the light to the pupil due to illuminating the pupil. Experiments have shown that for people of different ages, their pupils are also different in sensitivity to light, which makes the test results of these methods not accurate enough.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art and improve the prior art into a technology that can be applied to mobile devices. The implementation of the present invention discloses a method for judging emotional disorders based on the dynamic changes of pupils monitored by mobile devices, allowing people to self-diagnose at home, early warning and prevention of emotional mental diseases, and improving the quality of people's mental state.

In order to solve the above problems, the present invention proposes a system for detecting emotional disorders based on pupil information, the system includes the following modules:

The emotion induction module is used to induce emotion to the tester;

An information collection module, configured to collect images captured by each frame of the camera;

The information processing module is used for image processing of each frame of image, calculation of pupil information and judgment of whether the information belongs to neutral information;

The neutral information processing module is used to integrate and calculate the pupil diameter information of the neutral information, obtain the average value as the reference value of the neutral pupil diameter, and its range as the threshold range of the neutral pupil diameter;

The information feedback module is used to convert all pupil information into test emotional information and feed it back to the remote server, determine and diagnose the tester's emotional state and send the result to the tester.

Preferably, the emotion induction module includes:

Material capture unit. Used to extract emotionally stimulating materials.

Material handling unit. It is used to process each material into a playback material with a duration of 10s.

material playback unit. For playback of processed material.

Information transfer unit. It is used to transmit the normal emotional information corresponding to each material and its threshold value to the information feedback module as standard emotional information, and transmit the standard emotional information to the information feedback module.

Preferably, the information collection module includes:

Time control unit. Divide time with 2s as 1 frame.

Image acquisition unit. The image collected by the camera is captured for each frame, and the collected image is sent to the information processing module.

Preferably, the information processing module includes:

image processing unit. Image processing is performed on the image collected by the information collection module to obtain a pupil image.

information processing unit. Obtain the processed pupil image, and calculate pupil information.

information judgment unit. It is judged whether the pupil information is pupil information in a relaxed state. If yes, pass it as neutral information to the neutral information processing module, if not, pass it to the information feedback module.

Preferably, the neutral information processing module includes:

Neutral information integration unit. Extract and cache the pupil diameter data in the neutral information.

Neutral information computing unit. Calculate the average value of the cached data to obtain the reference value of neutral pupil diameter. Get the cached data range as the neutral pupil diameter threshold range.

Preferably, the information feedback module includes:

information computing unit. Using the tester's pupil information to calculate the tester's emotional bandwidth, the number of occurrences of different emotional bandwidths and the degree of concentration as reference information.

Information recording unit. Record the reference information throughout the test process, draw a graph according to the emotional bandwidth data, calculate the emotional reaction time, obtain the test emotional information including the reference information and emotional reaction time, and send it to the remote server.

information determination unit. Compare the test emotional information with the standard emotional information, judge and diagnose the emotional and mental state of the tester, obtain the judgment and diagnosis information and send it to the tester.

Correspondingly, another aspect of the present invention proposes a method for detecting emotional and mental states based on pupil information, the method comprising the following steps:

The testers were given a 10s relaxation time, and the neutral pupil diameter information was extracted from the testers, and the reference value of the neutral pupil diameter and its threshold range were calculated.

Extract emotion-inducing materials for emotional induction.

The pupil images of each frame of the tester during the emotion induction process were collected.

The pupil image is acquired, image processing is performed, and pupil diameter and gaze point coordinates of the pupil are calculated as pupil information.

Obtain pupil information, convert test emotional information, judge and diagnose the user's current emotional and mental state, send the judgment and diagnosis results to the tester, and feed back the test emotional information to the remote server.

Preferably, the image processing steps are as follows:

Convert the collected image into a grayscale image;

Obtain the coordinate range information of the reference comparison area in the grayscale image, filter out the reference comparison area in the image and cut it, and obtain the image of the reference comparison area and the image to be tested with the reference comparison area cut;

Obtain the gray value information of the image in the reference comparison area, filter out the gray value range of the reference comparison area, and obtain the gray threshold standard for judgment;

Obtain the grayscale threshold standard and the image to be tested, set the grayscale of the image area to be tested that meets the grayscale threshold standard to 0, and the grayscale of the image area to be tested that does not meet the grayscale threshold standard to 255, and obtain a binary image;

A binary image is obtained, and an opening operation and a closing operation are performed on the binary image to obtain a pupil image after image processing.

Preferably, the method for judging and diagnosing the emotional state is as follows:

Obtain test emotional information, calculate and judge the width and narrowness of emotional bandwidth, the number of occurrences of different types of emotional bandwidth, the advance and lag of emotional response time, and the degree of concentration of attention, and obtain the emotional and spiritual characteristics of the tester;

Obtain the emotional and mental characteristics of the tester, divide and process the emotional and mental state of the tester, and obtain three emotional and mental characteristics categories: positive emotional mental state, neutral emotional mental state and negative emotional mental state;

Obtaining the classification of the tester's emotional mental state, diagnosing the tester's emotional mental state, and obtaining the diagnosis result of the emotional mental state.

For the functions of each module unit in the system embodiment of the present invention, refer to the process processing in the method embodiment of the present invention, which will not be repeated here.

Implementing the embodiment of the present invention, due to the use of a method of threshold screening and extraction of pupil images, not only overcomes the influence of strong light on pupils with different aging degrees in pupil detection, but also can effectively detect emotional and mental states based on pupil information, At the same time, this detection method can be applied to mobile devices, which is of great significance in mobile medical care.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings that are required in the description of the embodiments or the prior art. Apparently, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings according to these drawings without creative efforts.

Fig. 1 is the module schematic diagram of the emotional disturbance detection system based on pupil information of the embodiment of the present invention;

Fig. 2 is the schematic flow chart of the emotional state detection method based on pupil information of the embodiment of the present invention;

3 is a schematic flow diagram of a neutral information extraction process method according to an embodiment of the present invention;

FIG. 4 is a schematic flowchart of an information processing method according to an embodiment of the present invention;

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

As shown in Figure 1, the embodiment of the present invention proposes a kind of emotional mental state detection system based on pupil information, and this system includes following modules:

The emotion-inducing module is used to collect three voice emotion-inducing materials, positive, neutral, and negative, and use these emotional materials to induce emotions in the testers.

The information collection module collects the images captured by the current camera at a rate of 2s as one frame.

The information processing module performs image processing on each frame of images collected; calculates the pupil diameter and the fixation point coordinates of the pupils in each frame obtained after image processing, and obtains pupil information; judges whether the information belongs to information in a relaxed state, if If it belongs, it will be regarded as neutral information, if it does not belong, it will be regarded as non-neutral information; the neutral information will be sent to the neutral information processing module, and the non-neutral information will be directly sent to the information feedback module.

The neutral information processing module integrates the neutral pupil diameter information processed in the information processing module, and calculates its average value as the reference value of the neutral pupil diameter, and its maximum value;

The information feedback module calculates and records the emotional bandwidth, emotional reaction time and concentration of the tester according to each frame of pupil information recorded in the information processing module and the reference value of the neutral pupil diameter, and obtains test emotional information and gives feedback to the remote server. According to the test emotional information and standard emotional information, determine and diagnose the emotional and mental state, obtain the determination and diagnosis information and send it to the tester.

Specifically, the emotion induction module includes:

Material capture unit. Randomly extract from the International Emotional Digital Sound System Database and the Chinese Emotional Digital Sound System Database: three copies of positive emotional stimulation materials, neutral emotional stimulation materials, and negative emotional stimulation materials.

Material handling unit. The materials extracted from the material capture unit are processed, and each material is processed into a playback material with a time length of 10s. Processed material can be cached in the mobile device so that it can be used directly the next time the same material is captured.

material playback unit. Three copies of positive emotion stimulation materials, neutral emotion stimulation materials, and negative emotion stimulation materials processed in the playback material processing unit.

Information transfer unit. The normal emotional bandwidth, emotional reaction time and attention concentration information and their thresholds corresponding to each material are passed to the information feedback module as standard emotional information. And pass the standard emotional information to the information feedback module to facilitate comparison of conclusions.

Further, the information collection module includes:

Time control unit. Divide time with 2s as 1 frame.

Image acquisition unit. The image collected by the camera is captured for each frame, and the collected image is sent to the information processing module.

Further, the information processing module includes:

image processing unit. Image processing is performed on the collected images.

information processing unit. Edge detection is performed on the extracted pupil binary image, and its edge points are extracted. The least square method is used to fit the circle to obtain the circular equation, and the pupil center coordinates and pupil diameter are obtained.

information judgment unit. It is judged whether the pupil information is pupil information in a relaxed state. If yes, pass it as neutral information to the neutral information processing module, if not, pass it as non-neutral information to the information feedback module.

Specifically, the image processing steps include:

Convert the collected image into a grayscale image;

Obtain the coordinate range information of the reference comparison area in the grayscale image, filter out the reference comparison area in the image and cut it, and obtain the image of the reference comparison area and the image to be tested with the reference comparison area cut;

Obtain the gray value information of the image in the reference comparison area, filter out the gray value range of the reference comparison area, and obtain the gray threshold standard for judgment;

Obtain the grayscale threshold standard and the image to be tested, set the grayscale of the image area to be tested that meets the grayscale threshold standard to 0, and the grayscale of the image area to be tested that does not meet the grayscale threshold standard to 255, and obtain a binary image;

Further, the neutral information processing module includes:

Neutral information integration unit. Extract and cache pupil diameter data in five sets of neutral information.

Neutral information computing unit. Obtain all cached data, calculate their average value, and obtain the reference value of neutral pupil diameter. Get the cached data range as the neutral pupil diameter threshold range.

Further, the information feedback module includes:

Feedback information calculation unit. According to the collected pupil information and the reference value of the neutral pupil diameter, the tester's emotional bandwidth, the number of occurrences of different emotional bandwidths, and the degree of concentration of attention are calculated as reference information.

Feedback information recording unit. Record the reference information throughout the test process; draw a scatter diagram based on the emotional bandwidth data in the reference information, and draw a corresponding curve according to the scatter diagram; calculate the tester's emotional reaction time; combine the reference information with the emotional reaction time As test emotion information and fed back to the remote server through the Internet. The remote server can adjust the emotion-inducing materials in the database according to the collected test emotion information, refine the gaze point threshold range, and improve the judgment and diagnosis scheme to make the test results more accurate.

Feedback information determination unit. Compare the test emotional information with the standard emotional information, judge and diagnose the tester's emotional and mental state according to the comparison results, obtain the judgment and diagnosis information and send it to the tester's mobile device, so that the tester can grasp his own emotional and mental state.

In a specific embodiment, the emotional bandwidth is the pupil diameter in the information minus the reference value of the neutral pupil diameter.

Further, the judgment method of emotional bandwidth is:

When the emotional bandwidth belongs to the threshold range of neutral pupil diameter, it is called neutral emotional bandwidth;

When the affective bandwidth is greater than the positive threshold range of the neutral pupil diameter, it is called positive affective bandwidth;

When the emotional bandwidth is smaller than the negative threshold range of the neutral pupil diameter, it is called the negative emotional bandwidth. The negative threshold range of the neutral pupil diameter is the minimum value of the neutral pupil diameter threshold and the average value of the neutral pupil diameter. Difference.

In a specific example, the degree of concentration of attention is graded according to the number of times that the central coordinates of the pupil deviate from the prescribed fixation point and its prescribed threshold range. The greater the number of deviations, the lower the level of concentration; conversely, the less the number of deviations, the higher the level of concentration.

Further, the degree of concentration level includes:

Grade A means that the number of pupil deviations is within five times;

Grade B means that the number of pupil deviations is more than five times and less than ten times;

Grade C means that the number of pupil deviations is more than 10 times and less than 15 times;

Grade D, indicating that the number of pupil deviations exceeds fifteen times.

In a specific embodiment, the emotional reaction time is the time from the start of playing a piece of material to the time when the emotional bandwidth is greater than the upper bound of the neutral emotional bandwidth or lower than the lower bound of the neutral emotional bandwidth for the first time after the material starts to play. We can obtain this time through the graph drawn by the feedback information recording unit. If the emotional bandwidth is not greater or smaller than the upper and lower bounds of the neutral emotional bandwidth during the playback of the material, record the emotional response time as ∞, which means that there is no clear emotional response phenomenon.

Correspondingly, as shown in FIG. 2, an embodiment of the present invention also proposes a method for detecting emotional and mental states based on pupil information, the method comprising:

Step S1: Give the tester a relaxation time of 10s, extract the neutral pupil diameter information from the tester with 2s as a frame, and calculate the reference value of the neutral pupil diameter.

Step S2: extract emotion-inducing materials, and induce emotion on testers.

Step S3: Take 2s as one frame to collect the tester's pupil image.

Step S4: Perform information processing on the collected images to obtain the pupil diameter and pupil center coordinates.

Step S5: . According to the processed pupil information, the emotional bandwidth, emotional reaction time and concentration degree are calculated, and the user's current emotional state is judged.

In a specific example, the tester must take the measurement in a location with sufficient natural light, and prepare a reference comparison card in addition to the mobile device;

Specifically, the reference comparison card is a black magnetic stripe card with the same pupil imaging response.

The reference comparison card should be placed on the same level as the eyes, and it will be captured by the camera of the mobile device and imaged in the reference comparison area given on the screen;

The camera of the mobile device needs to be placed 30cm away from the tester, on the same level as the eyes, and perpendicular to the bridge of the nose between the eyes;

During the test, the tester needs to focus on the coordinates of the viewpoint;

During the test, the tester needs to ensure that the image of the reference comparison card on the screen can fill the reference comparison area shown on the screen after being photographed by the camera of the mobile device.

In a specific example, as shown in Figure 3, step S1 includes:

S11, give the tester a relaxation time of 10s, and collect the image information at this time with 2s as one frame.

S12, transmit the collected image to the information processing module, perform image processing on the collected image, and extract the pupil diameter d _0j and the gaze point coordinates (x _0jk , y _0jk ) of the pupil as pupil information. Among them, 0 represents the neutral information extracted in the first step of the test, j represents the corresponding frame number, k represents the left and right eyes, when k=0, k represents the left eye, when k=1, k represents the right eye, d Represents the diameter, x represents the abscissa in image morphology, and y represents the ordinate in image morphology.

S13, judging whether the pupil information belongs to the information collected during the 10s relaxation time, and if so, transmitting the information to the neutral information processing module.

S14, calculate the average value d ₀ as the reference value of the neutral pupil diameter, and use the maximum value and minimum value obtained from the test as the tester's neutral pupil diameter threshold boundary.

Specifically, step S2 includes:

S21. From the International Emotional Digital Sound System Database and the Chinese Emotional Digital Sound System Database, three copies of positive, neutral and negative emotional stimulation materials are extracted respectively.

S22. Determine whether the playback material file corresponding to the extracted material is cached in the mobile device. If it is not cached, the extracted material will be processed into a playback material with a length of 10s, and the playback material will be cached in the mobile device, so that the next time the same material is extracted, the corresponding playback material can be directly extracted from the mobile device; cache, then directly call the playback material corresponding to the extracted material from the mobile device.

S23, playing all the playing materials after the material processing.

S24. Transmit the standard emotion information corresponding to all playing materials to the information feedback module.

Specifically, as shown in Figure 4, step S4 includes:

S41, converting the collected image into a grayscale image.

S42. Obtain the coordinate range information of the reference comparison area in the image, filter out the reference comparison area in the image and cut it, and obtain the image of the reference comparison area and the image to be tested with the reference comparison area cut out.

S43, extracting the gray levels g _xy that have appeared in the reference comparison area from the image of the reference comparison area, and dividing the gray level intervals of the reference comparison area according to all the extracted gray levels g _xy , as gray threshold standards.

S44, bringing the grayscale threshold standard into the image to be tested, screening out the parts of the image to be tested that meet the grayscale threshold standard, setting the grayscale to 0, screening out the parts that do not meet the grayscale threshold standard, and setting the grayscale For 255, a binary image is obtained.

S45, apply the circular template to the binary image, and perform morphological opening and closing operations on the binary image to obtain a pupil image. The closed operation can be used to fill in the bright spots that may be filtered out in the pupil, and the open operation can be used to filter out small dark spots other than the pupil.

S46, extract edge information of the pupil image, and collect boundary points of the pupil image.

S47, according to these boundary points, the least square method is used to fit the circle to the image, and then the coordinate equation of the boundary fitting circle in the image can be obtained, and then the center coordinate value (x _ijk , y _ijk ) of the pupil and the pupil radius r _ij can be obtained. The pupil diameter d _ij can be obtained by multiplying the pupil radius r _ij by 2. Among them, i represents that this is the i-th emotional stimulation material, and r represents the radius.

Specifically, step S5 includes:

S51. Calculate the corresponding emotional bandwidth, the number of occurrences of different emotional bandwidths, and the degree of concentration according to the pupil diameter and pupil center coordinate information obtained through information processing.

S52, according to the pupil bandwidth matrix information, with the pupil bandwidth as the ordinate and the frame during the test as the abscissa, draw a scatter diagram of the pupil bandwidth change during the test, and draw a pupil bandwidth change estimation curve on the basis of the scatter diagram picture.

S53, according to the drawn pupil bandwidth change estimation curve, find out in the graph from the start of playing a piece of material to after the material starts to play, the emotional bandwidth is greater than the upper bound of the neutral emotional bandwidth or smaller than the lower bound of the neutral emotional bandwidth for the first time time, as emotional reaction time. If there is no emotional bandwidth greater or smaller than the upper and lower bounds of the neutral emotional bandwidth during the playback of the material, record the emotional response time as ∞, which means that there is no clear emotional response phenomenon.

S54, taking the tester's emotional bandwidth, emotional reaction time, and concentration as emotional information, and comparing them with normal values to judge and diagnose the tester's emotional and mental state.

In a specific example, the calculation method of the emotional bandwidth described in S51 is as follows:

The corresponding affective bandwidth b _ij is calculated according to the pupil diameter information, where b _ij =d _ij -d ₀ . When the emotional bandwidth belongs to the threshold range of neutral pupil diameter, it is called neutral emotional bandwidth; when the emotional bandwidth is greater than the positive threshold range of neutral pupil diameter, it is called positive emotional bandwidth; when the emotional bandwidth is less than neutral pupil diameter, it is called positive emotional bandwidth. The negative threshold range of pupil diameter is called negative affective bandwidth.

Set the pupil bandwidth matrix, which is used to record the bandwidth information of the pupil bandwidth.

Among them, the matrix row represents the number of emotional stimulation tasks, the matrix column represents the number of frames under the emotional stimulation task corresponding to the matrix row, B represents the pupil bandwidth matrix, and b represents the corresponding pupil bandwidth length under the matrix row and column.

Specifically, the calculation method of the number of occurrences of different emotional bandwidths described in S51 is as follows:

According to the information of the pupil bandwidth matrix, the number of occurrences of positive emotional bandwidth, neutral emotional bandwidth and negative emotional bandwidth is calculated as the number of occurrences of different emotional bandwidths.

Specifically, the calculation method of the degree of concentration of attention described in S51 is as follows:

The number of times the center of the pupil deviates from the gaze point is calculated according to the center coordinate information of the pupil. The pupil deviation matrix A is set to record the pupil deviation information.

Among them, the matrix row represents the i-th emotional stimulation task, the matrix column represents the number of frames under the emotional stimulation task corresponding to the matrix row, S records whether the pupil deviates, when its value is 1, it means deviation, when its value is 0, it means no deviation .

Use the Pythagorean theorem to find the distance between the center of the pupil and the center of the fixation point:

where D represents the distance between the center of the pupil and the center of the fixation point.

When D _ijk is greater than the center threshold of the fixation point, S _ij is set to 1, indicating that the pupil center in the image deviates from the fixation point. If D _ijk is within the center threshold of the gaze point, then S _ij is set to 0, indicating that the pupil center in the image does not deviate from the gaze point. As long as any pupil deviates, it is counted as producing the pupil deviation of the frame corresponding to the emotional stimulation task.

According to the number of pupil deviations during the test, the degree of concentration is divided into four grades:

Grade A means that the number of pupil deviations is within five times;

Grade B means that the number of pupil deviations is more than five times and less than ten times;

Grade C means that the number of pupil deviations is more than ten times and less than fifteen times;

Grade D refers to pupil deviation more than fifteen times.

Further, the determination and diagnosis of the emotional and mental state described in S54 includes:

S541, obtain test emotion information, compare it with standard emotion information, determine the width and narrowness of emotion bandwidth, the number of occurrences of different types of emotion bandwidth, the advance and lag of emotional response time, and the degree of concentration, and obtain the emotion of the tester Spirituality.

S542. Classify the emotional and mental state of the tester according to the determined emotional and mental characteristics of the tester.

S543. According to the classification of the tester’s emotional state, give relevant suggestions to the tester. When there is no negative emotional state, the tester should be reminded to maintain the state; when there are one or two negative emotional states, the tester should be reminded The tester needs to adjust his mood; when there are more than two negative emotional mental states, the tester should be reminded to consult a relevant psychologist to adjust his mental state.

Specifically, the wide and narrow judgment method of the emotional bandwidth described in S541 is as follows:

When the absolute value of the positive and negative affective bandwidth of the test is greater than the absolute value of the positive and negative affective bandwidth and its threshold under normal conditions, it is judged as positive and negative affective overreaction;

When the absolute value of the positive and negative affective bandwidth of the test is less than the absolute value of the positive and negative affective bandwidth and its threshold under normal conditions, it is determined that the positive and negative affective responses are insensitive;

When the absolute value of the positive and negative affective bandwidth of the test is within the absolute value threshold of the positive and negative affective bandwidth under normal circumstances, it is determined that the positive and negative affective responses are normal;

Specifically, the method for judging the number of occurrences of different types of emotional bandwidth described in S541 is as follows:

When the frequency of neutral emotional bandwidth in the test is more than that of positive and negative emotional bandwidth, it is judged as affective mental response disorder;

When the negative emotional bandwidth of the test appears more than the positive and neutral emotional bandwidth, it is judged as a negative emotional mental response;

When the number of occurrences of positive emotional bandwidth in the test is more than that of negative and neutral emotional bandwidth, it is judged as a positive emotional and mental response;

When the positive, neutral, and negative emotional bandwidths of the test appear in equal numbers, the emotional and mental state is judged to be normal;

Specifically, the method for judging the advance and lag of the emotional response time described in S541 is as follows:

When the emotional response time of the test is advanced, it is judged as a positive emotional response;

When the emotional response time of the test is delayed, it is judged as emotionally unresponsive;

When the emotional reaction time of the test is within the threshold, it is judged that the emotional reaction time is normal;

Specifically, the method for determining the degree of concentration of attention described in S541 is as follows:

S5411. Calculate the number of pupil deviations occurring in positive, neutral and negative emotional bandwidths according to the pupil deviation matrix.

S5412. According to the number of pupil deviations under positive, neutral, and negative emotional bandwidths, determine the emotional concentration characteristics of the tester.

Further, the method for judging the characteristics of emotional concentration is as follows:

When the number of fixation deviations mainly appears in the bandwidth of positive and neutral emotions, it is judged that the concentration of negative emotions is concentrated.

When the number of fixation deviations mainly appears in the neutral and negative emotional bandwidth, it is judged as positive emotional concentration.

Specifically, the emotional and mental state classification method of the tester described in S542 is as follows:

According to the emotional mental state of the testers, the emotional mental state of the testers is divided into positive emotional mental state, neutral emotional mental state and negative emotional mental state.

Positive emotional mental state is: positive emotional mental state, positive emotional response, positive emotional concentration, A level of concentration;

Neutral emotional mental state is: positive and negative emotional reactions are normal, emotional mental state is normal, emotional reaction time is normal, B and C levels of concentration;

Negative affective mental states include: positive and negative affective overreaction, positive and negative affective insensitivity, affective mental response disturbance, negative affective mental response, emotional unresponsiveness, negative affective concentration, D-level attention concentration.

Implementing the embodiment of the present invention, due to the use of a method of threshold screening and extraction of pupil images, not only overcomes the influence of strong light on pupils with different aging degrees in pupil detection, but also can effectively detect emotional and mental states based on pupil information, At the same time, this detection method can be applied to mobile devices, which is of great significance in mobile medical care.

In addition, the pupil information-based emotional and mental state detection system and method provided by the embodiments of the present invention have been described in detail above. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; Those of ordinary skill in the art, according to the idea of the present invention, will have changes in the specific implementation and application methods. In summary, the content of this specification should not be construed as limiting the present invention.

Claims (9)

1. a kind of emotion state of mind detecting system based on pupil information, it is characterised in that the system includes：

Emotion induces module, for carrying out emotion induction to tester；

Information acquisition module, for gathering the image of each frame camera shooting；

Message processing module, for carrying out whether the image procossing of each two field picture, the calculating of pupil information and the information belong to In the judgement of neutral information；

Neutral message processing module, for the pupil diameter information of whole joint account neutrality information, obtain its average value using as Neutral pupil diameter reference value, its scope is as neutral pupil diameter threshold value；

Information feedback module, for all pupil informations to be converted into test emotion information and feed back to far-end server, judge With the emotion state of mind of diagnostic test person and its result is sent to tester.

2. the emotion state of mind detecting system based on pupil information as described in claim 1, it is characterised in that the feelings Sense, which induces module, to be included：

Material capture unit.For being extracted to emotional stimulus.

Materials handling cell.For being broadcasting material that time span is 10s by each material process.

Material broadcast unit.The material crossed for playback process.

Messaging unit.For normal emotion information and its threshold value corresponding to each material to be passed into information feedback module, As standard emotion information, the standard emotion information is transferred to information feedback module.

3. the emotion state of mind detecting system based on pupil information as described in claim 1, it is characterised in that the letter Breath acquisition module includes：

Time control unit.The time is divided by 1 frame of 2s.

Image acquisition units.The image that each frame camera collects is shot, and the image collected is sent to information processing Module.

4. the emotion state of mind detecting system based on pupil information as described in claim 1, it is characterised in that the letter Breath processing module includes：

Graphics processing unit.The image collected to information acquisition module carries out image procossing, obtains pupil image.

Information process unit.Treated pupil image is obtained, calculates and obtains pupil information.

Information judging unit.Judge whether the pupil information is pupil information under relaxation state.If so, then as neutrality Information transmission is to neutral message processing module, if it is not, being then passed to information feedback module.

5. the emotion state of mind detecting system based on pupil information as described in claim 1, peculiar is, in described Property message processing module includes：

Neutral information integration unit.Extract and cache the pupil diameter data in neutral information.

Neutral information calculating unit.Data cached average value is calculated, obtains neutral pupil diameter reference value.Obtain data cached Scope is to be used as neutral pupil diameter threshold range.

6. the emotion state of mind detecting system based on pupil information as described in claim 1, it is characterised in that the letter Breath feedback module includes：

Information calculating unit.The emotion bandwidth of tester is calculated using the pupil information of tester, emotion bandwidth not of the same race occurs Number and notice intensity to be used as with reference to information.

Information recording unit.The reference information in whole test process is recorded, and song is drawn according to emotion band data therein Line chart, the emotional responses time is calculated, obtain comprising the test emotion information with reference to information and emotional responses time and feed back to remote Hold server.

Information identifying unit.Compare test emotion information and standard emotion information, the emotion state of mind of tester is sentenced Fixed and diagnosis, obtain judgement diagnostic message and be sent to tester.

7. a kind of emotion state of mind detection method based on pupil information, it is characterised in that methods described includes：

Tester's 10s relaxation times are given, neutral pupil diameter information extraction is carried out to tester, it is straight that neutral pupil is calculated Footpath reference value and its threshold range.

Emotion inducing materials are extracted, carry out emotion induction.

Gather the pupil image of each frame tester during emotion induces.

Acquisition pupil image, progress image procossing, calculating pupil diameter and pupil watch point coordinates attentively to be used as pupil information.

Pupil information is obtained, carries out testing emotion information conversion, the emotion state of mind current to user is judged and diagnosed, And will judge to be sent to tester with the result of diagnosis, test emotion information is fed back into far-end server.

8. the emotion state of mind detection method based on pupil information as described in claim 7, it is characterised in that the figure As processing step includes：

The image collected is converted into gray level image；

Obtain in gray level image and refer to comparison area coordinate range information, filter out the reference comparison area in image and shear, obtain With reference to comparison area image and it cut by testing image with reference to comparison area；

The gray value information with reference to comparison area image is obtained, the gray value interval with reference to comparison area is filtered out, obtains the ash of judgement Spend threshold value standard；

Gray threshold standard and testing image are obtained, the testing image area grayscale that setting meets gray threshold standard is 0, is not inconsistent The testing image area grayscale for closing gray threshold standard is 255, obtains bianry image；

Bianry image is obtained, carries out opening operation and closed operation, obtains the pupil image that image procossing finishes.

9. the emotion state of mind detection method based on pupil information as described in claim 7, it is characterised in that the feelings The step of sense state of mind is judged and diagnosed, including：

Test emotion information is obtained, to the wide, narrow of emotion bandwidth, number that different type emotion bandwidth occurs, during emotional responses Between in advance calculated and judged with hysteresis, notice intensity, obtain the emotion spirit characteristic of tester；

The emotion spirit characteristic of tester is obtained, division processing is carried out to the emotion state of mind of tester, obtains positive affect Three kinds of state of mind, the neutral emotion state of mind and the negative affect state of mind emotion spirit characteristic classifications；

The emotion state of mind classification of tester is obtained, the emotion state of mind of tester is diagnosed, obtains emotion spirit The diagnostic result of state.