CN116884577A - Psychological therapy healing method, device, equipment and medium based on machine vision - Google Patents
Psychological therapy healing method, device, equipment and medium based on machine vision Download PDFInfo
- Publication number
- CN116884577A CN116884577A CN202310970208.5A CN202310970208A CN116884577A CN 116884577 A CN116884577 A CN 116884577A CN 202310970208 A CN202310970208 A CN 202310970208A CN 116884577 A CN116884577 A CN 116884577A
- Authority
- CN
- China
- Prior art keywords
- dialogue
- psychological
- picture
- patient
- aigc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/176—Dynamic expression
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Medical Informatics (AREA)
- Developmental Disabilities (AREA)
- Child & Adolescent Psychology (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The application relates to the technical field of artificial intelligence and the field of medical health, and discloses a psychological therapy healing method based on machine vision and AIGC, which comprises the following steps: acquiring initial dialogue videos of psychological disease patients and intelligent dialogue robots; extracting dialogue pictures in an initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set with the expression change of a psychological illness patient larger than a preset threshold value from the dialogue picture queue; extracting facial expressions of the psychological disease patients from the dialogue picture set by using a machine vision system, and inquiring dialogue texts corresponding to the facial expressions; and carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme based on the emotion of the patient by using AIGC to carry out psychological dispersion on the psychological illness patient. The application also provides a heart physiotherapy healing device, equipment and a storage medium based on machine vision and AIGC. The application can improve the efficiency of heart physiotherapy.
Description
Technical Field
The application relates to the technical field of artificial intelligence and the field of medical health, in particular to a psychological therapy healing method, device, equipment and medium based on machine vision and AIGC.
Background
With the development of information technology, more modern people have various psychological problems, however, a great deal of people-to-person dialogue content is required in traditional psychological treatment, and because patients suffer from psychological diseases, communication with doctors is often quite resistant, even if a scene of communication with the doctors exists, the real ideas of the doctors cannot be completely fed back to the doctors due to the doubt of the confidentiality of the doctors or the mind of the doctors on the own mindset, so that psychological diseases cannot be treated in time.
In addition, in the existing psychological therapy healing scene, the emotion of a doctor can also influence the emotion of a patient, so that the heart physiotherapy healing cannot achieve an ideal effect. If the patient selects an active treatment mode for inquiring the data such as books and the like by himself, the psychological diseases of the patient can be treated, but the experience of each person is unique, and the treatment method for inquiring the data such as books and the like cannot be matched with the user.
Disclosure of Invention
The application provides a psychological therapy healing method, device, equipment and medium based on machine vision and AIGC, and mainly aims to improve the efficiency of heart physiotherapy healing.
In order to achieve the above object, the present application provides a psychological treatment method based on machine vision and AIGC, comprising:
acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is greater than a preset threshold value from the dialogue picture queue;
extracting facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and inquiring dialogue texts corresponding to the facial expressions;
and carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
Optionally, the extracting the dialogue picture in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue includes:
inquiring the frame rate of the initial dialogue video;
and capturing the initial dialogue video according to the picture playing sequence of the initial dialogue video by using a preset video capturing tool based on the frame rate to obtain a dialogue picture queue of the initial dialogue video.
Optionally, the extracting, from the dialogue picture queue, a dialogue picture set in which the expression change of the psychological disease patient is greater than a preset threshold value includes:
step A, selecting any dialogue picture from the dialogue picture queue as a first dialogue picture;
step B, obtaining pixel values of all color channels in the first dialogue picture, and averaging the pixel values of all color channels to construct a graying picture of the first dialogue picture;
step C, obtaining pixel values of all pixel points in the gray-scale picture, and identifying the facial area of the psychological disease patient according to the pixel values to obtain facial pixels of the dialogue picture;
step D, acquiring a previous dialogue picture of the first dialogue picture in the dialogue picture queue, obtaining a second dialogue picture, and acquiring facial pixels of the second dialogue picture;
e, calculating the comprehensive pixel value difference of the face pixels in the first dialogue picture and the second dialogue picture;
f, if the difference of the comprehensive pixel values is larger than a preset pixel difference value, the expression change of the psychological disease patient is larger than a preset threshold value, and the first dialogue picture is pre-stored in a preset storage space;
step G, judging whether all the dialogue pictures in the dialogue picture queue are selected;
if the dialogue pictures in the dialogue picture queue are not all selected, returning to repeatedly execute the operations from the step A to the step F;
and if all the dialogue pictures in the dialogue picture queue are selected, step H obtains the dialogue picture set according to the dialogue pictures in the preset storage space.
Optionally, the extracting facial expression of the psychological disease patient from the dialogue picture set by using a machine vision system includes:
detecting the facial key points and facial information of each dialogue picture in the dialogue picture set by a key point detection method;
extracting features of the dialogue picture based on the facial key points and the facial information to obtain facial features;
and classifying and identifying the facial features by using a preset classifier to obtain the facial expression of the psychological disease patient.
Optionally, the querying the dialogue text corresponding to the facial expression includes:
acquiring a dialogue picture corresponding to the facial expression, and inquiring a time node of the dialogue picture;
acquiring an audio file in the initial dialogue video, and acquiring the language type of the audio file;
selecting a voice recognition tool based on the language category to perform voice recognition on the audio file to obtain a recognition text;
and acquiring text information corresponding to the time node in the identification text, and obtaining a dialogue text corresponding to the facial expression.
Optionally, the generating a psychological treatment regimen using AIGC based on the patient's emotion to psychologically dredge the psychological disease patient comprises:
analyzing causes of mood swings using AIGC based on the patient's mood and querying the medical history of the psychological illness patient;
and selecting an experience library based on the emotion fluctuation reasons and medical histories of the psychological disease patients, and selecting a psychological treatment scheme which accords with the psychological disease patients from the experience library to conduct psychological dispersion on the psychological disease patients.
Optionally, after the psychological dispersion is performed on the psychological disease patient by selecting a psychological treatment plan conforming to the psychological disease patient from the experience library, the method further comprises:
querying a communication means of a responsible physician of the psychological disorder patient and transmitting the psychological treatment plan to the responsible physician based on the communication means.
In order to solve the above problems, the present application also provides a psychological healing device based on machine vision and AIGC, the device comprising:
the video acquisition module is used for acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
the picture extraction module is used for extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is larger than a preset threshold value from the dialogue picture queue;
a text query module, configured to extract facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and query dialogue text corresponding to the facial expressions;
and the psychological treatment module is used for carrying out psychological emotion analysis on the psychological illness patient in the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
In order to solve the above-mentioned problems, the present application also provides an electronic apparatus including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the machine vision and AIGC-based psychotherapy method as described above.
In order to solve the above-mentioned problems, the present application also provides a computer-readable storage medium including a storage data area storing created data and a storage program area storing a computer program; wherein the computer program, when executed by a processor, implements a machine vision and AIGC based psychotherapy recovery method as described above.
According to the embodiment of the application, psychological emotion analysis is carried out on the psychological illness patients based on the dialogue videos of the psychological illness patients and the intelligent dialogue robot, psychological treatment schemes are generated by using AIGC based on the emotion of the patients to carry out psychological dispersion on the psychological illness patients, excessive participation of psychological doctors is not needed, so that the psychological illness patients can be described without reservation, the accuracy of subsequent treatment is ensured, and the psychological dispersion is carried out on the psychological illness patients through machine vision and AIGC therapy, so that the disease treatment efficiency is improved. Therefore, the psychological therapy method, the psychological therapy device, the psychological therapy electronic equipment and the psychological therapy computer readable storage medium based on the machine vision and the AIGC can solve the problems of low accuracy and low efficiency of the psychological therapy for the psychological disease patients.
Drawings
FIG. 1 is a flow chart of a psychological treatment method based on machine vision and AIGC according to an embodiment of the present application;
FIG. 2 is a detailed flow chart of a step in a psychological treatment method based on machine vision and AIGC according to an embodiment of the present application;
FIG. 3 is a detailed flow chart of another step in a psychological treatment method based on machine vision and AIGC according to an embodiment of the present application;
FIG. 4 is a detailed flow chart of another step in a psychological treatment method based on machine vision and AIGC according to an embodiment of the present application;
FIG. 5 is a block diagram of a psychological treatment device based on machine vision and AIGC according to an embodiment of the present application;
fig. 6 is a schematic diagram of an internal structure of an electronic device for implementing a psychological healing method based on machine vision and AIGC according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The embodiment of the application provides a psychological therapy healing method based on machine vision and AIGC. The execution subject of the psychological healing method based on machine vision and AIGC includes, but is not limited to, at least one of a server, a terminal and the like which can be configured to execute the method provided by the embodiment of the application. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. In other words, the machine vision and AIGC-based psychotherapeutic method may be performed by software or hardware installed at a remote device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Referring to fig. 1, a flow chart of a psychological treatment method based on machine vision and AIGC according to an embodiment of the present application is shown. In this embodiment, the psychological treatment method based on machine vision and AIGC includes the following steps S1-S4:
s1, acquiring initial dialogue videos of a psychological disease patient and an intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient.
In the embodiment of the application, the psychological disease patient is in psychological communication with the intelligent dialogue robot in advance, and the psychological disease patient is subjected to auxiliary inquiry of psychological diseases mainly through the intelligent dialogue robot.
Further, the initial dialogue video is a video which is recorded by the image pickup device and is responded to questions or answers provided by the intelligent dialogue robot, and sound, expression and limb actions of the psychological illness patient are recorded.
In the embodiment of the application, the intelligent dialogue robot is a computer program for automatically executing the psychological diagnosis and treatment task, and has no limitation of the shape of the substance.
In another embodiment of the present application, before the obtaining the initial dialogue video of the psychological disease patient and the intelligent dialogue robot based on the psychological diagnosis and treatment requirement of the psychological disease patient, the method further includes:
and sending the environment recording standard to the disease management patient through the intelligent dialogue robot.
In the embodiment of the present application, the environmental recording standard is an environmental requirement standard for performing psychological diagnosis and treatment on the psychological disease patient, for example, the wearing is required to cover the facial expression in a large area, and the recording environment is required to have no noise.
S2, extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is larger than a preset threshold value from the dialogue picture queue.
In the embodiment of the present application, the dialogue picture is a still image included in the initial dialogue video, typically, a certain frame in the initial dialogue video, and the facial expression, limb motion, dressing and background environment of the psychological disease patient are recorded.
In the embodiment of the application, the dialogue picture set is a picture set which is based on the purpose of psychological diagnosis and treatment, and the facial expression change and the limb action change of the psychological disease patient extracted from the initial picture queue are used as psychological diagnosis and treatment references.
Wherein the facial expression further comprises pupil changes of the psychological disorder patient.
Referring to fig. 2, further, the extracting, according to the video playing order, the dialogue picture in the initial dialogue video to obtain a dialogue picture queue includes:
s2011, inquiring the frame rate of the initial dialogue video;
s2012, screenshot is carried out on the initial dialogue video according to the picture playing sequence of the initial dialogue video by using a preset video screenshot tool based on the frame rate, so as to obtain a dialogue picture queue of the initial dialogue video.
In the embodiment of the present application, the frame rate refers to the number of frames of video displayed per second, which is generally indicated by fps (frames per second), that is, the number of frames played per second, and is generally greater than or equal to 24 frames per second. The video capture tool is a tool for capturing pictures from the initial dialogue video, and is typically a multimedia player with built-in video capture functions, such as VLC, VLC, potPlayer.
Further, the pictures in the dialogue picture queue are arranged according to the sequence, the sequence is the same as the picture playing sequence in the initial dialogue video.
Referring to fig. 3, in the embodiment of the present application, the extracting, from the dialog picture queue, a dialog picture set in which the expression change of the psychological disease patient is greater than a preset threshold value includes:
s2021, selecting any dialog picture from the dialog picture queue as a first dialog picture;
s2022, obtaining pixel values of all color channels in the first dialogue picture, and averaging the pixel values of all color channels to construct a graying picture of the first dialogue picture;
s2023, obtaining pixel values of all pixel points in the gray-scale picture, and identifying the facial area of the psychological disease patient according to the pixel values to obtain facial pixels of the dialogue picture;
s2024, acquiring a previous dialogue picture of the first dialogue picture in the dialogue picture queue, obtaining a second dialogue picture, and acquiring facial pixels of the second dialogue picture;
s2025, calculating the difference of the comprehensive pixel values of the face pixels in the first dialogue picture and the second dialogue picture;
s2026, if the difference of the comprehensive pixel values is larger than a preset pixel difference value, pre-storing the first dialogue picture into a preset storage space if the expression change of the psychological disease patient is larger than a preset threshold;
s2027, judging whether all the dialogue pictures in the dialogue picture queue are selected;
if the dialog pictures in the dialog picture queue are not all selected, the above operations S2021 to S2026 are repeatedly executed.
If all the dialogue pictures in the dialogue picture queue are selected, S2028 obtains the dialogue picture set according to the dialogue pictures in the preset storage space.
In the embodiment of the application, the pixel values of the pixel points are obtained by graying the dialogue picture, so that the operation of each channel of the first dialogue picture is avoided, and the calculation resources and the time cost are saved.
Further, the embodiment of the application judges whether the expression change of the psychological disease patient is larger than a preset threshold value or not through the pixel values of the pixel points between the dialogue pictures.
In another embodiment of the present application, after extracting the session pictures in the initial session video according to the video playing order to obtain the session picture queue, the method further includes:
and extracting limb actions of the psychological disease patient from the dialogue picture queue, and carrying out emotion prediction on the psychological disease patient according to the limb actions.
The embodiment of the application can also obtain the psychological emotion of the psychological illness patient through the limb actions.
And S3, extracting the facial expression of the psychological disease patient from the dialogue picture set by using a machine vision system, and inquiring dialogue text corresponding to the facial expression.
In the embodiment of the application, the machine vision system is a system which uses a machine to replace human eyes to be used for various measurement and judgment, a captured object can be converted into an image signal through a machine vision product and is transmitted to a special image processing system, then the image signal is converted into a digital signal according to information such as pixel distribution, brightness, color and the like, and finally the image processing system performs various operations on the digital signals to extract the characteristics of the object, and further the on-site equipment action is controlled according to the judging result of the characteristics. The dialog text is text that is recognized from the initial dialog video using speech recognition techniques.
Referring to fig. 4, in an embodiment of the present application, the extracting, by using a machine vision system, the facial expression of the psychological disease patient from the dialog picture set includes:
s301, detecting facial key points and facial information of each dialogue picture in the dialogue picture set by a key point detection method;
s302, extracting features of the dialogue picture based on the face key points and the face information to obtain face features;
s303, classifying and identifying the facial features by using a preset classifier to obtain the facial expression of the psychological illness patient.
The classifier in the embodiment of the application is a classifier based on SVM, HOG and other methods, and is a classifier which is trained by using a labeled facial expression data set in advance so as to improve expression recognition accuracy and robustness.
Further, the querying the dialogue text corresponding to the facial expression includes:
acquiring a dialogue picture corresponding to the facial expression, and inquiring a time node of the dialogue picture;
acquiring an audio file in the initial dialogue video, and acquiring the language type of the audio file;
selecting a voice recognition tool based on the language category to perform voice recognition on the audio file to obtain a recognition text;
and acquiring text information corresponding to the time node in the identification text, and obtaining a dialogue text corresponding to the facial expression.
In an embodiment of the present application, the speech recognition tool is a tool for converting a speech file into a text file, such as nuance Dragon, IBM Watson, hundred degree speech conversion API, and the like.
In another embodiment of the present application, the method further comprises extracting limb movements, i.e. gesture movements of the psychological illness patient, etc.
S4, carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
In the embodiment of the application, the content generated in artificial intelligence (AI-Generated Content, AIGC) refers to a technology for generating related content with proper generalization capability by learning and pattern recognition on existing data based on a method for generating artificial intelligence technologies such as a countermeasure network (GAN), a large-scale pre-training model and the like.
In the embodiment of the application, the psychological analysis is carried out on the psychological disease patient, namely whether the emotion of the psychological disease patient is impatient or mild, or whether the emotion of the psychological disease patient is rapid or low is analyzed according to the facial expression and the dialogue text, so that the subsequent diagnosis and treatment for the psychological disease patient are facilitated.
In an embodiment of the present application, the generating a psychological treatment plan by AIGC based on the emotion of the patient to psychological groom the patient with psychological disease includes:
analyzing causes of mood swings using AIGC based on the patient's mood and querying the medical history of the psychological illness patient;
and selecting an experience library based on the emotion fluctuation reasons and medical histories of the psychological disease patients, and selecting a psychological treatment scheme which accords with the psychological disease patients from the experience library to conduct psychological dispersion on the psychological disease patients.
In an embodiment of the present application, after the psychological treatment plan conforming to the psychological disease patient is selected from the experience library to psychologically lead the psychological disease patient, the method further includes:
inquiring a communication mode of a responsible doctor of the psychological disease patient, and transmitting the psychological treatment scheme to the responsible doctor based on the communication mode.
In the embodiment of the application, besides diagnosis and treatment can be carried out on the psychological disease patient through the AIGC, the psychological treatment scheme generated by artificial intelligence can be sent to doctors responsible for treating the psychological disease patient, and thought reference is provided for the doctors.
According to the embodiment of the application, psychological emotion analysis is carried out on the psychological illness patients based on the dialogue videos of the psychological illness patients and the intelligent dialogue robot, psychological treatment schemes are generated by using AIGC based on the emotion of the patients to carry out psychological dispersion on the psychological illness patients, excessive participation of psychological doctors is not needed, so that the psychological illness patients can be described without reservation, the accuracy of subsequent treatment is ensured, and the psychological dispersion is carried out on the psychological illness patients through machine vision and AIGC therapy, so that the disease treatment efficiency is improved. Therefore, the psychological therapy method, the psychological therapy device, the psychological therapy electronic equipment and the psychological therapy computer readable storage medium based on the machine vision and the AIGC can solve the problems of low accuracy and low efficiency of the psychological therapy for the psychological disease patients.
FIG. 5 is a schematic block diagram of the psychological treatment device based on machine vision and AIGC according to the present application.
The machine vision and AIGC-based psychotherapy device 100 of the present application may be installed in an electronic apparatus. Depending on the functions implemented, the machine vision and AIGC-based psychotherapy device may include a video acquisition module 101, a picture extraction module 102, a text query module 103, and a psychotherapy module 104. The module of the application, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
the video acquisition module 101 is configured to acquire an initial dialogue video of the psychological disease patient and the intelligent dialogue robot based on a psychological diagnosis and treatment requirement of the psychological disease patient;
the picture extraction module 102 is configured to extract dialogue pictures in the initial dialogue video according to a video playing sequence, obtain a dialogue picture queue, and extract a dialogue picture set of the psychological illness patient whose expression change is greater than a preset threshold value from the dialogue picture queue;
a text query module 103, configured to extract facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and query dialogue text corresponding to the facial expressions;
and the psychological treatment module 104 is configured to perform psychological emotion analysis on the psychological illness patient according to the facial expression and the dialogue text, obtain a patient emotion, and generate a psychological treatment scheme by using an AIGC based on the patient emotion to perform psychological dispersion on the psychological illness patient.
In detail, each module in the machine vision and AIGC-based psychological treatment device 100 according to the embodiment of the present application adopts the same technical means as the machine vision and AIGC-based psychological treatment method described in fig. 1 to 3, and can produce the same technical effects, and is not described herein.
As shown in fig. 6, there is a schematic structural diagram of an electronic device implementing a psychological healing method based on machine vision and AIGC according to the present application.
The electronic device may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program stored in the memory 11 and executable on the processor 10, such as a machine vision and AIGC based psychological healing program.
The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing Unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the entire electronic device using various interfaces and lines, executes or executes programs or modules stored in the memory 11 (for example, executes a psychological healing program based on machine vision and AIGC, etc.), and invokes data stored in the memory 11 to perform various functions of the electronic device and process data.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. The memory 11 may in other embodiments also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used not only for storing application software installed in an electronic device and various types of data, such as codes of a psychological healing program based on machine vision and AIGC, etc., but also for temporarily storing data that has been output or is to be output.
The communication bus 12 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
The communication interface 13 is used for communication between the electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface.
Fig. 6 shows only an electronic device with components, and it will be understood by those skilled in the art that the structure shown in fig. 6 is not limiting of the electronic device and may include fewer or more components than shown, or may combine certain components, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power source (such as a battery) for supplying power to the respective components, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device may further include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described herein.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The psychological healing program based on machine vision and AIGC stored by the memory 11 in the electronic device is a combination of a plurality of computer programs, which when run in the processor 10, can implement:
acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is greater than a preset threshold value from the dialogue picture queue;
extracting facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and inquiring dialogue texts corresponding to the facial expressions;
and carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
In particular, the specific implementation method of the processor 10 on the computer program may refer to the description of the relevant steps in the corresponding embodiment of fig. 1, which is not repeated herein.
Further, the electronic device integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a non-volatile computer readable storage medium. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present application also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:
acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is greater than a preset threshold value from the dialogue picture queue;
extracting facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and inquiring dialogue texts corresponding to the facial expressions;
and carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means. The terms second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.
Claims (10)
1. A method of psychological healing based on machine vision and AIGC, the method comprising:
acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is greater than a preset threshold value from the dialogue picture queue;
extracting facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and inquiring dialogue texts corresponding to the facial expressions;
and carrying out psychological emotion analysis on the psychological illness patient based on the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
2. The method for psychotherapeutic healing based on machine vision and AIGC according to claim 1, wherein the extracting the dialogue pictures in the initial dialogue video according to the video play order to obtain the dialogue picture queue comprises:
inquiring the frame rate of the initial dialogue video;
and capturing the initial dialogue video according to the picture playing sequence of the initial dialogue video by using a preset video capturing tool based on the frame rate to obtain a dialogue picture queue of the initial dialogue video.
3. The machine vision and AIGC based psychotherapy recovery method of claim 1, wherein the extracting the dialog picture set from the dialog picture queue for the psychologically ill patient having an expression change greater than a preset threshold comprises:
step A, selecting any dialogue picture from the dialogue picture queue as a first dialogue picture;
step B, obtaining pixel values of all color channels in the first dialogue picture, and averaging the pixel values of all color channels to construct a graying picture of the first dialogue picture;
step C, obtaining pixel values of all pixel points in the gray-scale picture, and identifying the facial area of the psychological disease patient according to the pixel values to obtain facial pixels of the dialogue picture;
step D, acquiring a previous dialogue picture of the first dialogue picture in the dialogue picture queue, obtaining a second dialogue picture, and acquiring facial pixels of the second dialogue picture;
e, calculating the comprehensive pixel value difference of the face pixels in the first dialogue picture and the second dialogue picture;
f, if the difference of the comprehensive pixel values is larger than a preset pixel difference value, the expression change of the psychological disease patient is larger than a preset threshold value, and the first dialogue picture is pre-stored in a preset storage space;
step G, judging whether all the dialogue pictures in the dialogue picture queue are selected;
if the dialogue pictures in the dialogue picture queue are not all selected, returning to repeatedly execute the operations from the step A to the step F;
and if all the dialogue pictures in the dialogue picture queue are selected, step H obtains the dialogue picture set according to the dialogue pictures in the preset storage space.
4. The machine vision and AIGC based psychotherapy recovery method of claim 1, wherein the extracting facial expression of the psychologically ill patient from the conversation picture set using a machine vision system comprises:
detecting the facial key points and facial information of each dialogue picture in the dialogue picture set by a key point detection method;
extracting features of the dialogue picture based on the facial key points and the facial information to obtain facial features;
and classifying and identifying the facial features by using a preset classifier to obtain the facial expression of the psychological disease patient.
5. The machine vision and AIGC based psychotherapy recovery method of claim 1, wherein said querying the dialog text corresponding to the facial expression comprises:
acquiring a dialogue picture corresponding to the facial expression, and inquiring a time node of the dialogue picture;
acquiring an audio file in the initial dialogue video, and acquiring the language type of the audio file;
selecting a voice recognition tool based on the language category to perform voice recognition on the audio file to obtain a recognition text;
and acquiring text information corresponding to the time node in the identification text, and obtaining a dialogue text corresponding to the facial expression.
6. The machine vision and AIGC based psychotherapy recovery method of claim 1, wherein the generating a psychotherapy regimen using AIGC based on the patient's emotion comprises:
analyzing causes of mood swings using AIGC based on the patient's mood and querying the medical history of the psychological illness patient;
and selecting an experience library based on the emotion fluctuation reasons and medical histories of the psychological disease patients, and selecting a psychological treatment scheme which accords with the psychological disease patients from the experience library to conduct psychological dispersion on the psychological disease patients.
7. The machine vision and AIGC based psychotherapy recovery method of claim 6, wherein after selecting a psychotherapy regimen from the library of experiences that meets the psychological condition patient to psychologically dredge the psychological condition patient, the method further comprises:
querying a communication means of a responsible physician of the psychological disorder patient and transmitting the psychological treatment plan to the responsible physician based on the communication means.
8. A machine vision and AIGC-based psychotherapy device, the device comprising:
the video acquisition module is used for acquiring initial dialogue videos of the psychological disease patient and the intelligent dialogue robot based on psychological diagnosis and treatment requirements of the psychological disease patient;
the picture extraction module is used for extracting dialogue pictures in the initial dialogue video according to the video playing sequence to obtain a dialogue picture queue, and extracting a dialogue picture set of which the expression change of the psychological illness patient is larger than a preset threshold value from the dialogue picture queue;
a text query module, configured to extract facial expressions of the psychological disease patient from the dialogue picture set by using a machine vision system, and query dialogue text corresponding to the facial expressions;
and the psychological treatment module is used for carrying out psychological emotion analysis on the psychological illness patient in the facial expression and the dialogue text to obtain the emotion of the patient, and generating a psychological treatment scheme by utilizing AIGC based on the emotion of the patient to carry out psychological dispersion on the psychological illness patient.
9. An electronic device, the electronic device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the machine vision and AIGC-based psychotherapy method of any of claims 1 to 7.
10. A computer-readable storage medium comprising a storage data area storing created data and a storage program area storing a computer program; wherein the computer program, when executed by a processor, implements the machine vision and AIGC-based psychotherapy method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310970208.5A CN116884577A (en) | 2023-08-02 | 2023-08-02 | Psychological therapy healing method, device, equipment and medium based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310970208.5A CN116884577A (en) | 2023-08-02 | 2023-08-02 | Psychological therapy healing method, device, equipment and medium based on machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116884577A true CN116884577A (en) | 2023-10-13 |
Family
ID=88266282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310970208.5A Pending CN116884577A (en) | 2023-08-02 | 2023-08-02 | Psychological therapy healing method, device, equipment and medium based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116884577A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118016277A (en) * | 2024-02-21 | 2024-05-10 | 众爱数字医疗科技(广东)有限公司 | Digital five-step disease management model system developed based on AIGC technology |
-
2023
- 2023-08-02 CN CN202310970208.5A patent/CN116884577A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118016277A (en) * | 2024-02-21 | 2024-05-10 | 众爱数字医疗科技(广东)有限公司 | Digital five-step disease management model system developed based on AIGC technology |
| CN118016277B (en) * | 2024-02-21 | 2025-01-24 | 众爱数字医疗科技(广东)有限公司 | Digital five-step disease management model system developed based on AIGC technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115004308A (en) | Method and system for providing an interface for activity recommendations | |
| CN114783618B (en) | Traditional Chinese medicine model training method and device based on knowledge structure and four diagnostic traits | |
| CN114581823A (en) | Virtual reality video emotion recognition method and system based on time sequence characteristics | |
| CN112307947A (en) | Method and apparatus for generating information | |
| CN116311539B (en) | Sleep motion capturing method, device, equipment and storage medium based on millimeter waves | |
| Das et al. | Interpretable self-supervised facial micro-expression learning to predict cognitive state and neurological disorders | |
| CN113990482A (en) | Health data processing system and method | |
| CN118942736A (en) | Intelligent pre-diagnosis method, device, intelligent device and storage medium | |
| CN118319307A (en) | Dynamic abnormal emotion early warning and monitoring system based on Internet of things | |
| CN116523734A (en) | Video generation method and device based on artificial intelligence, electronic equipment and medium | |
| Weerasinghe et al. | Real-Time Multimodal Cognitive Assistant for Emergency Medical Services | |
| CN115206491A (en) | Sleep intervention equipment control method and device, storage medium and computer equipment | |
| CN116884577A (en) | Psychological therapy healing method, device, equipment and medium based on machine vision | |
| CN118016326A (en) | Internet-based intelligent patient care follow-up system and method | |
| Dunn et al. | Brief review and primer of key terminology for artificial intelligence and machine learning in hypertension | |
| CN111223549B (en) | A mobile terminal system and method for disease prevention based on posture correction | |
| CN116798623A (en) | Sleep evaluation method, device, equipment and storage medium based on artificial intelligence | |
| CN114493952A (en) | Education software data processing system and method based on big data | |
| US20190163872A1 (en) | Cognitive acupuncture assistant for treatment suggestions | |
| CN118097345A (en) | Medical image recognition method, device, electronic equipment and storage medium | |
| CN114202741B (en) | User learning monitoring method and device, computer equipment and storage medium | |
| CN117789931A (en) | A multi-modal fusion learning method and system for sleep management | |
| CN116702776A (en) | Multi-task semantic division method, device, equipment and medium based on cross-Chinese and western medicine | |
| CN119538037B (en) | Emotion recognition method, device, equipment and medium | |
| CN118248346B (en) | Case symptom extraction method and device based on pseudo tag, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |