WO2018028166A1 - Remote interaction system and method based on virtual reality head-mounted display device - Google Patents
Remote interaction system and method based on virtual reality head-mounted display device Download PDFInfo
- Publication number
- WO2018028166A1 WO2018028166A1 PCT/CN2017/073978 CN2017073978W WO2018028166A1 WO 2018028166 A1 WO2018028166 A1 WO 2018028166A1 CN 2017073978 W CN2017073978 W CN 2017073978W WO 2018028166 A1 WO2018028166 A1 WO 2018028166A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- display device
- mounted display
- virtual reality
- reality head
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B17/00—Surgical instruments, devices or methods
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
- A61B5/021—Measuring pressure in heart or blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration or pH-value ; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid or cerebral tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B7/00—Instruments for auscultation
- A61B7/003—Detecting lung or respiration noise
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B7/00—Instruments for auscultation
- A61B7/02—Stethoscopes
- A61B7/04—Electric stethoscopes
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Definitions
- the present invention relates to the field of medical treatment, and in particular to the field of telemedicine, and more particularly to a remote interactive system and method based on a virtual reality head mounted display device.
- Telemedicine is a special way for hospitals to treat patients in recent years, which is conducive to alleviating medical resources and improving the efficiency of diagnosis and treatment.
- treatment for some critical illnesses is generally concentrated in major hospitals in major cities.
- the doctors of different qualifications in different hospitals and hospitals in different departments of the hospital will be affected by the above external factors for the diagnosis and treatment of patients with severe diseases, which may result in patients not enjoying the best diagnosis and treatment results.
- the existing telemedicine system can integrate the superior medical resources of each hospital to treat patients with various diseases steadily, and eliminate the limitations of realistic factors such as distance location and asymmetry of patient diagnosis and treatment information.
- the technical problem to be solved by the present invention is to provide a virtual reality based head based on the shortcomings of the existing telemedicine interactive technology that the support system is difficult to provide accurate and accurate surgical guidance and effective solutions for the surgical site.
- a remote interactive system and method for a wearable display device that provides an authoritative surgical guide and an effective solution for the surgical field.
- the present invention provides a remote interactive system based on a virtual reality head mounted display device, including a clinical system And a support system, the clinical system comprising a data collection device, a data transmission channel, a cloud database, and a data exchange hub, the support system comprising a virtual reality head mounted display device;
- the data collection device is connected to the data transmission channel for collecting human physiological data
- the data transmission channel includes at least one of an Ethernet, a WiFi, a wireless private network, a telephone PSTN network, a satellite, or a mobile Internet, for transmitting data;
- the cloud database connecting the data transmission channel, including a cloud computing platform and a cloud storage platform, for providing a unified data access interface and arranging corresponding data access rights;
- the data exchange hub is connected to the data transmission channel for completing data exchange between the clinical system and the support system;
- the virtual reality head-mounted display device is connected to the data exchange hub by a network, and is configured to generate a corresponding image according to the physiological data.
- the system has a human body simulation processor for generating a three-dimensional model of a patient site, and is used to generate a three-dimensional model of the surgical instrument.
- a tool simulation processor, the virtual reality head mounted display device having a screen for dynamically displaying a three-dimensional model of the patient site and a three-dimensional model of the surgical instrument.
- the display screen has a graphical user interface for selectively displaying the physiological data.
- the virtual reality head-mounted display device has a motion displacement sensing module, and the support system further has the motion displacement sensing A wearable accessory that is electrically connected to the module.
- the system has a virtual human body model library for storing a three-dimensional model of human tissue and organs.
- the support system and the clinical system are geographically distant from each other.
- the cloud computing platform includes a first hardware system and a first software system
- the first hardware system includes a first server and a first storage system
- the first software system comprising a first operating system and a first database system
- the cloud storage platform includes a second hardware system and a second software system, where the second hardware system includes A second server and a second storage system, the second software system comprising a second operating system and a second database system.
- the present invention also provides a remote interactive medical method based on a virtual reality head mounted display device, the method comprising the following steps:
- the medical system of the clinical system uses the data collection device to collect the physiological data of the patient and sends it to the cloud database through the data transmission channel;
- the physiological data in the cloud database is transmitted to the virtual reality head-mounted display device through the data exchange hub;
- the virtual reality head-mounted display device establishes and dynamically displays a three-dimensional model of a patient's diseased part according to the physiological data
- the virtual reality head-mounted display device establishes and dynamically displays a three-dimensional model of a surgical instrument for treating a patient's diseased part of the patient;
- a medical staff supporting the system manipulates a three-dimensional model of the surgical instrument to perform a virtual surgical treatment on a three-dimensional model of the patient's patient site, and the virtual reality head-mounted display device records the support system for performing a surgical procedure
- the gesture operation step is combined with the three-dimensional model of the patient's patient site and the three-dimensional model of the surgical instrument to be transmitted to the cloud database through the data exchange center for reference by the clinical system.
- the method includes the following steps:
- the screen display acquires and dynamically displays a three-dimensional model of the patient's site.
- the method includes the following steps: the implementer of the virtual surgery in the support system wears the virtual reality head mounted display device sexually connected wearable accessories.
- a remote interactive medical system and method based on virtual reality head mounted display device provided by the present invention Has the following beneficial effects:
- the virtual reality head-mounted display device provides the medical staff of the support system with the actual physiological data of the patient's patient part and the real dynamic three-dimensional model, and can generate a three-dimensional model of the medical device required for various operations. Therefore, the medical staff of the support system can formulate a corresponding treatment plan according to the tamping dynamic three-dimensional model of the patient part, and perform virtual surgery using the three-dimensional model of the medical device, and the virtual reality head-mounted display device records support system is formulated.
- the treatment plan transmits data related to the virtual surgery to the cloud database for reference by the clinical system.
- the data collection device may include multiple professional-level medical information collection devices such as a multi-parameter monitor, an electrocardiogram, a high-definition speculum, an electronic stethoscope, a camera, etc., and may also be accessed according to work needs. Many other medical information collection devices. Realize the physical examination of various physiological indicators, such as electrocardiogram, body temperature, pulse, blood pressure, blood oxygen, heart sound, lung sound, eye, nose and throat, etc., so that medical personnel can obtain more comprehensive diagnostic data for analysis and judgment.
- the medical information collected above can be connected to the private network through a wired network, a WiFi wireless network, a satellite network, or a 3G/4G wireless broadband network to ensure the security of data transmission.
- the cloud database implements real interaction or offline interaction between multiple medical diagnosis terminals.
- the cloud database implements network storage and control interaction of multiple types of data of multiple medical diagnostic terminals.
- the medical staff of the support system can realize the physiological data of the patient through the virtual reality helmet display device, and the medical system of the clinical system can also obtain the treatment plan provided by the support system by accessing the cloud database, and realize the innovative telemedicine. Diagnostic application mode.
- the massive data and data of the sick patient are stored in the cloud database, which facilitates access and access by authorized institutions at various levels through various devices, and realizes disease diagnosis, medical quality tracking, and statistical analysis of various services. It can also be based on big data to realize medical knowledge base, human-oriented health tracking and regional disease monitoring.
- FIG. 1 is a schematic structural diagram of a remote interactive medical system based on a virtual reality head mounted display device according to Embodiment 1 of the present invention
- FIG. 2 is a schematic structural diagram of a virtual reality head-mounted display device according to Embodiment 1 of the present invention
- 3 is a flow chart of steps of a remote interactive medical method based on a virtual reality head mounted display device according to Embodiment 2 of the present invention
- step S3 is a specific flowchart of step S3 in the remote interactive medical method based on the virtual reality head mounted display device according to the second embodiment of the present invention.
- the present invention aims to provide a virtual reality based headset.
- the remote interaction system and method of the display device has the core idea of: acquiring the physiological data collected by the clinical medical structure by using the virtual reality head-mounted display device, and then generating the corresponding three-dimensional model image for the medical personnel of the support system to formulate and virtualize A corresponding treatment regimen is transmitted to the cloud database for reference by the clinical system.
- Embodiment 1 The remote interactive medical system based on the virtual reality head mounted display device 201 provided by the present invention, see FIG. 1, the system includes a clinical system 100 and a support system 200, and the clinical system 100 and the support system 200 are geographically isolated from each other.
- the clinical system 100 includes a data collection device 101, a data transmission channel 102, a cloud database 103, a data exchange hub 104, and a virtual reality head mounted display device 201, the support system 200 including a virtual reality head mounted display device 201 and wearable Accessory 202;
- the data collection device 101 is connected to the data transmission channel 102 for collecting human physiological data
- the data transmission channel 102 including at least one of an Ethernet, a WiFi, a wireless private network, a telephone PSTN network, a satellite, or a mobile Internet, for transmitting data;
- the cloud database 103 is connected to the data transmission channel 102, and includes a cloud computing platform and a cloud storage platform, for providing a unified data access interface and arranging corresponding data access rights;
- the data exchange hub 104 is connected to the data transmission channel 102 for completing data exchange between the clinical system 100 and the support system 200;
- the virtual reality head mounted display device 201 is connected to the data exchange hub 104 for generating a corresponding image according to the physiological data.
- the wearing accessory 202 is connected to the virtual reality head mounted display device 201 in a network.
- the data collection device 101 includes a portable multi-parameter vital sign monitor for detecting blood pressure, blood oxygen, pulse, body temperature, and electrocardiogram, a 12-lead network electrocardiograph, and an optical glimpse of the ear, nose and throat.
- a portable multi-parameter vital sign monitor for detecting blood pressure, blood oxygen, pulse, body temperature, and electrocardiogram, a 12-lead network electrocardiograph, and an optical glimpse of the ear, nose and throat.
- Mirror, optical goggles for detecting skin, professional cardiopulmonary high-fidelity electronic stethoscope, and webcam are used to collect various life indicators and physiological characteristics of patients.
- the data collection device 101 may also include other detection devices in the future.
- the data transmission channel 102 provides the entire clinical system 100 with data transmission between the collection, storage, processing, and transmission links, and the data transmission channel 102 performs transmission link encryption through the VPN.
- the clinical system 100 supports a variety of network transmission means such as wired, wireless, and satellite, and the physiological data can be uploaded to the cloud database 103 through the network.
- the cloud computing platform of the cloud database 103 includes a first hardware system and a first software system, the first hardware system includes a first server and a first storage system, and the first software system includes a first operating system and a first database system; a cloud storage platform of the cloud database 103 includes a second hardware system and a second software system, the second hardware system including a second server and a second storage system, The second software system includes a second operating system and a second database system.
- This part adopts cloud computing technology to realize cloud computing platform and cloud storage platform to realize a secure and scalable computing environment.
- Cloud Database 03 can be deployed in the hospital's computer room or deployed on a proprietary private cloud provided by a third party.
- the medical staff supporting the system 200 and the medical staff of the clinical system 100 can realize the interaction of the medical information through the cloud database 103. If the network is disconnected, the support system 200 and the clinical system 100 can also operate separately, and the connection is restored in the network. At this time, the data is uploaded to the cloud database 103.
- the human body simulation processor 221 is used to generate a three-dimensional model of the patient's site. In the virtual surgery synchronization environment, the human body simulation processor 221 first collects data obtained by performing CT scan on the patient's patient part before the operation. It should be noted that the human body simulation processor 221 passes the DICOM 3.0 data format in this embodiment. Data collection, DICOM (digital imaging and communication in medicine), which is a standard format for medical digital imaging and communication data, can promote interoperability between medical imaging devices and provide a data for medical information. Exchange standards.
- DICOM digital imaging and communication in medicine
- the three-dimensional model of the corresponding part generated in advance is taken out in the virtual human model library 211 for the patient's patient part, and the B-scan curve of the CT scan data of the body part of the real patient part of the patient is three-dimensionally reconstructed, bbox algorithm, Fourier algorithm, small value dichotomy and other algorithms are processed to complete the geometric modeling, computational modeling and deformation calculation of physiological data.
- 3DMAX10 the three-dimensional model reconstruction of the physiological characteristics of the patient, the mapping and the physiological characteristics of the organs and tissues, and the collision detection and graphic drawing of the skeletal skin tissue, and the three-dimensional model of the patient's diseased part.
- the tool simulation processor 231 is used to generate a three-dimensional model of the surgical instrument.
- the tool emulation processor 231 writes a three-dimensional model of various surgical instruments used in virtual surgery through a Virtual Reality Modeling Language (VRML), for example, a hand drill, a conventional disinfecting drape, a scalpel, an electric knife, and cotton.
- a three-dimensional model of the sheet and gauze is output as the display content of the subsequent virtual reality head mounted display device 201.
- VRML is an abbreviation of virtual reality modeling language. It is a standard for describing virtual scenes. It defines the language descriptions commonly used in 3D applications, such as hierarchical transformation, light source, pilot, geometry, animation, fog, material properties and texture. Mapping, etc., and has a simple behavioral characterization function. Assume One of the main goals of VRML is to ensure that it becomes an efficient three-dimensional file exchange format between multiple virtual reality systems or their components.
- the on-screen display 241 can dynamically display a three-dimensional model of the patient site and a three-dimensional model of the surgical instrument.
- the screen display 241 has a graphical user interface for selectively and dynamically displaying the physiological data.
- the virtual reality head mounted display device 201 displays a partial partial area in the area in advance.
- the above examples of real dynamic electrocardiogram data are only examples of many human physiological dynamic data, and are not limitations of the present invention. Since comprehensive medical diagnostic physiological data is the basis for implementing remote interaction, there are more new ones in the future.
- the physiological data needs to be supported by the graphical user interface, and can be dynamically displayed through a graphical user interface (GUI) set in the virtual reality head mounted display device 201.
- GUI graphical user interface
- the virtual human body model library 211 is used to store a three-dimensional model of a human tissue organ that is generated in advance.
- the motion displacement sensing module 251 is configured to sense and record motion track information data of the wear accessory 202.
- the wearing accessory 202 is used to be worn by an expert who performs virtual surgery on the support system 200, and the wearing accessory 202 is electrically connected to the motion displacement sensing module 251.
- the present invention provides a remote interactive medical system based on the virtual reality head mounted display device 201, which is applicable to the remote interactive medical system based on the virtual reality head mounted display device 201 shown in the first embodiment, see FIG. 3.
- the method includes:
- the medical staff of the clinical system 100 uses the data collection device 101 to collect the physiological data of the patient and send it to the cloud database 103 through the data transmission channel 102.
- the medical system 100 medical staff detects the blood ray, blood oxygen, pulse, body temperature, electrocardiogram multi-parameter vital signs monitor, 12-lead network electrocardiograph, detects the optical otoscope of the ear, nose and throat, and detects the optical speculum of the skin. , professional cardio-pulmonary high-fidelity electronic stethoscope, CT scan and webcam to obtain various physiological data of patients.
- the virtual reality head mounted display device 201 receives the physiological data.
- the physiological data in the cloud database 103 is transmitted to the virtual reality head mounted display device 201 through the data exchange hub 104.
- the virtual reality head mounted display device 201 acquires a patient's part Physiological data.
- the virtual reality head-mounted display device 201 collects data obtained by a medical staff of the pre-operative clinical system 100 on a real CT scan of a patient's patient part (by DICOM 3.0 data format acquisition) .
- the virtual reality head-mounted display device 201 establishes and dynamically displays a three-dimensional model of the patient's patient's part according to the physiological data. Referring to FIG. 4, the following steps are specifically included:
- the SA extracts a three-dimensional model of the human tissue and organs corresponding to the patient's part from the virtual human model library 211.
- a three-dimensional model of a human tissue organ corresponding to a patient's patient's disease site generated in advance is taken out in the virtual human model library 211.
- the 3D model of the physiological features of the patient's diseased part is reconstructed, mapped, and the organ and tissue physiological characteristics are given, and the bone skin tissue collision detection and graphic drawing are performed to generate a three-dimensional model of the patient's diseased part.
- the SC and the screen display 241 acquire and implement a three-dimensional model for dynamically displaying the patient's part.
- the virtual reality head-mounted display device 201 establishes and dynamically displays a three-dimensional model of a surgical instrument for treating a patient's patient's condition.
- the tool emulation processor 231 writes a three-dimensional model of various surgical instruments used in virtual surgery through a Virtual Reality Modeling Language (VRML), for example, a hand drill, a conventional disinfecting drape, a scalpel, an electric knife, and cotton.
- VRML Virtual Reality Modeling Language
- the three-dimensional model of the sheet and the gauze is output as the display content of the screen display 241 of the virtual reality head mounted display device 201.
- VRML is an abbreviation of virtual reality modeling language. It is a standard for describing virtual scenes.
- the virtual surgery implementer of the support system 200 manipulates the three-dimensional model of the surgical instrument to perform virtual surgery on the three-dimensional model of the patient's patient site, and the virtual reality head mounted display device 201 records the support system 200
- the gesture operation step of the surgical procedure is performed, and the three-dimensional model of the patient's patient site and the three-dimensional model of the surgical instrument are combined to form a complete surgical plan and transmitted to the cloud database 103 through the data exchange center 104 for reference by the clinical system 100.
- the remote interactive medical method based on the virtual reality head mounted display device 201 further includes the following steps:
- the implementer of the virtual surgery in the support system 200 wears the wear accessory 202 that is networked with the virtual reality head mounted display device 201.
- the wear accessory 202 worn by the implementer of the virtual surgery includes a finger sleeve of ten fingers, and the displacement sensing module can sense and record the movement track of the finger sleeve by inductively connecting with the finger socket network, thereby recording the virtual surgery
- the implementer performs a gesture of virtual surgery.
- the medical staff of the SE, clinical system 100 acquires a surgical plan.
- the medical staff of the clinical system 100 can obtain the surgical plan provided by the support system 200 by accessing the cloud database 103.
- step SD is completed before step S5 and after step S4, and step SE is in step S.
- the remote interactive medical system and method based on the virtual reality head mounted display device 201 provided by the present invention has the following beneficial effects:
- the virtual reality head-mounted display device 201 provides the medical staff supporting the system 200 with the actual physiological data of the patient's diseased part and the real dynamic three-dimensional model, and can generate various medical instruments required for the operation.
- the three-dimensional model whereby the medical staff of the support system 200 can formulate a corresponding treatment plan according to the tamping dynamic three-dimensional model of the patient site, and perform virtual surgery using the three-dimensional model of the medical device, the virtual reality head-mounted display device 201
- the treatment plan developed by the support system 200 is recorded and the data related to the virtual surgery is transmitted to the cloud database 103 for reference by the clinical system 100.
- the data collection device 101 may include an access multi-parameter monitor, an electrocardiogram, a high-definition speculum, and an electronic
- An access multi-parameter monitor such as an electrocardiogram, a high-definition speculum
- an electronic such as an electrocardiogram, a high-definition speculum
- an electronic such as an electrocardiogram, a high-definition speculum
- an electronic such as stethoscopes and cameras can also access more medical information collection devices according to work needs. Realize the physical examination of various physiological indicators, such as electrocardiogram, body temperature, pulse, blood pressure, blood oxygen, heart sound, lung sound, eye, nose and throat, etc., so that medical personnel can obtain more comprehensive diagnostic data for analysis and judgment.
- the medical information collected above can be connected to the private network through a wired network, a WiFi wireless network, a satellite network, or a 3G/4G wireless broadband network to ensure the security of data transmission.
- the cloud database 103 implements real interaction or offline interaction between multiple medical diagnosis terminals.
- the cloud database 103 enables network storage and control interaction of multiple types of data for a plurality of medical diagnostic terminals.
- the medical personnel of the support system 200 can realize the physiological data of the patient through the virtual reality helmet display device, and the medical staff of the clinical system 100 can also obtain the treatment plan provided by the support system 200 by accessing the cloud database 103, thereby realizing innovative telemedicine diagnostic application model.
- the massive data and data of the patient are stored in the cloud database 103, which facilitates access and access by various authorized organizations at various levels, and realizes disease diagnosis, medical quality tracking, and statistical analysis of various services. Etc., it can also be based on big data, to achieve medical knowledge base, human-oriented health tracking and regional disease monitoring.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Cardiology (AREA)
- Physiology (AREA)
- Optics & Photonics (AREA)
- Acoustics & Sound (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pulmonology (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Radiology & Medical Imaging (AREA)
- Vascular Medicine (AREA)
- User Interface Of Digital Computer (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
一种基于虚拟现实头戴式显示装置的远程互动系统及方法 Remote interaction system and method based on virtual reality head-mounted display device
技术领域 Technical field
[0001] 本发明涉及医疗领域, 具体涉及远程医疗领域, 更具体而言, 涉及一种基于虚 拟现实头戴式显示装置的远程互动系统及方法。 [0001] The present invention relates to the field of medical treatment, and in particular to the field of telemedicine, and more particularly to a remote interactive system and method based on a virtual reality head mounted display device.
背景技术 Background technique
[0002] 远程医疗是近年来医院治疗患者的一种特殊方式, 有利于缓解医疗资源紧张, 提高诊断治疗效率。 特别是针对一些重症疾病治疗一般都集中在各大城市的各 大医院。 不同区域的各医院与医院中各个科室中不同资历的各个医生对患有重 症疾病患者所实施的诊断治疗方式均会受到以上外界因素影响, 潜在导致病患 不能享受到最佳的诊断治疗结果。 现有的远程医疗系统可整合各医院优势专科 医疗资源对各类病患者进行稳妥地治疗, 排除距离位置与病患诊疗资讯不对称 等现实因素的限制。 [0002] Telemedicine is a special way for hospitals to treat patients in recent years, which is conducive to alleviating medical resources and improving the efficiency of diagnosis and treatment. In particular, treatment for some critical illnesses is generally concentrated in major hospitals in major cities. The doctors of different qualifications in different hospitals and hospitals in different departments of the hospital will be affected by the above external factors for the diagnosis and treatment of patients with severe diseases, which may result in patients not enjoying the best diagnosis and treatment results. The existing telemedicine system can integrate the superior medical resources of each hospital to treat patients with various diseases steadily, and eliminate the limitations of realistic factors such as distance location and asymmetry of patient diagnosis and treatment information.
[0003] 但是, 如何为外科手术现场提供更精准权威实吋的治疗方案, 是目前远程医疗 发展中亟待解决的问题。 为此, 有必要设计一种新的基于虚拟现实头戴式显示 装置的远程互动医疗系统及方法, 以解决上述问题。 [0003] However, how to provide a more precise and authoritative treatment plan for the surgical site is an urgent problem to be solved in the development of telemedicine. To this end, it is necessary to design a new remote interactive medical system and method based on a virtual reality head-mounted display device to solve the above problems.
技术问题 technical problem
[0004] 本发明要解决的技术问题在于, 针对现有远程医疗互动技术中支持系统难以为 外科手术现场提供准确权威实吋的手术指导及有效的解决方案的缺陷, 提供一 种基于虚拟现实头戴式显示装置的远程互动系统及方法, 其可为外科手术现场 提供权威实吋的手术指导及有效的解决方案。 [0004] The technical problem to be solved by the present invention is to provide a virtual reality based head based on the shortcomings of the existing telemedicine interactive technology that the support system is difficult to provide accurate and accurate surgical guidance and effective solutions for the surgical site. A remote interactive system and method for a wearable display device that provides an authoritative surgical guide and an effective solution for the surgical field.
问题的解决方案 Problem solution
技术解决方案 Technical solution
[0005] 本发明就上述技术问题而提出的技术方案如下: [0005] The technical solution proposed by the present invention with respect to the above technical problems is as follows:
[0006] 本发明提供一种基于虚拟现实头戴式显示装置的远程互动系统, 包括临床系统 和支持系统, 所述临床系统包括数据采集设备、 数据传输通道、 云端数据库以 及数据交换中枢, 所述支持系统包括虚拟现实头戴式显示装置; The present invention provides a remote interactive system based on a virtual reality head mounted display device, including a clinical system And a support system, the clinical system comprising a data collection device, a data transmission channel, a cloud database, and a data exchange hub, the support system comprising a virtual reality head mounted display device;
[0007] 所述数据采集设备, 连接所述数据传输通道, 用于采集人体生理数据; [0007] the data collection device is connected to the data transmission channel for collecting human physiological data;
[0008] 所述数据传输通道, 包括以太网、 WiFi、 无线专网、 电话 PSTN网络、 卫星或 移动互联网中的至少一种, 用于传输数据; [0008] the data transmission channel includes at least one of an Ethernet, a WiFi, a wireless private network, a telephone PSTN network, a satellite, or a mobile Internet, for transmitting data;
[0009] 所述云端数据库, 连接所述数据传输通道, 包括云计算平台和云存储平台, 用 于提供统一的数据访问接口及安排相应的数据访问权限; [0009] the cloud database, connecting the data transmission channel, including a cloud computing platform and a cloud storage platform, for providing a unified data access interface and arranging corresponding data access rights;
[0010] 所述数据交换中枢, 连接所述数据传输通道, 用于完成所述临床系统与所述支 持系统之间的数据的交换; [0010] the data exchange hub is connected to the data transmission channel for completing data exchange between the clinical system and the support system;
[0011] 所述虚拟现实头戴式显示装置, 网络连接所述数据交换中枢, 用于根据所述生 理数据生成相应的影像。 [0011] The virtual reality head-mounted display device is connected to the data exchange hub by a network, and is configured to generate a corresponding image according to the physiological data.
[0012] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述系 统具有用于生成病患部位的三维模型的人体仿真处理器, 用于生成手术器械的 三维模型的工具仿真处理器, 所述虚拟现实头戴式显示装置具有用于动态实吋 展示所述病患部位的三维模型和手术器械的三维模型的屏显。 [0012] In the remote interactive medical system based on the virtual reality head mounted display device provided by the present invention, the system has a human body simulation processor for generating a three-dimensional model of a patient site, and is used to generate a three-dimensional model of the surgical instrument. A tool simulation processor, the virtual reality head mounted display device having a screen for dynamically displaying a three-dimensional model of the patient site and a three-dimensional model of the surgical instrument.
[0013] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述显 屏具有一图形用户界面, 用于选择性地显示所述生理数据。 [0013] In the remote interactive medical system based on the virtual reality head mounted display device provided by the present invention, the display screen has a graphical user interface for selectively displaying the physiological data.
[0014] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述虚 拟现实头戴式显示装置具有运动位移感应模块, 所述支持系统还具有与所述运 动位移感应模块电性连接的穿戴配件。 [0014] In the remote interactive medical system based on the virtual reality head-mounted display device provided by the present invention, the virtual reality head-mounted display device has a motion displacement sensing module, and the support system further has the motion displacement sensing A wearable accessory that is electrically connected to the module.
[0015] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述系 统具有用于存储人体组织器官三维模型的虚拟人体模型库。 [0015] In the remote interactive medical system based on the virtual reality head mounted display device provided by the present invention, the system has a virtual human body model library for storing a three-dimensional model of human tissue and organs.
[0016] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述支 持系统与所述临床系统分布在地域上相互远离。 [0016] In the remote interactive medical system based on the virtual reality head-mounted display device provided by the present invention, the support system and the clinical system are geographically distant from each other.
[0017] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统中, 所述云 计算平台包括第一硬件系统和第一软件系统, 所述第一硬件系统包括第一服务 器和第一存贮系统, 所述第一软件系统包括第一操作系统和第一数据库系统; [0017] In the remote interactive medical system based on the virtual reality head-mounted display device provided by the present invention, the cloud computing platform includes a first hardware system and a first software system, and the first hardware system includes a first server and a first storage system, the first software system comprising a first operating system and a first database system;
[0018] 所述云存储平台包括第二硬件系统和第二软件系统, 所述第二硬件系统包括第 二服务器和第二存贮系统, 所述第二软件系统包括第二操作系统和第二数据库 系统。 [0018] the cloud storage platform includes a second hardware system and a second software system, where the second hardware system includes A second server and a second storage system, the second software system comprising a second operating system and a second database system.
[0019] 本发明还提供一种基于虚拟现实头戴式显示装置的远程互动医疗方法, 所述方 法包括以下步骤: [0019] The present invention also provides a remote interactive medical method based on a virtual reality head mounted display device, the method comprising the following steps:
[0020] 临床系统的医疗人员利用数据采集设备采集病患者的生理数据, 并通过数据传 输通道发送至云端数据库; [0020] The medical system of the clinical system uses the data collection device to collect the physiological data of the patient and sends it to the cloud database through the data transmission channel;
[0021] 所述云端数据库中的生理数据通过数据交换中枢传输至虚拟现实头戴式显示装 置; [0021] the physiological data in the cloud database is transmitted to the virtual reality head-mounted display device through the data exchange hub;
[0022] 所述虚拟现实头戴式显示装置根据所述生理数据建立并动态实吋展示病患者病 患部位的三维模型; [0022] the virtual reality head-mounted display device establishes and dynamically displays a three-dimensional model of a patient's diseased part according to the physiological data;
[0023] 所述虚拟现实头戴式显示装置建立并动态实吋展示用于治疗所述病患者病患部 位的手术器械的三维模型; [0023] the virtual reality head-mounted display device establishes and dynamically displays a three-dimensional model of a surgical instrument for treating a patient's diseased part of the patient;
[0024] 支持系统的医疗人员操控所述手术器械的三维模型对所述病患者病患部位的三 维模型进行虚拟手术处理, 所述虚拟现实头戴式显示装置记录所述支持系统进 行手术吋的手势操作步骤, 并结合所述病患者病患部位的三维模型和手术器械 的三维模型一起通过数据交换中枢传输至云端数据库, 以供临床系统参考。 [0024] a medical staff supporting the system manipulates a three-dimensional model of the surgical instrument to perform a virtual surgical treatment on a three-dimensional model of the patient's patient site, and the virtual reality head-mounted display device records the support system for performing a surgical procedure The gesture operation step is combined with the three-dimensional model of the patient's patient site and the three-dimensional model of the surgical instrument to be transmitted to the cloud database through the data exchange center for reference by the clinical system.
[0025] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗方法中, 所述方 法包括以下步骤: [0025] In the remote interactive medical method based on the virtual reality head mounted display device provided by the present invention, the method includes the following steps:
[0026] 从虚拟人体模型库中提取与病患部位对应的人体组织器官的三维模型; Extracting a three-dimensional model of a human tissue organ corresponding to a patient site from a virtual human model library;
[0027] 根据所述病患部位的生理数据对所述与病患部位对应的人体组织器官的三维模 型进行三维重建得到所述病患部位的三维模型; [0027] three-dimensional reconstruction of the three-dimensional model of the human tissue organ corresponding to the patient site according to the physiological data of the patient site to obtain a three-dimensional model of the patient site;
[0028] 屏显获取并实吋动态展示病患部位的三维模型。 [0028] The screen display acquires and dynamically displays a three-dimensional model of the patient's site.
[0029] 在本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗方法中, 所述方 法包括以下步骤: 支持系统中虚拟手术的实施者佩戴与所述虚拟现实头戴式显 示装置电性连接的穿戴配件。 [0029] In the remote interactive medical method based on the virtual reality head mounted display device provided by the present invention, the method includes the following steps: the implementer of the virtual surgery in the support system wears the virtual reality head mounted display device Sexually connected wearable accessories.
发明的有益效果 Advantageous effects of the invention
有益效果 Beneficial effect
[0030] 实施本发明提供的基于虚拟现实头戴式显示装置的远程互动医疗系统及方法具 有以下有益效果: [0030] A remote interactive medical system and method based on virtual reality head mounted display device provided by the present invention Has the following beneficial effects:
[0031] 1、 虚拟现实头戴式显示装置为支持系统的医疗人员提供病患者病患部位的实 吋生理数据以及实吋动态三维模型, 且可生成各种手术所需的医疗器械的三维 模型, 由此, 支持系统的医疗人员可根据病患部位的吋实动态三维模型制定对 应的治疗方案, 并利用医疗器械的三维模型进行虚拟手术, 所述虚拟现实头戴 式显示装置记录支持系统制定的治疗方案并将虚拟手术的相关数据传输到云端 数据库, 以供临床系统参考。 [0031] 1. The virtual reality head-mounted display device provides the medical staff of the support system with the actual physiological data of the patient's patient part and the real dynamic three-dimensional model, and can generate a three-dimensional model of the medical device required for various operations. Therefore, the medical staff of the support system can formulate a corresponding treatment plan according to the tamping dynamic three-dimensional model of the patient part, and perform virtual surgery using the three-dimensional model of the medical device, and the virtual reality head-mounted display device records support system is formulated. The treatment plan transmits data related to the virtual surgery to the cloud database for reference by the clinical system.
[0032] 2、 所述数据采集设备可以包括接入多参数监护仪、 心电、 高清窥镜、 电子听 诊器、 摄像头等多种专业级的医疗信息采集设备, 也可根据工作需要, 接入更 多的其他医疗信息采集设备。 实现多种生理指标的实吋检査, 如心电图、 体温 、 脉搏、 血压、 血氧、 心音、 肺音、 眼耳喉鼻图像等, 以便医疗人员获取更多 的综合诊断数据进行分析判断。 上述采集的医疗信息可通过有线网络、 WiFi无 线网络、 卫星网络、 3G/4G无线宽带网络实现专网连接, 保障数据传输的安全性 [0032] 2. The data collection device may include multiple professional-level medical information collection devices such as a multi-parameter monitor, an electrocardiogram, a high-definition speculum, an electronic stethoscope, a camera, etc., and may also be accessed according to work needs. Many other medical information collection devices. Realize the physical examination of various physiological indicators, such as electrocardiogram, body temperature, pulse, blood pressure, blood oxygen, heart sound, lung sound, eye, nose and throat, etc., so that medical personnel can obtain more comprehensive diagnostic data for analysis and judgment. The medical information collected above can be connected to the private network through a wired network, a WiFi wireless network, a satellite network, or a 3G/4G wireless broadband network to ensure the security of data transmission.
[0033] 3、 所述云端数据库实现了多个医疗诊断终端之间的实吋互动或离线互动。 所 述云端数据库实现多个医疗诊断终端的多类型数据的网络存储以及控制交互。 支持系统的医疗人员通过虚拟现实头盔显示装置, 可以实吋了解到病患的生理 数据, 临床系统的医疗人员通过访问所述云端数据库也可获取支持系统提供的 治疗方案, 实现了创新的远程医疗诊断应用模式。 [0033] 3. The cloud database implements real interaction or offline interaction between multiple medical diagnosis terminals. The cloud database implements network storage and control interaction of multiple types of data of multiple medical diagnostic terminals. The medical staff of the support system can realize the physiological data of the patient through the virtual reality helmet display device, and the medical system of the clinical system can also obtain the treatment plan provided by the support system by accessing the cloud database, and realize the innovative telemedicine. Diagnostic application mode.
[0034] 4、 病患者的海量数据和资料存储在云端数据库中, 可方便各级授权机构通过 多种设备进行访问、 调阅, 实现疾病诊断、 医疗质量追踪、 进行各种业务的统 计分析等, 还可以大数据为基础, 实现医疗知识库、 面向人的健康跟踪以及区 域性的疾病监控等。 [0034] 4. The massive data and data of the sick patient are stored in the cloud database, which facilitates access and access by authorized institutions at various levels through various devices, and realizes disease diagnosis, medical quality tracking, and statistical analysis of various services. It can also be based on big data to realize medical knowledge base, human-oriented health tracking and regional disease monitoring.
对附图的简要说明 Brief description of the drawing
附图说明 DRAWINGS
[0035] 图 1为本发明实施例一提供的基于虚拟现实头戴式显示装置的远程互动医疗系 统的结构示意图; 1 is a schematic structural diagram of a remote interactive medical system based on a virtual reality head mounted display device according to Embodiment 1 of the present invention;
[0036] 图 2为本发明实施例一提供的虚拟现实头戴式显示装置的结构示意图; [0037] 图 3为本发明实施例二提供的基于虚拟现实头戴式显示装置的远程互动医疗方 法的步骤流程图; 2 is a schematic structural diagram of a virtual reality head-mounted display device according to Embodiment 1 of the present invention; 3 is a flow chart of steps of a remote interactive medical method based on a virtual reality head mounted display device according to Embodiment 2 of the present invention;
[0038] 图 4为本发明实施例二提供的基于虚拟现实头戴式显示装置的远程互动医疗方 法中步骤 S3的具体流程图。 4 is a specific flowchart of step S3 in the remote interactive medical method based on the virtual reality head mounted display device according to the second embodiment of the present invention.
[0039] 具体实施方式的附图标号说明: DESCRIPTION OF REFERENCE NUMERALS OF THE PREFERRED EMBODIMENTS
[] [表 1] [] [Table 1]
本发明的实施方式 Embodiments of the invention
[0040] 为了解决现有技术中远程医疗互动技术中支持系统难以为外科手术现场提供权 威实吋的手术指导及有效的解决方案的技术问题, 本发明旨在提供一种基于虚 拟现实头戴式显示装置的远程互动系统和方法, 其核心思想是: 利用虚拟现实 头戴式显示装置获取临床医疗结构采集的生理数据, 然后生成相应的三维模型 影像以供支持系统的医疗人员制定并虚拟实吋相应的治疗方案, 并将所述治疗 方案传输至云端数据库以供临床系统参考。 从而解决了现有技术中支持系统难 以为外科手术现场提供权威实吋的手术指导及有效的解决方案的的缺陷。 [0040] In order to solve the technical problem that the support system in the prior art in the telemedicine interaction technology is difficult to provide an authoritative surgical guide and an effective solution for the surgical site, the present invention aims to provide a virtual reality based headset. The remote interaction system and method of the display device has the core idea of: acquiring the physiological data collected by the clinical medical structure by using the virtual reality head-mounted display device, and then generating the corresponding three-dimensional model image for the medical personnel of the support system to formulate and virtualize A corresponding treatment regimen is transmitted to the cloud database for reference by the clinical system. Thus, the drawbacks of the prior art support system that are difficult to provide an authoritative surgical guide and an effective solution for the surgical site are solved.
[0041] 为了对本发明的技术特征、 目的和效果有更加清楚的理解, 现对照附图详细说 明本发明的具体实施方式。 [0041] In order to more clearly understand the technical features, objects and effects of the present invention, the embodiments of the present invention are described in detail with reference to the accompanying drawings.
[0042] 实施例一 [0043] 本发明提供的基于虚拟现实头戴式显示装置 201的远程互动医疗系统, 参见图 1 , 该系统包括临床系统 100和支持系统 200, 临床系统 100和支持系统 200在地域 上相互隔离, 所述临床系统 100包括数据采集设备 101、 数据传输通道 102、 云端 数据库 103、 数据交换中枢 104以及虚拟现实头戴式显示装置 201, 所述支持系统 200包括虚拟现实头戴式显示装置 201和穿戴配件 202; [0042] Embodiment 1 [0043] The remote interactive medical system based on the virtual reality head mounted display device 201 provided by the present invention, see FIG. 1, the system includes a clinical system 100 and a support system 200, and the clinical system 100 and the support system 200 are geographically isolated from each other. The clinical system 100 includes a data collection device 101, a data transmission channel 102, a cloud database 103, a data exchange hub 104, and a virtual reality head mounted display device 201, the support system 200 including a virtual reality head mounted display device 201 and wearable Accessory 202;
[0044] 所述数据采集设备 101, 连接所述数据传输通道 102, 用于采集人体生理数据; [0044] the data collection device 101 is connected to the data transmission channel 102 for collecting human physiological data;
[0045] 所述数据传输通道 102, 包括以太网、 WiFi、 无线专网、 电话 PSTN网络、 卫星 或移动互联网中的至少一种, 用于传输数据; [0045] the data transmission channel 102, including at least one of an Ethernet, a WiFi, a wireless private network, a telephone PSTN network, a satellite, or a mobile Internet, for transmitting data;
[0046] 所述云端数据库 103, 连接所述数据传输通道 102, 包括云计算平台和云存储平 台, 用于提供统一的数据访问接口及安排相应的数据访问权限; [0046] The cloud database 103 is connected to the data transmission channel 102, and includes a cloud computing platform and a cloud storage platform, for providing a unified data access interface and arranging corresponding data access rights;
[0047] 所述数据交换中枢 104, 连接所述数据传输通道 102, 用于完成所述临床系统 10 0与所述支持系统 200之间的数据的交换; [0047] the data exchange hub 104 is connected to the data transmission channel 102 for completing data exchange between the clinical system 100 and the support system 200;
[0048] 所述虚拟现实头戴式显示装置 201, 网络连接所述数据交换中枢 104, 用于根据 所述生理数据生成相应的影像。 [0048] The virtual reality head mounted display device 201 is connected to the data exchange hub 104 for generating a corresponding image according to the physiological data.
[0049] 所述穿戴配件 202, 与所述虚拟现实头戴式显示装置 201网络连接。 [0049] The wearing accessory 202 is connected to the virtual reality head mounted display device 201 in a network.
[0050] 具体地, 所述数据采集设备 101包括便携的检测血压、 血氧、 脉搏、 体温、 心 电的多参生命体征监护仪, 十二导联的网络心电图机, 检测耳鼻喉的光学窥镜 , 检测皮肤的光学窥镜、 专业心肺音高保真电子听诊器以及网络摄像头等, 用 于采集病患者的各项生命指标及生理特征。 所述数据采集设备 101未来还可以包 括其他检测设备。 [0050] Specifically, the data collection device 101 includes a portable multi-parameter vital sign monitor for detecting blood pressure, blood oxygen, pulse, body temperature, and electrocardiogram, a 12-lead network electrocardiograph, and an optical glimpse of the ear, nose and throat. Mirror, optical goggles for detecting skin, professional cardiopulmonary high-fidelity electronic stethoscope, and webcam are used to collect various life indicators and physiological characteristics of patients. The data collection device 101 may also include other detection devices in the future.
[0051] 具体地, 所述数据传输通道 102为整个临床系统 100提供了数据在采集、 存储、 处理、 发送环节之间的传输, 所述数据传输通道 102通过 VPN进行传输链路加密 。 所述临床系统 100支持有线、 无线、 卫星等多种网络传输手段, 可将生理数据 通过网络上传到云端数据库 103。 [0051] Specifically, the data transmission channel 102 provides the entire clinical system 100 with data transmission between the collection, storage, processing, and transmission links, and the data transmission channel 102 performs transmission link encryption through the VPN. The clinical system 100 supports a variety of network transmission means such as wired, wireless, and satellite, and the physiological data can be uploaded to the cloud database 103 through the network.
[0052] 具体地, 所述云端数据库 103的云计算平台包括第一硬件系统和第一软件系统 , 所述第一硬件系统包括第一服务器和第一存贮系统, 所述第一软件系统包括 第一操作系统和第一数据库系统; 云端数据库 103的云存储平台包括第二硬件系 统和第二软件系统, 所述第二硬件系统包括第二服务器和第二存贮系统, 所述 第二软件系统包括第二操作系统和第二数据库系统。 该部分采用云计算技术, 实现云计算平台和云存储平台, 实现安全的、 可伸缩的计算环境。 云端数据库 1 03可部署在医院的机房内, 也可部署到第三方提供的专属私有云上。 支持系统 2 00的医疗人员和临床系统 100的医疗人员可通过云端数据库 103实现医疗信息的 交互, 如果在断网的情况下, 支持系统 200和临床系统 100也可以单独操作, 在 网络恢复连接的吋候, 再将数据上传到云端数据库 103。 [0052] Specifically, the cloud computing platform of the cloud database 103 includes a first hardware system and a first software system, the first hardware system includes a first server and a first storage system, and the first software system includes a first operating system and a first database system; a cloud storage platform of the cloud database 103 includes a second hardware system and a second software system, the second hardware system including a second server and a second storage system, The second software system includes a second operating system and a second database system. This part adopts cloud computing technology to realize cloud computing platform and cloud storage platform to realize a secure and scalable computing environment. Cloud Database 03 can be deployed in the hospital's computer room or deployed on a proprietary private cloud provided by a third party. The medical staff supporting the system 200 and the medical staff of the clinical system 100 can realize the interaction of the medical information through the cloud database 103. If the network is disconnected, the support system 200 and the clinical system 100 can also operate separately, and the connection is restored in the network. At this time, the data is uploaded to the cloud database 103.
[0053] 具体地, 参见图 2, 所述虚拟现实头戴式显示装置 201具有人体仿真处理器 221 、 工具仿真处理器 231、 屏显 241、 虚拟人体模型库 211以及运动位移感应模块 25 1 = Specifically, referring to FIG. 2, the virtual reality head mounted display device 201 has a human body simulation processor 221, a tool simulation processor 231, a screen display 241, a virtual human body model library 211, and a motion displacement sensing module 25 1 =
[0054] 人体仿真处理器 221用于生成病患部位的三维模型。 在虚拟手术同步环境下, 人体仿真处理器 221首先采集手术前对病患者的病患部位进行 CT扫描得到的数据 , 需要说明的是, 本实施例中人体仿真处理器 221通过 DICOM3.0数据格式进行 数据的采集, 其中, DICOM (digital imaging and communication in medicine) 即 医学数字成像与通讯数据标准格式能促进医学影像设备之间的互操作性, 提供 一种用于医学信息的幵放性的数据交换标准。 针对病患者的病患部位在虚拟人 体模型库 211中取出事先生成的对应部位的三维模型, 将该病患者的真实病患部 位身体部位的 CT扫描数据进行三维重建的 B样曲线, bbox算法, 傅里叶算法, 小 值二分法等算法的处理, 完成生理数据的几何建模、 计算建模、 变形计算。 通 过 3DMAX10对病患者生理特征的三维模型重建、 贴图及赋予器官组织生理特征 , 并进行骨骼皮肤组织碰撞检测和图形绘制, 同吋生成病患者病患部位的三维 模型。 [0054] The human body simulation processor 221 is used to generate a three-dimensional model of the patient's site. In the virtual surgery synchronization environment, the human body simulation processor 221 first collects data obtained by performing CT scan on the patient's patient part before the operation. It should be noted that the human body simulation processor 221 passes the DICOM 3.0 data format in this embodiment. Data collection, DICOM (digital imaging and communication in medicine), which is a standard format for medical digital imaging and communication data, can promote interoperability between medical imaging devices and provide a data for medical information. Exchange standards. The three-dimensional model of the corresponding part generated in advance is taken out in the virtual human model library 211 for the patient's patient part, and the B-scan curve of the CT scan data of the body part of the real patient part of the patient is three-dimensionally reconstructed, bbox algorithm, Fourier algorithm, small value dichotomy and other algorithms are processed to complete the geometric modeling, computational modeling and deformation calculation of physiological data. Through 3DMAX10, the three-dimensional model reconstruction of the physiological characteristics of the patient, the mapping and the physiological characteristics of the organs and tissues, and the collision detection and graphic drawing of the skeletal skin tissue, and the three-dimensional model of the patient's diseased part.
[0055] 工具仿真处理器 231用于生成手术器械的三维模型。 所述工具仿真处理器 231通 过虚拟现实造型语言 (Virtual Reality Modeling Language, VRML) 编写虚拟手 术使用的各种手术器械的三维模型, 例如, 手钻、 常规消毒铺巾、 手术刀、 电 刀、 棉片、 纱布的三维模型, 并作为后续虚拟现实头戴式显示装置 201的显示内 容输出。 其中 VRML是虚拟现实造型语言的缩写形式, 它是描述虚拟场景的一种 标准, 定义了三维应用系统中常用的语言描述, 如层次变换、 光源、 试点、 几 何、 动画、 雾、 材料特性和纹理映射等, 并具有简单的行为特征描述功能。 设 计 VRML的一个主要目标就是保证它成为多个虚拟现实系统或其组成部分间有效 的三维文件交换格式。 [0055] The tool simulation processor 231 is used to generate a three-dimensional model of the surgical instrument. The tool emulation processor 231 writes a three-dimensional model of various surgical instruments used in virtual surgery through a Virtual Reality Modeling Language (VRML), for example, a hand drill, a conventional disinfecting drape, a scalpel, an electric knife, and cotton. A three-dimensional model of the sheet and gauze is output as the display content of the subsequent virtual reality head mounted display device 201. VRML is an abbreviation of virtual reality modeling language. It is a standard for describing virtual scenes. It defines the language descriptions commonly used in 3D applications, such as hierarchical transformation, light source, pilot, geometry, animation, fog, material properties and texture. Mapping, etc., and has a simple behavioral characterization function. Assume One of the main goals of VRML is to ensure that it becomes an efficient three-dimensional file exchange format between multiple virtual reality systems or their components.
[0056] 屏显 241可动态实吋展示所述病患部位的三维模型和手术器械的三维模型。 屏 显 241具有一个图形用户界面用于选择性地动态实吋展示所述生理数据, 例如, 对于实吋的动态心电图数据而言, 虚拟现实头戴式显示装置 201显示区域中的部 分局部区域预先设定了的用于展示病患者动态心电图的图形用户界面的展现区 域。 上述对实吋动态心电图数据的举例仅是对诸多体现人体生理动态数据的举 例说明, 并非是对本发明的限制, 鉴于综合医疗诊断生理数据是实施远程互动 的基础, 未来还有更多的新的生理数据需要得到图形用户界面的支持, 可通过 虚拟现实头戴式显示装置 201中设定的图形用户界面 (GUI) 给予实吋动态展示 [0056] The on-screen display 241 can dynamically display a three-dimensional model of the patient site and a three-dimensional model of the surgical instrument. The screen display 241 has a graphical user interface for selectively and dynamically displaying the physiological data. For example, for real dynamic electrocardiogram data, the virtual reality head mounted display device 201 displays a partial partial area in the area in advance. A display area for displaying a graphical user interface for displaying a patient's dynamic electrocardiogram. The above examples of real dynamic electrocardiogram data are only examples of many human physiological dynamic data, and are not limitations of the present invention. Since comprehensive medical diagnostic physiological data is the basis for implementing remote interaction, there are more new ones in the future. The physiological data needs to be supported by the graphical user interface, and can be dynamically displayed through a graphical user interface (GUI) set in the virtual reality head mounted display device 201.
[0057] 虚拟人体模型库 211用于存储事先生成的人体组织器官的三维模型。 [0057] The virtual human body model library 211 is used to store a three-dimensional model of a human tissue organ that is generated in advance.
[0058] 运动位移感应模块 251用于感应并记录所述穿戴配件 202的运动轨迹信息数据。 [0058] The motion displacement sensing module 251 is configured to sense and record motion track information data of the wear accessory 202.
[0059] 具体地, 所述穿戴配件 202用于佩戴在所述支持系统 200的实施虚拟手术的专家 身上, 所述穿戴配件 202与所述运动位移感应模块 251电性连接。 [0059] Specifically, the wearing accessory 202 is used to be worn by an expert who performs virtual surgery on the support system 200, and the wearing accessory 202 is electrically connected to the motion displacement sensing module 251.
[0060] 实施例二 [0060] Embodiment 2
[0061] 本发明实施提供了一种基于虚拟现实头戴式显示装置 201的远程互动医疗方法 , 适用于实施例一所示的基于虚拟现实头戴式显示装置 201的远程互动医疗系统 , 参见图 3, 该方法包括: The present invention provides a remote interactive medical system based on the virtual reality head mounted display device 201, which is applicable to the remote interactive medical system based on the virtual reality head mounted display device 201 shown in the first embodiment, see FIG. 3. The method includes:
[0062] Sl、 采集并传输生理数据。 [0062] Sl, collecting and transmitting physiological data.
[0063] 在本实施例中, 临床系统 100的医疗人员利用数据采集设备 101采集病患者的生 理数据, 并通过数据传输通道 102发送至云端数据库 103。 临床系统 100的医疗人 员通过检测血压、 血氧、 脉搏、 体温、 心电的多参生命体征监护仪, 十二导联 的网络心电图机, 检测耳鼻喉的光学窥镜, 检测皮肤的光学窥镜, 专业心肺音 高保真电子听诊器, CT扫描以及网络摄像头获取病患者的各项生理数据。 In the present embodiment, the medical staff of the clinical system 100 uses the data collection device 101 to collect the physiological data of the patient and send it to the cloud database 103 through the data transmission channel 102. The medical system 100 medical staff detects the blood ray, blood oxygen, pulse, body temperature, electrocardiogram multi-parameter vital signs monitor, 12-lead network electrocardiograph, detects the optical otoscope of the ear, nose and throat, and detects the optical speculum of the skin. , professional cardio-pulmonary high-fidelity electronic stethoscope, CT scan and webcam to obtain various physiological data of patients.
[0064] S2、 虚拟现实头戴式显示装置 201接收生理数据。 [0064] S2. The virtual reality head mounted display device 201 receives the physiological data.
[0065] 在本实施例中, 所述云端数据库 103中的生理数据通过数据交换中枢 104传输至 虚拟现实头戴式显示装置 201。 所述虚拟现实头戴式显示装置 201获取病患部位 的生理数据。 在虚拟手术同步环境下, 所述虚拟现实头戴式显示装置 201采集手 术前临床系统 100的医疗人员对病患者的病患部位进行真实 CT扫描得到的数据 ( 通过 DICOM3.0数据格式的采集) 。 [0065] In the embodiment, the physiological data in the cloud database 103 is transmitted to the virtual reality head mounted display device 201 through the data exchange hub 104. The virtual reality head mounted display device 201 acquires a patient's part Physiological data. In the virtual surgery synchronization environment, the virtual reality head-mounted display device 201 collects data obtained by a medical staff of the pre-operative clinical system 100 on a real CT scan of a patient's patient part (by DICOM 3.0 data format acquisition) .
[0066] S3、 建立并动态实吋展示病患者病患部位的三维模型。 [0066] S3. Establish and dynamically display a three-dimensional model of the patient's diseased part.
[0067] 在本实施例中, 所述虚拟现实头戴式显示装置 201根据所述生理数据建立并动 态实吋展示病患者病患部位的三维模型, 参见图 4, 具体包括以下步骤: In the embodiment, the virtual reality head-mounted display device 201 establishes and dynamically displays a three-dimensional model of the patient's patient's part according to the physiological data. Referring to FIG. 4, the following steps are specifically included:
[0068] SA、 从虚拟人体模型库 211中提取与病患部位对应的人体组织器官的三维模型 。 在虚拟人体模型库 211中取出事先生成的对应于病患者的病患部位的人体组织 器官的三维模型。 [0068] The SA extracts a three-dimensional model of the human tissue and organs corresponding to the patient's part from the virtual human model library 211. A three-dimensional model of a human tissue organ corresponding to a patient's patient's disease site generated in advance is taken out in the virtual human model library 211.
[0069] SB、 跟据所述病患部位的生理数据对所述与病患部位对应的人体组织器官的三 维模型进行三维重建得到所述病患部位的三维模型。 将该病患者的真实病患部 位身体部位的 CT扫描数据进行三维重建的 B样曲线, bbox算法, 傅里叶算法, 小 值二分法等算法的处理, 完成生理数据的几何建模、 计算建模、 变形计算。 通 过 3DMAX10对病患者的病患部位的生理特征的三维模型重建、 贴图及赋予器官 组织生理特征, 并进行骨骼皮肤组织碰撞检测和图形绘制, 生成病患者病患部 位的三维模型。 [0069] SB, performing a three-dimensional reconstruction of the three-dimensional model of the human tissue organ corresponding to the patient site according to the physiological data of the patient site to obtain a three-dimensional model of the patient site. The B-scan curve of the three-dimensional reconstruction of the CT scan data of the body part of the patient's real patient part, the bbox algorithm, the Fourier algorithm, the small value dichotomy and other algorithms are processed to complete the geometric modeling and calculation of the physiological data. Modulus, deformation calculation. Through 3DMAX10, the 3D model of the physiological features of the patient's diseased part is reconstructed, mapped, and the organ and tissue physiological characteristics are given, and the bone skin tissue collision detection and graphic drawing are performed to generate a three-dimensional model of the patient's diseased part.
[0070] SC、 屏显 241获取并实吋动态展示病患部位的三维模型。 [0070] The SC and the screen display 241 acquire and implement a three-dimensional model for dynamically displaying the patient's part.
[0071] S4、 建立并动态实吋展示手术器械的三维模型。 [0071] S4. Establish and dynamically display a three-dimensional model of the surgical instrument.
[0072] 在本实施例中, 所述虚拟现实头戴式显示装置 201建立并动态实吋展示用于治 疗所述病患者病患部位的手术器械的三维模型。 所述工具仿真处理器 231通过虚 拟现实造型语言 (Virtual Reality Modeling Language, VRML) 编写虚拟手术使 用的各种手术器械的三维模型, 例如, 手钻、 常规消毒铺巾、 手术刀、 电刀、 棉片、 纱布的三维模型, 并作为虚拟现实头戴式显示装置 201的屏显 241的显示 内容输出。 其中 VRML是虚拟现实造型语言的缩写形式, 它是描述虚拟场景的一 种标准, 定义了三维应用系统中常用的语言描述, 如层次变换、 光源、 试点、 几何、 动画、 雾、 材料特性和纹理映射等, 并具有简单的行为特征描述功能。 设计 VRML的一个主要目标就是保证它成为多个虚拟现实系统或其组成部分间有 效的三维文件交换格式。 [0073] S5、 手术方案的形成及传输。 [0072] In the present embodiment, the virtual reality head-mounted display device 201 establishes and dynamically displays a three-dimensional model of a surgical instrument for treating a patient's patient's condition. The tool emulation processor 231 writes a three-dimensional model of various surgical instruments used in virtual surgery through a Virtual Reality Modeling Language (VRML), for example, a hand drill, a conventional disinfecting drape, a scalpel, an electric knife, and cotton. The three-dimensional model of the sheet and the gauze is output as the display content of the screen display 241 of the virtual reality head mounted display device 201. VRML is an abbreviation of virtual reality modeling language. It is a standard for describing virtual scenes. It defines the language descriptions commonly used in 3D applications, such as hierarchical transformation, light source, pilot, geometry, animation, fog, material properties and texture. Mapping, etc., and has a simple behavioral characterization function. One of the main goals of designing VRML is to ensure that it becomes an effective three-dimensional file exchange format between multiple virtual reality systems or their components. [0073] S5. Formation and transmission of a surgical plan.
[0074] 支持系统 200的虚拟手术实施者操控所述手术器械的三维模型对所述病患者病 患部位的三维模型进行虚拟手术, 所述虚拟现实头戴式显示装置 201记录所述支 持系统 200进行手术吋的手势操作步骤, 并结合所述病患者病患部位的三维模型 和手术器械的三维模型形成完整的手术方案一起通过数据交换中枢 104传输至云 端数据库 103, 以供临床系统 100参考。 [0044] The virtual surgery implementer of the support system 200 manipulates the three-dimensional model of the surgical instrument to perform virtual surgery on the three-dimensional model of the patient's patient site, and the virtual reality head mounted display device 201 records the support system 200 The gesture operation step of the surgical procedure is performed, and the three-dimensional model of the patient's patient site and the three-dimensional model of the surgical instrument are combined to form a complete surgical plan and transmitted to the cloud database 103 through the data exchange center 104 for reference by the clinical system 100.
[0075] 具体地, 所述基于虚拟现实头戴式显示装置 201的远程互动医疗方法还包括以 下步骤: [0075] Specifically, the remote interactive medical method based on the virtual reality head mounted display device 201 further includes the following steps:
[0076] SD、 佩戴穿戴配件 202。 [0076] SD, wearing the wearable accessory 202.
[0077] 本实施例中, 支持系统 200中虚拟手术的实施者佩戴与所述虚拟现实头戴式显 示装置 201网络连接的穿戴配件 202。 虚拟手术的实施者佩戴的穿戴配件 202包括 十个手指的指套, 所述位移感应模块通过与所述指套网络感应连接可感知并记 录所述指套的运动轨迹, 以此记录虚拟手术的实施者进行虚拟手术吋的手势动 作。 In this embodiment, the implementer of the virtual surgery in the support system 200 wears the wear accessory 202 that is networked with the virtual reality head mounted display device 201. The wear accessory 202 worn by the implementer of the virtual surgery includes a finger sleeve of ten fingers, and the displacement sensing module can sense and record the movement track of the finger sleeve by inductively connecting with the finger socket network, thereby recording the virtual surgery The implementer performs a gesture of virtual surgery.
[0078] SE、 临床系统 100的医疗人员获取手术方案。 [0078] The medical staff of the SE, clinical system 100 acquires a surgical plan.
[0079] 本实施例中, 临床系统 100的医疗人员可通过访问所述云端数据库 103获取支持 系统 200提供的手术方案。 [0079] In this embodiment, the medical staff of the clinical system 100 can obtain the surgical plan provided by the support system 200 by accessing the cloud database 103.
[0080] 需要说明的是, 上述步骤 SD在步骤 S5之前和步骤 S4之后完成, 步骤 SE在步骤 S[0080] It should be noted that the above step SD is completed before step S5 and after step S4, and step SE is in step S.
5之后完成。 After 5 is completed.
[0081] 实施本发明提供的基于虚拟现实头戴式显示装置 201的远程互动医疗系统及方 法具有以下有益效果: [0081] The remote interactive medical system and method based on the virtual reality head mounted display device 201 provided by the present invention has the following beneficial effects:
[0082] 1、 虚拟现实头戴式显示装置 201为支持系统 200的医疗人员提供病患者病患部 位的实吋生理数据以及实吋动态三维模型, 且可生成各种手术所需的医疗器械 的三维模型, 由此, 支持系统 200的医疗人员可根据病患部位的吋实动态三维模 型制定对应的治疗方案, 并利用医疗器械的三维模型进行虚拟手术, 所述虚拟 现实头戴式显示装置 201记录支持系统 200制定的治疗方案并将虚拟手术的相关 数据传输到云端数据库 103, 以供临床系统 100参考。 [0082] 1. The virtual reality head-mounted display device 201 provides the medical staff supporting the system 200 with the actual physiological data of the patient's diseased part and the real dynamic three-dimensional model, and can generate various medical instruments required for the operation. The three-dimensional model, whereby the medical staff of the support system 200 can formulate a corresponding treatment plan according to the tamping dynamic three-dimensional model of the patient site, and perform virtual surgery using the three-dimensional model of the medical device, the virtual reality head-mounted display device 201 The treatment plan developed by the support system 200 is recorded and the data related to the virtual surgery is transmitted to the cloud database 103 for reference by the clinical system 100.
[0083] 2、 所述数据采集设备 101可以包括接入多参数监护仪、 心电、 高清窥镜、 电子 听诊器、 摄像头等多种专业级的医疗信息采集设备, 也可根据工作需要, 接入 更多的其他医疗信息采集设备。 实现多种生理指标的实吋检査, 如心电图、 体 温、 脉搏、 血压、 血氧、 心音、 肺音、 眼耳喉鼻图像等, 以便医疗人员获取更 多的综合诊断数据进行分析判断。 上述采集的医疗信息可通过有线网络、 WiFi 无线网络、 卫星网络、 3G/4G无线宽带网络实现专网连接, 保障数据传输的安全 性。 [0083] 2. The data collection device 101 may include an access multi-parameter monitor, an electrocardiogram, a high-definition speculum, and an electronic A variety of professional-grade medical information collection devices such as stethoscopes and cameras can also access more medical information collection devices according to work needs. Realize the physical examination of various physiological indicators, such as electrocardiogram, body temperature, pulse, blood pressure, blood oxygen, heart sound, lung sound, eye, nose and throat, etc., so that medical personnel can obtain more comprehensive diagnostic data for analysis and judgment. The medical information collected above can be connected to the private network through a wired network, a WiFi wireless network, a satellite network, or a 3G/4G wireless broadband network to ensure the security of data transmission.
[0084] 3、 所述云端数据库 103实现了多个医疗诊断终端之间的实吋互动或离线互动。 [0084] 3. The cloud database 103 implements real interaction or offline interaction between multiple medical diagnosis terminals.
所述云端数据库 103实现多个医疗诊断终端的多类型数据的网络存储以及控制交 互。 支持系统 200的医疗人员通过虚拟现实头盔显示装置, 可以实吋了解到病患 的生理数据, 临床系统 100的医疗人员通过访问所述云端数据库 103也可获取支 持系统 200提供的治疗方案, 实现了创新的远程医疗诊断应用模式。 The cloud database 103 enables network storage and control interaction of multiple types of data for a plurality of medical diagnostic terminals. The medical personnel of the support system 200 can realize the physiological data of the patient through the virtual reality helmet display device, and the medical staff of the clinical system 100 can also obtain the treatment plan provided by the support system 200 by accessing the cloud database 103, thereby realizing Innovative telemedicine diagnostic application model.
[0085] 4、 病患者的海量数据和资料存储在云端数据库 103中, 可方便各级授权机构通 过多种设备进行访问、 调阅, 实现疾病诊断、 医疗质量追踪、 进行各种业务的 统计分析等, 还可以大数据为基础, 实现医疗知识库、 面向人的健康跟踪以及 区域性的疾病监控等。 [0085] 4. The massive data and data of the patient are stored in the cloud database 103, which facilitates access and access by various authorized organizations at various levels, and realizes disease diagnosis, medical quality tracking, and statistical analysis of various services. Etc., it can also be based on big data, to achieve medical knowledge base, human-oriented health tracking and regional disease monitoring.
[0086] 上面结合附图对本发明的实施例进行了描述, 但是本发明并不局限于上述的具 体实施方式, 上述的具体实施方式仅仅是示意性的, 而不是限制性的, 本领域 的普通技术人员在本发明的启示下, 在不脱离本发明宗旨和权利要求所保护的 范围情况下, 还可做出很多形式, 这些均属于本发明的保护范围之内。 The embodiments of the present invention have been described above with reference to the drawings, but the present invention is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative and not restrictive. A person skilled in the art can make various forms within the scope of the present invention without departing from the scope of the invention and the scope of the invention.
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610664575.2A CN107714006A (en) | 2016-08-12 | 2016-08-12 | A kind of remote interaction system and method based on virtual reality head-mounted display apparatus |
| CN201610664575.2 | 2016-08-12 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018028166A1 true WO2018028166A1 (en) | 2018-02-15 |
Family
ID=61161655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2017/073978 Ceased WO2018028166A1 (en) | 2016-08-12 | 2017-02-17 | Remote interaction system and method based on virtual reality head-mounted display device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107714006A (en) |
| WO (1) | WO2018028166A1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108389488B (en) * | 2018-03-05 | 2020-12-15 | 泉州医学高等专科学校 | Interactive oral cavity simulation system |
| CN108563199A (en) * | 2018-03-27 | 2018-09-21 | 北京科技大学 | A kind of hot continuous rolling production real-time monitoring system |
| CN110459075B (en) * | 2019-07-22 | 2024-12-27 | 温州市中心医院 | A neurosurgery virtual reality simulation surgery system |
| CN111834021A (en) * | 2020-07-20 | 2020-10-27 | 北京百度网讯科技有限公司 | Data interaction method, apparatus, device and storage medium |
| CN112691357B (en) * | 2020-12-29 | 2021-09-24 | 深圳市艾利特医疗科技有限公司 | Cardiopulmonary function exercise system and method |
| CN113705836A (en) * | 2021-08-31 | 2021-11-26 | 国网山东省电力公司微山县供电公司 | System and device for power grid field operation guidance |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101019765A (en) * | 2007-03-29 | 2007-08-22 | 新奥博为技术有限公司 | Surgical operation system under the guide of magnetic resonant image and the operation navigating method |
| CN101904770A (en) * | 2009-06-05 | 2010-12-08 | 复旦大学 | A surgical navigation system and method based on optical augmented reality technology |
| CN102868755A (en) * | 2012-09-27 | 2013-01-09 | 中国科学院深圳先进技术研究院 | Remote medical communication system and working method of remote medical communication system |
| CN103077633A (en) * | 2013-01-11 | 2013-05-01 | 深圳超多维光电子有限公司 | Three-dimensional virtual training system and method |
| CN104706422A (en) * | 2013-12-13 | 2015-06-17 | Ge医疗系统环球技术有限公司 | Head-worn type medical device, medical system and operation method of medical system |
| WO2015126466A1 (en) * | 2014-02-21 | 2015-08-27 | The University Of Akron | Imaging and display system for guiding medical interventions |
| US20150264339A1 (en) * | 2014-03-17 | 2015-09-17 | Nicholas V. Riedel | Stereoscopic display |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101393654A (en) * | 2008-10-07 | 2009-03-25 | 华南师范大学 | A Computer Aided Organ Surgery Simulation System |
| CN104771892A (en) * | 2015-04-02 | 2015-07-15 | 深圳市虚拟现实科技有限公司 | Synchronous display method and system based on virtual reality |
| CN105184720A (en) * | 2015-09-01 | 2015-12-23 | 北京昌盛医学技术有限公司 | Integrated remote medical diagnosis system based on Internet of things |
| CN105615831A (en) * | 2015-12-18 | 2016-06-01 | 李占和 | Medical image holographic AR (augmented reality) display system |
| CN105787941B (en) * | 2016-03-01 | 2020-01-03 | 天津天堰科技股份有限公司 | Rapid registration method of mixed reality simulation system |
-
2016
- 2016-08-12 CN CN201610664575.2A patent/CN107714006A/en active Pending
-
2017
- 2017-02-17 WO PCT/CN2017/073978 patent/WO2018028166A1/en not_active Ceased
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101019765A (en) * | 2007-03-29 | 2007-08-22 | 新奥博为技术有限公司 | Surgical operation system under the guide of magnetic resonant image and the operation navigating method |
| CN101904770A (en) * | 2009-06-05 | 2010-12-08 | 复旦大学 | A surgical navigation system and method based on optical augmented reality technology |
| CN102868755A (en) * | 2012-09-27 | 2013-01-09 | 中国科学院深圳先进技术研究院 | Remote medical communication system and working method of remote medical communication system |
| CN103077633A (en) * | 2013-01-11 | 2013-05-01 | 深圳超多维光电子有限公司 | Three-dimensional virtual training system and method |
| CN104706422A (en) * | 2013-12-13 | 2015-06-17 | Ge医疗系统环球技术有限公司 | Head-worn type medical device, medical system and operation method of medical system |
| WO2015126466A1 (en) * | 2014-02-21 | 2015-08-27 | The University Of Akron | Imaging and display system for guiding medical interventions |
| US20150264339A1 (en) * | 2014-03-17 | 2015-09-17 | Nicholas V. Riedel | Stereoscopic display |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107714006A (en) | 2018-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102458587B1 (en) | Universal device and method to integrate diagnostic testing into treatment in real-time | |
| Wu et al. | Pilot study of robot-assisted teleultrasound based on 5G network: A new feasible strategy for early imaging assessment during COVID-19 pandemic | |
| WO2018028166A1 (en) | Remote interaction system and method based on virtual reality head-mounted display device | |
| US11776123B2 (en) | Systems and methods for displaying augmented anatomical features | |
| JP2022540898A (en) | Augmented Reality Systems and Methods for Remote Supervision of Surgical Procedures | |
| JP5309187B2 (en) | MEDICAL INFORMATION DISPLAY DEVICE, ITS OPERATION METHOD, AND MEDICAL INFORMATION DISPLAY PROGRAM | |
| CN108778100A (en) | Use the medical record system of patient's incarnation | |
| CN110547869B (en) | Preoperative auxiliary planning device based on virtual reality | |
| Yang et al. | Artificial intelligence in medical metaverse: applications, challenges, and future prospects | |
| Oyama | Virtual reality for the palliative care of cancer | |
| US20240237953A1 (en) | Methods for collecting and presenting physiological signal data and location information, and servers and systems implementing the same | |
| JP7685262B2 (en) | SYSTEM AND METHOD FOR DISPLAYING ENHANCED ANATOMICAL FEATURES - Patent application | |
| CN112447297A (en) | 3D prints operation preview operating system who combines VR virtual reality formation of image | |
| US10854005B2 (en) | Visualization of ultrasound images in physical space | |
| US20240290449A1 (en) | Report generation for radiology | |
| US11894126B1 (en) | Systems and methods for tracking movement of a wearable device for advanced image stabilization | |
| CN111613280A (en) | H.I.P.S multi-point touch propaganda and education interaction system for medical treatment | |
| Lin et al. | Future Trends in Biomedical and Health Informatics and Cybersecurity in Medical Devices: Proceedings of the International Conference on Biomedical and Health Informatics, ICBHI 2019, 17-20 April 2019, Taipei, Taiwan | |
| CN113164067B (en) | System for visualizing patient stress | |
| US12496009B2 (en) | System for visualizing patient stress | |
| TWM660651U (en) | Virtual reality platform of digital twins integrated internet of things in telemedicine | |
| JP2024533043A (en) | How to provide electrocardiogram interpretation services | |
| TW202512219A (en) | Virtual reality platform of digital twins integrated internet of things in telemedicine | |
| CN118615021A (en) | Pelvic surgery autonomic nerve detection navigation system | |
| GB2611556A (en) | Augmented reality ultrasound-control system for enabling remote direction of a user of ultrasound equipment by experienced practitioner |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17838303 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 17838303 Country of ref document: EP Kind code of ref document: A1 |