WO2024154647A1 - Learning method and learning system - Google Patents

Abstract

Provided are a learning method and a learning system by which, while viewing a virtual extension of a physical anatomical model projected onto a human body model via a worn mixed-reality display, a learner can learn a procedure or a measure for a skill to be performed, using tools being learned, for a real human body model.　This learning system 1 comprises: a human body model 2 that is to be projected; a mixed-reality display 3 that is capable of visualizing a first virtual extension 31, which is a physical anatomical model, in a superimposed manner over all or part of the human body model 2, the mixed-reality display 3 being worn on the head of a learner H; and a tool 4 being learned, which is a medical tool or an inspection tool with which a skill is to be learned.

Description

Learning method and learning system

The present invention relates to a learning method and a learning system. More specifically, the present invention relates to a learning method and a learning system that enable a learner to learn the procedure or steps of a procedure to be performed by using a learning tool on a real human body model while viewing a virtual extension of a physical anatomical model projected onto the human body model via a worn mixed reality display.

Traditionally, in medical learning settings such as medical faculties at universities, nursing high schools and vocational schools, human body models known as medical simulators have been used to study and train about human anatomy and procedures, such as those described in Non-Patent Document 1 below.

The human body mannequin described in Non-Patent Document 1 is a model of part of the human body (lower jaw to chest to right shoulder) tailored to the purpose of learning (instruction and practice of medical techniques such as puncture and intravenous catheter management), and the bone structure and blood vessels of the modeled parts are accurately reproduced. It is said to enable practical training from selecting the puncture position to inserting the catheter.

On the other hand, in recent years, learning methods have been proposed that use simulated experience systems that utilize VR (Virtual Reality) rather than actual human body models, such as the one described in Patent Document 1.

The simulated experience system described in Patent Document 1 includes a video display device, a main body formed in the same or nearly the same shape as an object of education, research, or training, a controller provided on the main body and having a signal transmitter capable of transmitting a signal for synchronizing the movement of an image of a controlled object simulating the object displayed on the video display device with the operation of the main body, a signal receiver connected to the controller and the video display device and capable of receiving a signal transmitted from the signal transmitter, a calculation unit connected to the signal receiver and capable of analyzing the operation of the main body based on the received signal and calculating operation data, an image generation unit capable of generating an image of the controlled object based on data on the shape of the object, a synchronization processing unit capable of synchronizing the operation data calculated by the calculation unit so that the image of the controlled object generated by the image generation unit moves in accordance with the operation of the main body, and a computer having an image output unit capable of outputting the image of the controlled object processed by the synchronization processing unit to the video display device.

The simulated experience system described in Patent Document 1 allows the user to touch a main body that has the same or nearly the same shape as an educational object, and to operate an image to be operated that imitates the object displayed on an image display device, stimulating the user's sense of sight and touch, providing an intuitive and immersive simulated experience.

Avis Co., Ltd. Human body models, medical models, medical simulators, human dummy sales Human Body Medical and nursing simulator KY11347-300 CVC puncture insertion simulator 2 Internet <URL: http://humanbody.jp/simulator/item/ky11347-300.html>

JP 2020-038272 A

The human body dummies described in Non-Patent Document 1, which are existing technology, are real objects that provide a sense of realism and texture during learning, making them ideal for confirming procedures when using the target tool. However, these human body dummies are often models of parts of the human body tailored to the learning purpose, and in addition, the structure of the model usually does not include surrounding human body structures that are not intended for learning, making them impossible or unsuitable for use outside of learning purposes. Also, while full-body human body dummies do exist, they are expensive compared to models of parts of the human body, and there is a difference between parts that are sophisticated and parts that are not, making them difficult to say that they are suitable for general-purpose use.

On the other hand, the simulated experience system described in Patent Document 1 allows learning by visually recognizing objects such as organs displayed on a video display device and touching a controller that imitates the objects, but the objects are still virtual reality with no substance. In other words, since there is no actual object such as a human body model to which the target tools are applied, the simulated experience system is not suitable for practical training such as confirming procedures and procedures that use the target tools.

The present invention has been devised in light of the above, and aims to provide a learning method and learning system that enables a learner to learn the procedure or steps of a procedure to be performed by using the tools to be studied on a real human body model while viewing a virtual extension of a physical anatomical model projected onto the human body model via a worn mixed reality display.

In order to achieve the above object, the learning method of the present invention is carried out using a human body model as a projection target, a mixed reality display capable of visualizing a first virtual augmentation, which is a physical anatomical model, superimposed on a part or all of the human body model, and a learning target instrument, which is a medical or examination instrument, for learning a procedure, and includes a first step in which a learner wears the mixed reality display and visually views the human body model and the first virtual augmentation projected thereon through the mixed reality display, and a second step in which the learner, having completed the first step, picks up the learning target instrument and applies it to the human body model while visualizing the human body model and the first virtual augmentation, thereby learning the procedure or steps of the procedure to be performed.

Here, in the first step, the learner completes the preparation by wearing the mixed reality display, and the learner can view the human body model and the first virtual augmentation projected onto it through the mixed reality display.

The term "learner" is used to include not only pre-employment students in medical schools, nursing schools, vocational schools, etc. who aim to become medical professionals, but also those who are already medical professionals. For example, new graduates from medical schools, etc. may become familiar with different types of equipment than those used in school, or may undergo practical training beyond what they received in school. Also, even those who have already graduated from medical schools, etc. may undergo training to become familiar with newly introduced equipment, new treatment methods, or as part of continuous learning to further improve their skills. Therefore, these individuals are also included in the term "learner." The learning method of the present invention is essentially used by learners, but this does not exclude instructors (those who practice the present invention as models in explaining the method before and after learning).

Examples of the "first virtual augmentation" include images of human organs and bones. Projecting images of organs, etc. onto appropriate locations on the human body model increases the sense of immersion in learning and also allows for advance confirmation of the positions of organs, etc. Furthermore, the "first virtual augmentation" includes not only single images of human organs, etc., but also projection of multiple images in an overlapping manner (superimposed manner). One example of projection in an overlapping manner is a manner in which an image of a bone and an image of an organ located below the same bone image are projected in an overlapping manner. In this case, more practical learning is possible by referring to the arrangement of the multiple projected images.

The image data projected in the "first virtual augmentation" may be installed in the mixed reality display or stored in an auxiliary storage device connected to the aircraft, or may be received from an external device such as a server connected to the mixed reality display by wire or wireless means. Furthermore, image processing related to the "first virtual augmentation" may be performed by a function provided in the mixed reality display, or may receive data processed by an external device such as a server connected to the mixed reality display by wireless means or the like.

The mixed reality display can also project the first virtual augmentation onto disembodied parts of the human model (parts that have no substance, which can also be considered space). For example, if the only part of the solid human model in front of the viewer's eyes is the torso, the first virtual augmentation of the disembodied parts of the human model, such as the head, lower limbs, and arms, can be visualized in three dimensions as if they were added to the human model as if it were a real object. Furthermore, the display of the disembodied parts can be switched on and off as needed. In other words, the mixed reality display allows a variety of learning to be done even if the viewer does not own multiple human models, and switching between displaying and hiding the disembodied parts is expected to improve the efficiency and effectiveness of learning, and can also reduce the introduction and operating costs associated with owning multiple human models.

In the second step, the learner can pick up the tool to be studied that will be used in the actual treatment, examination, etc., when learning the procedure or treatment of the procedure, and apply the tool to be studied to the human body model onto which the first virtual augmentation is projected.

This allows the learner to visually get a sense of realism as if they were in front of a patient, and because they hold the actual tools being studied and apply them to the human body model, rather than using virtual images that do not involve any sensation, they get a tactile response that is close to that of actual work. As a result, the learner receives visual and tactile stimulation that makes them feel as if they are performing treatment on a patient in an actual location (in other words, they get realistic visual and tactile sensations), and a high learning effect can be expected.

Among the "learning instruments," medical instruments include, for example, puncture needles, drainage tubes, suture needles, syringes, forceps, scalpels and other medical blades, plates and pins used in fracture treatment, etc., and examination instruments include, for example, probes in ultrasound diagnostic equipment, electrodes in electrocardiogram measuring equipment, endoscopes, etc. It goes without saying that the medical instruments and examination instruments mentioned above are merely examples, and various instruments can be the subject of the study.

In the past, when learning about treatments and examinations, it was common to prepare various types of human dummies depending on the examination area and case, and preparing multiple human dummies would result in high procurement costs and require a lot of storage space when not in use. In contrast, the learning method of the present invention makes it possible to change the content of the first virtual augmentation projected by the mixed reality display, and by projecting various images or videos (first virtual augmentation) onto one human dummies, it is possible to obtain the same effect as actually owning multiple human dummies. In other words, the learning method of the present invention requires less procurement costs and less storage space when not in use compared to conventional learning methods.

Furthermore, according to the learning method of the present invention, the mixed reality display can project the first virtual augmentation onto the non-physical parts of the human model (parts that have no substance, which can also be considered space). For example, if the only part of the physical human model in front of the viewer's eyes is the torso, the first virtual augmentation of the non-physical parts of the human model, such as the head, lower limbs, and arms, can be visualized in a three-dimensional, superimposed manner, making it appear as if the head and other parts are added to the human model as if they were real. In addition, the non-physical parts can be switched between display and non-display as necessary. In other words, according to the learning method of the present invention, the mixed reality display can be used to carry out various types of learning even if the viewer does not own multiple human models, and switching between display and non-display of the non-physical parts can be expected to improve the efficiency and effectiveness of learning, and the introduction and operation costs associated with owning multiple human models can be reduced.

In addition, the above-mentioned learning method may be such that the mixed reality display is capable of visualizing a second virtual augmentation, which is a facial expression model of the patient, superimposed on the facial portion of the human body model, and in the first and second steps, the second virtual augmentation is applied to the human body model, and in at least the second step, the learner studies while appropriately viewing the facial expression model of the patient projected by the second virtual augmentation, or studies while appropriately viewing the facial expression model of the patient and engaging in conversation according to the situation.

According to this learning method, at least in the second step, the second virtual augmentation is visualized (applied) superimposed on the facial portion of the human body model, allowing the learner to study while also visually checking the facial expression model of the patient role projected by the second virtual augmentation as appropriate, or to study while visually checking the facial expression model of the patient role and also engaging in conversation according to the situation.

In other words, this learning method allows trainees to practice the procedure or treatment as if they were dealing with an actual patient, without having to prepare an actual patient or a role-playing patient (in other words, even though they are only dealing with a human model), and trainees are provided with visual stimulation as if they were actually treating a patient in an actual setting (in other words, they get a realistic visual experience), which is expected to result in even greater learning effects.

The second virtual augmentation may be applied to the human body model not only in the second step, but also in the first step. In this case, not only can the trainee directly learn the technique, but he or she can also train to observe the changes in the patient's facial expression due to a sudden change in the patient's condition before the procedure begins, and to talk to the patient to ease their anxiety and tension.

Furthermore, the facial expression model of the patient represented by the second virtual augmentation may be, for example, a standard model preset in the mixed reality display, or it may be the academic instructor or a fellow attendee receiving instruction. If the academic instructor or fellow attendee is used as the facial expression model of the patient, a sense of tension is provided during learning, which is expected to result in a high learning effect (on the other hand, it may be possible to provide humor and expect a high learning effect in a relaxed atmosphere).

Furthermore, when the learning instructor or other attendees are used as facial expression models for the patient role, image processing software may be used to display real-time facial expressions captured by a camera. This may also be used in conjunction with a setting that generates a sound such as "Ouch!" in accordance with the facial expression of a standard model playing the patient role represented in the second virtual augmentation. When the learning instructor or other attendees are used as real-time facial expression models for the patient role, the model and the learner may be set to be able to converse, allowing training in how to respond flexibly to situations.

The above-mentioned learning method may also be such that, among the facial expression models visualized in the second virtual augmentation, at least the depiction of the eyes is set so that the gaze can be moved toward the learner at any time, and the gaze can be recognized by the learner via the mixed reality display.

According to this learning method, when the learner is studying while visually checking the facial expression model of the patient role projected by the second virtual augmentation, the line of sight of the patient role can be changed and the learner can recognize this.

When actual patients or people undergoing medical examinations feel anxious or in pain, they may look at the face (expression) of the practitioner (doctor, technician, etc.). In other words, according to this learning method, at least the line of sight of the facial expression model visualized and applied to the human body mannequin moves in the direction of the learner at appropriate times (in other words, the line of sight is directed toward the learner), allowing the learner to experience a sense of tension and realism close to that of an actual treatment, which is expected to further improve the effectiveness of learning.

In this learning method, the timely movement of the gaze may be, for example, a standard action preset (programmed) in the mixed reality display, but is not limited to this. It may also be that a supervising instructor or attendee observing the learner's training status uses software or the like to intentionally move the gaze, or a pressure sensor or pressure sensor provided in the human model is linked to the software of the mixed reality display, and the gaze is set to move toward the learner at the appropriate time when a specified pressure is detected. Furthermore, when linked to a pressure sensor or the like provided in the human model, the presence or absence of gaze movement and the duration of the gaze movement may be set to vary depending on the strength of the detected pressure.

In addition, in the learning method of this aspect, the second virtual augmentation may be applied to the human body model in the first step as well as in the second step. In addition, the facial expression model in the learning method of this aspect may be a standard model preset in the mixed reality display, or may be that of a teaching instructor or a person in attendance, etc.

In addition, the above-mentioned learning method may be such that the first virtual augmentation allows medical images and three-dimensional anatomical images, set to an appropriate size to fit the human body model, to be superimposed on the same screen.

In this context, "medical images" includes two-dimensional computed tomography images (hereinafter referred to as "2D CT images"), magnetic resonance images (hereinafter referred to as "2D MRI images"), ultrasound images (hereinafter referred to as "echo images"), X-ray images (hereinafter referred to as "X-ray images"), nuclear medicine images (hereinafter referred to as "RI images"), etc., and does not exclude other types of medical images.

　Traditionally, in actual medical practice, medical images (e.g., 2D CT images or 2D MRI images) were not observed in life-size, and there was no environment in which the aforementioned 2D medical images and 3D anatomical images could be studied in life-size and stereoscopically. As a result, many students and inexperienced practitioners (doctors and technicians) had difficulty understanding the relationship between 2D medical images and 3D anatomical images.

However, according to the learning method of this aspect, the learner can learn while appropriately viewing the stereoscopic life-size medical images and 3D anatomical images superimposed and projected by the first virtual expansion. In other words, the learner can learn by associating the spatial positional relationship between the medical images (especially 2D ones) and the 3D anatomical images (3D), which allows training to understand the relationship between the medical images and the 3D anatomical images, which many students and inexperienced practitioners (doctors and technicians) have struggled with, and further deepening their understanding, and further improving the learning effect can be expected. Furthermore, it is expected that by deepening the learner's understanding of the relationship between the medical images and the 3D anatomical images, the learner (i.e., future practitioners) will ultimately be able to intuitively understand the positions of organs, etc., just by looking at the medical images (especially 2D medical images).

In the learning method of this aspect, the medical images and 3D anatomical images may be, for example, standard healthy images, but are not limited to this, and may be images of organs, bones, etc. of a patient or individual with a lesion or specific characteristics, or may be used as photographed medical images of an individual and 3D anatomical images constructed based on the medical images. When images of organs, bones, etc. of a patient with a lesion are applied, they can be used not only by learners but also by medical professionals including doctors in considering treatment plans and pre-operative meetings before actual surgery or treatment.

"Able to superimpose medical images and 3D anatomical images on the same screen" means that medical images and 3D anatomical images can be superimposed on the same screen, and includes both simultaneous overlapping display and switching between medical images and 3D anatomical images that are displayed alternately in the same position.

Furthermore, the above-mentioned learning method may be such that a pseudo-structure reproducing any one of skin, muscle, bone, blood vessel, membrane or organ is applied to at least a specific part of the human body model that is the learning target, or a combination of multiple pseudo-structures may be applied.

According to this learning method, when applying the learning tool to the human body model in training that simulates examination or treatment, a pseudo structure that reproduces skin, etc. is applied to the area where the examination or treatment is performed, so that a tactile response closer to that of the human body can be obtained. This allows the learner to receive tactile stimulation as if they were performing treatment on a patient in an actual location (in other words, they can get a realistic sense of touch), and a higher learning effect can be expected.

The human body model may be one in which multiple pseudo structures of skin, muscle, bone, blood vessel, membrane or organ are applied to a specific part to be studied. For example, if skin, bone and target organ are selected as pseudo structures and these are arranged in a superimposed (layered) manner, in training for puncturing the target organ, the learner can learn by actual touch how to palpate the gap between the bones, how much force to use to pierce the skin, how to insert the needle between the bones, and how much force and depth to use to insert the needle into the target organ. Furthermore, if pseudo structures of muscle, membrane and blood vessel are applied, the learner can feel the resistance of the muscle or membrane when puncturing, and learn the technique of inserting the needle between the muscle fibers or blood vessels without damaging them.

Note that although the pseudo-structure is applied to "at least the specified portion to be studied," the pseudo-structure may be applied to the entire human body model, not just a part of it. If the pseudo-structure is applied to the entire human body model, the structure becomes more complex, but one human body model can be used to train a variety of tests and procedures, improving convenience and versatility.

In addition, in the above-mentioned learning method, the simulated structures may be made of soft materials for skin, muscles, blood vessels, membranes, and organs, and semi-hard or hard materials for bones.

According to this learning method, when applying the learning tool to the human body model in the training simulating the above-mentioned examination or treatment, a pseudo structure made of a material with a hardness corresponding to the part of the body to be examined or treated is applied, providing a tactile response that is even closer to that of the human body. This allows the learner to receive tactile stimulation as if they were actually performing treatment on a patient in an actual setting (in other words, they can get a realistic sense of touch), and an even greater learning effect can be expected.

As "soft materials", materials such as resin and rubber are preferably used that have a hardness close to that of the target skin, muscles, blood vessels, membranes, and organs. As "hard materials", materials such as resin, rubber, stone materials, and metal materials are preferably used that have a hardness close to that of the target bones. In addition, semi-hard materials such as resins and rubbers that have been prepared to be harder may be used when reproducing cartilage.

In addition, the above-mentioned learning method may be configured such that the human body model and the learning target device are provided with a position sensor and a pressure sensor or pressure sensor, and when the position information detected by the position sensor and/or the pressure value detected by the pressure sensor or pressure sensor exceed a preset value, the second virtual augmentation is changed to an expression expressing discomfort or distress.

According to this learning method, when applying the training tool to the human body model in training that mimics the above-mentioned examination or treatment, if a treatment that would be painful for a human is performed, the facial expression model represented by the second virtual extension changes to one that expresses discomfort or agony, allowing the learner to immediately determine that they have performed an inappropriate treatment, and to train in performing treatment while observing (visually checking) the facial expression, just as they would when performing treatment on a real human.

Whether or not a treatment would cause pain in a human body is determined by whether or not the position information and/or pressure value detected by the position sensor and pressure sensor or pressure sensor installed on the human body model and the training target device exceed a preset value.

For example, if position information on the puncture depth is transmitted by a position sensor attached to the needle, which is the learning target instrument, and the position information received by the mixed reality display directly or via an external device that analyzes the position information is inappropriate (e.g., the puncture position is too deep), the facial expression model represented by the second virtual augmentation displayed on the mixed reality display changes to an expression expressing discomfort or distress. Also, for example, if a pressure sensor or pressure sensor is provided on a specific part of the human body model that is the learning target, and during training to press an examination device (e.g., a probe in an ultrasound diagnostic device), which is the learning target instrument, against the part, a pressure value is transmitted by a pressure sensor or the like attached to the human body model, and if the pressure value received by the mixed reality display directly or via an external device that analyzes the pressure value is inappropriate (e.g., the pressing force is too strong), the facial expression model represented by the second virtual augmentation displayed on the mixed reality display changes to an expression expressing discomfort or distress.

Furthermore, this can be used in conjunction with a setting that generates a sound such as "Ouch!" in response to changes in facial expression that express discomfort or distress, which increases the sense of realism and allows the training to be done with a sense of tension.

In addition, the above-mentioned learning method may be such that the human body model is provided with a camera capable of photographing the learner at the eye position if the model has a head, or at a position equivalent to the eye if the model does not have a head.

According to this learning method, it is possible to capture video from the perspective of a learner observing training (learning) that simulates examinations and treatments (in other words, video taken from the side of the human body model), and obtain the video. In other words, the learner can objectively view the facial expressions and behavior of the therapist as seen by the patient during actual treatment, and can also experience the psychology of the patient, such as how they are perceived by the patient, making it possible to learn about both the therapist and the treated in a single training session.

The aforementioned "camera" may be provided in any way that allows it to photograph the learner, but if it is a fixed structure, it is preferable that it has a wide angle of view, and it may also be a movable structure. Examples of movable structures include a structure in which the head or eye unit (including those located in a position equivalent to the eye unit) can be operated manually, automatically, or remotely to face the learner and photograph the learner.

The images captured by the camera may be displayed in real time on a mixed reality display worn by the learner, or may be recorded on a hard disk or the like of a personal computer (hereinafter referred to as "PC") connected to the mixed reality display. In the case of a real time display on a mixed reality display, the images may be displayed in a sub-window that opens next to the image of the human body model that the learner is looking at, or may be displayed full screen by appropriately switching with the image of the human body model that the learner is looking at. In addition, in the case of a recording on a hard disk or the like of a PC, the learner, etc. can check the images on another monitor or the like after the training (learning). Furthermore, the captured images may be simultaneously output to a large monitor, in which case the images can be shared with other learners waiting other than the learner currently training (learning), and the learner who will next begin training can easily understand the patient's perspective and state of mind, which is expected to improve the learning efficiency of the entire learner group.

In addition, the above-mentioned learning method may be such that a human body model is installed in an indoor space that can be used as a teaching space, and a mixed reality display is capable of visualizing a third virtual augmentation, which is a model of an examination device and/or a treatment device, superimposed on the indoor space, and in both or either of the first and second steps, the learner learns the procedure or procedure to be performed while visually viewing the third virtual augmentation projected onto the indoor space via the mixed reality display.

In addition, the expression "examination device and/or treatment device" used in this embodiment means both an examination device and a treatment device, and either an examination device or a treatment device.

According to this learning method, by visually viewing the third virtual augmentation projected onto the indoor space via the mixed reality display, the indoor space can be viewed as an examination room or treatment room, and the human body model placed there can be viewed as a subject to be examined or treated, etc., for learning purposes.

In this case, in both or either of the first and second steps, the learner may learn the procedure or procedure to be performed while further viewing the third virtual augmentation projected into the indoor space via the mixed reality display. For example, while viewing the image of the third virtual augmentation (such as an inspection device model) projected into the indoor space, the learner may learn or practice preparation of the inspection device, etc. before starting, check procedures, etc., and may also learn or practice how to handle the learner and his/her movements according to the positions of the inspection device, etc. and the human body model after starting.

Examples of images projected by the third virtual augmentation include MRI (Magnetic Resonance Imaging) inspection equipment, CT (Computed Tomography) inspection equipment, X-ray inspection equipment, radiation therapy equipment, proton beam therapy equipment, and endoscopic equipment (endoscopes and the surgical tools used therewith). Learners can learn and train their movements by moving a human body model relative to the inspection equipment projected by virtual augmentation, or by moving the arm of the inspection equipment projected by virtual augmentation and applying it to the human body model.

In other words, this learning method allows students to practice the procedure or treatment they need to perform without having to prepare actual examination or treatment equipment, as if they were treating a patient in a space with actual examination or treatment equipment. This provides students with visual stimulation as if they were actually treating a patient in a real setting (in other words, they get a realistic visual experience), and is expected to have an even greater learning effect.

In order to achieve the above object, the learning system of the present invention comprises a human body model to be projected, a mixed reality display that can visualize a first virtual augmentation, which is a physical anatomical model, superimposed on a part or all of the human body model and can be worn on the learner's head, and a learning target instrument that is a medical instrument or examination instrument to be used for procedural learning.

Here, by wearing the mixed reality display, the learner can view the human body model and the first virtual augmentation projected onto it via the mixed reality display. When learning the procedure or treatment of a procedure, the learner can pick up the tool to be studied that will be used in the actual treatment, examination, etc., and apply the tool to be studied to the human body model onto which the first virtual augmentation is projected.

This allows the learner to visually obtain a sense of realism and immersion, as if they were in front of a patient, and because they hold the actual tools being studied and apply them to the human body model, rather than using a virtual image that does not involve any sensation, they can obtain a tactile response that is close to that of actual work. As a result, the learner receives visual and tactile stimulation that makes them feel as if they are actually performing treatment on a patient in an actual location (in other words, they get realistic visual and tactile sensations), and a high learning effect can be expected.

The "human body model" may be anything onto which the first virtual augmentation described above can be projected, and for example, a life-size model, such as a so-called training model doll, is preferably used. Furthermore, as described below, the human body model does not necessarily have to be full-body size, since the mixed reality display can project the first virtual augmentation onto parts of the human body model that do not have a physical body, and it may be in a form consisting of only the parts particularly necessary for learning (for example, only the torso without the upper limbs, lower limbs, or head).

The "learning subject instrument" may be any medical or testing instrument that is the subject of skill learning, and various instruments may be the subject of learning. Examples of medical instruments include needles, tubes, scalpels, and other medical blades, and examples of testing instruments include probes in ultrasound diagnostic equipment and electrodes in electrocardiogram measuring equipment.

The "mixed reality display" is at least capable of visualizing the first virtual augmentation, which is a physical anatomical model, on a part or all of the human body model in a superimposed manner, and is constructed so as to be wearable on the learner's head. A head-mounted display (goggles, glasses, helmet, etc.) capable of implementing so-called XR (Extended reality) technology is used. Among XR (Extended reality) technologies, MR (Mixed Reality) technology, which has a high degree of integration between the human body model and the first virtual augmentation and excels in three-dimensional expression, is preferably used, but is not limited to this, and for example, AR (Augmented Reality) technology can also be used and is not excluded.)

Examples of the "first virtual augmentation" include images of human organs and bones. Projecting images of organs, etc. onto appropriate locations on the human body model increases the sense of immersion in learning and also allows for advance confirmation of the positions of organs, etc. Furthermore, the "first virtual augmentation" includes not only single images of human organs, etc., but also projection of multiple images in an overlapping manner (superimposed manner). One example of projection in an overlapping manner is a manner in which an image of a bone and an image of an organ located below the same bone image are projected in an overlapping manner. In this case, more practical learning is possible by referring to the arrangement of the multiple projected images.

The image data projected in the "first virtual augmentation" may be installed in the mixed reality display or stored in an auxiliary storage device connected to the aircraft, or may be received from an external device such as a server connected to the mixed reality display by wire or wireless means. Furthermore, image processing related to the "first virtual augmentation" may be performed by a function provided in the mixed reality display, or may receive data processed by an external device such as a server connected to the mixed reality display by wireless means or the like.

In other words, according to the learning system of the present invention, the content of the first virtual augmentation projected by the mixed reality display can be changed, and by projecting various images or videos (first virtual augmentation) onto one human body model, it is possible to obtain the same effect as having multiple human body models, which means that compared to the previously described conventional systems for learning about treatments and examinations, the system requires less procurement costs and requires less storage space when not in use.

Furthermore, according to the learning system of the present invention, the mixed reality display can project the first virtual augmentation onto the non-physical parts of the human model, so that the non-physical parts projected by the first virtual augmentation can be expressed as if they were added to the human model as if they were actually present. In addition, the non-physical parts can be switched between display and non-display as needed. In other words, according to the learning system of the present invention, the mixed reality display allows various learning activities to be carried out even if the user does not own multiple human models, and switching between display and non-display of the non-physical parts is expected to improve the efficiency and effectiveness of learning, and can also reduce the introduction and operating costs associated with owning multiple human models.

The learning system described above may also be configured so that the mixed reality display is configured to be able to visualize a second virtual augmentation, which is a facial expression model of the patient, superimposed on the face of the human body model.

According to the learning system of this embodiment, the second virtual augmentation can be visualized (applied) superimposed on the facial portion of the human body model. This allows the learner to study while also visually checking the facial expression model of the patient role projected by the second virtual augmentation as needed, or to study while visually checking the facial expression model of the patient role as needed and engaging in conversation according to the situation. Furthermore, in this case, this learning is not only for directly acquiring skills, but also for training in observing changes in the patient's facial expression due to a sudden change in the patient's condition before the start of the procedure, and in conversation to ease the patient's anxiety and tension.

Furthermore, with this learning system, it is possible to train in the procedure or treatment to be performed as if the student were dealing with a real patient, without having to prepare an actual patient or a role-playing patient. This provides the student with visual stimulation as if they were actually performing treatment on a patient in a real setting, and is expected to have an even greater learning effect.

The facial expression model of the patient represented in the second virtual augmentation may be a standard model or a person present, as described above, and in the case of a person present, it may be possible to display real-time facial expressions processed by video processing software. Also, as described above, this may be used in conjunction with a setting in which sound is generated in accordance with the facial expression of the standard model represented in the second virtual augmentation, and if a person present is used as the facial expression model, it may be set so that the model and the learner can converse, allowing training in how to respond flexibly to situations.

The learning system described above may also be configured such that the gaze of at least the depiction of the eyes in the facial expression model visualized in the second virtual augmentation can be moved toward the learner at any time, and the gaze can be recognized by the learner via the mixed reality display.

With this type of learning system, when the learner is studying while visually checking the facial expression model of the patient role projected by the second virtual augmentation, the line of sight of the patient role can be changed and the learner can recognize this.

Actual patients and those receiving medical treatment may look at the face of the practitioner when they feel anxious or in pain. In other words, according to this learning method, at least the line of sight of the facial expression model visualized and applied to the human body mannequin moves toward the learner at appropriate times, allowing the learner to experience a sense of tension and realism close to that of an actual treatment, which is expected to further improve the effectiveness of learning.

In the learning system of this aspect, the timely movement of the gaze can be, for example, a standardized action preset in the mixed reality display, a learning instructor observing the learner's training status using software or the like to intentionally move the gaze, or a pressure sensor or the like provided on the human model is linked to the software or the like of the mixed reality display, and the gaze is set to move toward the learner at the appropriate time when a specified pressure is detected. Furthermore, when linked to a pressure sensor or the like provided on the human model, the presence or absence of gaze movement and the duration of the gaze movement can be set to vary depending on the strength of the detected pressure.

In addition, in the learning system of this embodiment, the second virtual augmentation may be applied to the human body model not only in the second step but also in the first step. Also, the facial expression model in the learning system of this embodiment may be a standard model preset in the mixed reality display, or may be that of a learning instructor or a companion, etc.

Furthermore, the above-mentioned learning system may be one in which the first virtual augmentation is provided so that medical images and three-dimensional anatomical images, set to an appropriate size to fit the human body model, can be superimposed on the same screen. Note that "medical images" here includes the above-mentioned two-dimensional CT images, etc., and does not exclude other types of medical images.

According to the learning system of this aspect, the learner can study while appropriately viewing the stereoscopic life-size medical images and 3D anatomical images superimposed and projected by the first virtual extension. In other words, the spatial positional relationship between the medical images (particularly 2D ones) and the 3D anatomical images (3D) can be associated and studied on the same screen, which allows training to understand the relationship between the medical images and the 3D anatomical images, which many students and inexperienced practitioners have struggled with, and further deepens their understanding, and is expected to further improve the learning effect. Furthermore, it is expected that as the learner's understanding of the relationship between the medical images and the 3D anatomical images deepens, the learner will ultimately be able to intuitively understand the position of organs, etc., just by looking at the medical images (particularly the 2D medical images).

"Able to superimpose medical images and 3D anatomical images on the same screen" means that medical images and 3D anatomical images can be superimposed on the same screen, and includes both simultaneous overlapping display and switching between medical images and 3D anatomical images that are displayed alternately in the same position.

In this learning system, the medical images and 3D anatomical images may be standard healthy images, images of organs, bones, etc. of a patient or individual with a lesion or specific characteristics, or images of a photographed individual's medical images and 3D anatomical images constructed based on the medical images. When images of organs, bones, etc. of a patient with a lesion are used, they can be used not only by learners but also by medical professionals, including doctors, in considering treatment plans and pre-operative meetings before carrying out actual surgery or treatment.

In addition, the learning system described above may be configured such that the human body model and the learning object instrument are provided with a position sensor and a pressure sensor or pressure sensor, the mixed reality display has a receiving function capable of receiving the position information detected by the position sensor and the pressure value detected by the pressure sensor or pressure sensor, and a set value storage function, and when both or either one of the position information and the pressure value received by the receiving function exceeds a preset value, the second virtual augmentation is configured to change to an expression expressing discomfort or distress.

According to this learning system, when applying the learning tool to the human body model in training that mimics the above-mentioned examination or treatment, if a treatment that would be painful for a human is performed, the facial expression model represented by the second virtual extension changes to one that expresses discomfort or agony. This allows the learner to immediately determine if they have performed an inappropriate treatment, and allows training to be performed while observing the facial expression in the same way as when performing treatment on a real human.

Whether or not a treatment would cause pain in a human body is determined by whether or not the position information and/or pressure value detected by the position sensor and pressure sensor or pressure sensor installed on the human body model and the training target device exceed a preset value.

For example, if position information on the puncture depth is transmitted by a position sensor attached to the needle, which is the learning target instrument, and the position information received by the mixed reality display directly or via an external device that analyzes the position information is inappropriate, the facial expression model represented by the second virtual extension displayed on the mixed reality display changes to an expression expressing discomfort or agony. Also, for example, in a case where a pressure sensor or pressure sensor is provided on a specific part of the human body model that is the learning target, and during training in which the testing device, which is the learning target instrument, is pressed against the same part, a pressure value is transmitted by a pressure sensor or the like attached to the human body model, and if the pressure value received by the mixed reality display directly or via an external device that analyzes the pressure value is inappropriate, the facial expression model represented by the second virtual extension displayed on the mixed reality display changes to an expression expressing discomfort or agony. Furthermore, a setting in which sound is generated in accordance with the change to an expression expressing discomfort or agony may be used in combination, in which case the sense of realism is increased and training can be performed with a sense of tension.

Furthermore, the aforementioned learning system may be such that a pseudo-structure reproducing one of skin, muscle, bone, blood vessel, membrane or organ is applied to at least a specific part of the human body model that is the subject of learning, or a combination of multiple such pseudo-structures may be applied.

In this learning system, when applying the learning tool to the human body model in training that simulates examinations and treatments, a pseudo structure that reproduces skin, etc. is applied to the area where the examination or treatment is performed, so that a tactile response closer to that of the human body is obtained. This allows the learner to receive tactile stimulation as if they were actually performing treatment on a patient in the actual field, and a higher learning effect can be expected.

The human body model may also be one in which multiple pseudo structures of skin, muscle, bone, blood vessel, membrane or organ are applied to a specific part to be studied. For example, if skin, bone and target organ are selected as pseudo structures and arranged in a superimposed manner, in training for puncturing the target organ, the learner can learn, through actual touch, how to palpate the gap between the bones, how much force to use to pierce the skin, how to insert the needle between the bones, and how much force and depth to use to insert the needle into the target organ. Furthermore, if pseudo structures of muscle, membrane and blood vessel are applied, the learner can feel the resistance of the muscle or membrane when puncturing, and learn the technique of inserting the needle between the muscle fibers or blood vessels without damaging them.

In addition, the pseudo-structure may be applied not only to a part of the human body model, but also to the entire body. In this case, although the structure becomes more complex, one human body model can be used to train a variety of tests and procedures, which is convenient and further improves versatility.

Furthermore, in the above-mentioned learning system, the simulated structures may be made of soft materials for skin, muscles, blood vessels, membranes, and organs, and semi-hard or hard materials for bones.

According to this learning system, when applying the learning tool to the human body model in the training that mimics the above-mentioned examination or treatment, a pseudo structure made of a material with a hardness corresponding to the part of the body to be examined or treated is applied, so that a tactile response that is even closer to that of the human body can be obtained. This provides the learner with a tactile stimulation that makes them feel as if they are actually performing treatment on a patient in the actual field, and an even greater learning effect can be expected.

In addition, the learning system described above may be such that the human body model is provided with a camera capable of photographing the learner at the eye position if the model has a head, or at a position equivalent to the eye if the model does not have a head.

According to the learning system of this embodiment, it is possible to capture video from the perspective of an observer of a learner undergoing training (learning) that simulates an examination or treatment, and obtain the video. In other words, the learner can objectively view the facial expressions and behavior of the practitioner as seen by the patient during actual treatment, and can also experience the psychology of the patient, such as how they are perceived by the patient, making it possible to learn about both the practitioner and the treated in a single training session.

The aforementioned "camera" may be provided in any way that allows it to photograph the learner, but if it is a fixed structure, it is preferable that it has a wide angle of view, and it may also be a movable structure. Examples of movable structures include a structure in which the head or eye unit (including those located in a position equivalent to the eye unit) can be operated manually, automatically, or remotely to face the learner and photograph the learner.

The images captured by the camera may be displayed in real time on a mixed reality display worn by the learner, or may be recorded on a hard disk of a personal computer connected to the mixed reality display. In the case of a real-time display on a mixed reality display, the images may be displayed in a sub-window that opens next to the image of the human body model that the learner is looking at, or may be displayed full screen by appropriately switching with the image of the human body model that the learner is looking at. In addition, in the case of a recording on a hard disk of a personal computer, the learner can check the images on another monitor after the training (learning). Furthermore, the captured images may be simultaneously output to a large monitor, in which case the images can be shared with other learners waiting other than the learner currently training (learning), and the learner who will next begin training can easily understand the patient's perspective and state of mind, which is expected to improve the learning efficiency of the entire group of learners.

The learning system described above may also be configured so that the mixed reality display can visualize a third virtual augmentation, which is a model of an examination device and/or a treatment device, superimposed on the indoor space in which the human body model is placed.

In addition, the expression "examination device and/or treatment device" used in this embodiment means both an examination device and a treatment device, and either an examination device or a treatment device.

According to the learning system of this aspect, by visually viewing the third virtual augmentation projected into the indoor space via a mixed reality display, the indoor space can be viewed as an examination room or treatment room, and the human body model placed there can be viewed as the subject to be examined or treated, etc., for learning. For example, while viewing an image of the third virtual augmentation (examination device model, etc.) projected into the indoor space, learning or training can be performed on preparation of the examination device, etc., and check procedures before starting, and learning or training on how to handle the situation and the learner's movements according to the positions of the examination device, etc. and the human body model after starting can also be performed.

In other words, this learning method allows students to practice the procedures or treatments they need to perform without having to prepare actual examination or treatment equipment, as if they were treating a patient in a space with actual examination or treatment equipment. This provides students with visual stimulation as if they were actually treating a patient in a real setting, and is expected to have an even greater learning effect.

The learning method and learning system of the present invention allow a learner to learn the procedure or steps of a procedure to be performed by using the learning tool on a real human body model while viewing a virtual extension of a physical anatomical model projected onto the human body model via a worn mixed reality display.

1 is a schematic diagram showing a configuration of a learning system according to a first embodiment of the present invention; FIG. 2 is an image diagram showing a human body model and a first virtual augmentation projected onto the human body model in the learning system shown in FIG. 1 . 1 shows a learning system for a second embodiment of the present invention, in which (a) is an oblique view showing the state before the first virtual extension and the second virtual extension are projected onto the human body model, (b) is an oblique view showing the state after the first virtual extension and the second virtual extension have been projected onto the human body model, and (c) is an oblique view showing the state after only the bone image has been erased from the first virtual extension projected in (b). This is a modified example (variant example 5) of the learning system according to the second embodiment shown in Figure 3, and is a front view showing a state in which the first virtual extension and the second virtual extension (chest CT image), virtual operation buttons, etc. are projected onto the human body model. 5 is an explanatory diagram of the usage state of the learning system shown in FIG. 4, in which (a) shows a state in which the display position of the second virtual extension (transverse chest CT image) projected onto the human body model has been lowered to approximately the middle position in the chest height direction by operation, and (b) shows a state in which the display position has been lowered further than the position in (a) by operation. 5 is an explanatory diagram of the usage state of the learning system shown in FIG. 4, in which (a) shows a state in which the display position of the second virtual expansion (chest CT coronal image) projected onto the human body model by operation is approximately in the middle position in the chest depth direction, and (b) shows a state in which the display position is positioned further back than the position in (a) by operation. 5 is an explanatory diagram of the use state of the learning system shown in FIG. 4, in which (a) shows a state in which the display position of the second virtual extension (chest CT sagittal image) projected onto the human body model by operation is approximately the middle position in the chest width direction, and (b) shows a state in which a 3D anatomical image is simultaneously superimposed on the image of (a) by operation. FIG. 13 is an explanatory diagram showing the configuration of a human body model used in a learning system according to a third embodiment of the present invention. An image diagram showing a third virtual augmentation projected onto a classroom in a learning system according to the fourth embodiment of the present invention.

The embodiments of the present invention will be described in more detail with reference to Figures 1 to 9. The following description will be given in the order of the first embodiment, the second embodiment, variation 1, variation 2, variation 3, variation 4, variation 5, the third embodiment, and the fourth embodiment. Furthermore, the reference numerals in each drawing are given to the extent that they reduce complexity and facilitate understanding, and in cases where multiple equivalent parts are given the same reference numeral, the reference numeral may only be given to some of the parts.

First Embodiment
(Learning System 1)
Please refer to Figures 1 and 2. A learning system 1 used in a classroom R1 includes a human body model 2 as a projection target, a mixed reality display 3 worn by a learner H, and a learning target tool 4 as a target for learning manual techniques by the learner H. Each part of the learning system 1 will be described in detail below.

(Human body model 2)
The human body model 2 can be the projection target for the first virtual augmentation described below. In this embodiment, it is a life-size training model doll with a head and torso, and a commercially available chest drain insertion simulator is used.

(Mixed reality display 3)
The mixed reality display 3 is a device capable of visualizing a first virtual augmentation 31, which is a physical anatomical model, in a superimposed manner on a part or all of the human body mannequin 2, and is provided so as to be wearable on the head of the learner H. In this embodiment, the mixed reality display 3 is a head-mounted display (goggle type) capable of implementing MR technology (see FIG. 1 ).

In this embodiment, the first virtual augmentation 31 is an image of a person's bones, organs, blood vessels, and trachea, and the data of these images is installed in the memory function unit of the mixed reality display, and the image generation function unit constructs an image in which multiple images are superimposed (superimposed mode), and the image is projected via the display unit.

(Learning Object 4)
The learning object instrument 4 is a subject of procedure learning for the learner H, and in this embodiment is a chest drain catheter and an inner tube (medical instrument) (see FIGS. 1 and 2).

[Function of learning system 1 and learning method using the same]
1 and 2, the operation of the learning system 1 and a learning method using the same will be described. The learning method includes at least the following first and second steps.

in particular,
(1) In the first step, the learner H wears the mixed reality display 3 on his/her head and visually recognizes the human body model 2 and the first virtual augmentation 31 projected thereon through the mixed reality display 3;
(2) In the second step, the learner H, who has completed the first step, visually checks the human body model 2 and the first virtual extension 31, picks up the tool 4 to be studied and applies it to the human body model 2, thereby learning the procedure or treatment to be performed.

The operation of the learning system 1 (and the learning method using it) will now be described. In the first step, before learner H begins learning, the instructor (or the learner himself) places the human model 2 in an appropriate position (on the bed in FIG. 1), wears the mixed reality display 3, and sets it up so that the first virtual augmentation 31 is correctly projected onto the human model 2. In this setting, the instructor, wearing the mixed reality display 3, operates the virtual controller that appears on the display to select the first virtual augmentation 31 to be projected, and adjusts the position etc. so that the first virtual augmentation 31 is visualized superimposed on the human model 2.

By wearing the mixed reality display 3, learner H can view the human body model 2 and the first virtual augmentation 31 projected onto it via the mixed reality display 3. When learning the procedure or treatment of a procedure, learner H can pick up the tool 4 to be studied that will be used in the actual treatment and apply the tool 4 to the human body model 2 onto which the first virtual augmentation 31 is projected.

As a result, learner H can visually obtain a sense of realism and immersion, as if he were in front of a patient, and because he holds the actual target tool 4 in his hands and applies it to the human body model 2, rather than using a virtual image that does not involve any sensation, he can obtain a tactile response that is close to that of actual work. As a result, learner H receives visual and tactile stimulation as if he were actually performing treatment on a patient in an actual workplace, and a high learning effect can be expected.

In this embodiment, the human body model 2 only includes the head and torso, but the mixed reality display 3 can project a first virtual extension, such as virtual upper and lower limbs, in a three-dimensional, superimposed manner onto the non-physical parts of the human body model 2, allowing the user to visually identify and learn about body parts that are connected to the part being studied (that are added to the part being studied).

Furthermore, the mixed reality display 3 can also switch between displaying and hiding the parts that do not have a physical presence as necessary. In addition, the mixed reality display 3 can easily change the content of the first virtual augmentation 31 to be projected by installing image data, and by projecting various images and videos onto one human model, it is possible to obtain the same effect as having multiple human models.

As a result, even if a user does not own multiple types of human body models, various types of learning can be done by changing the contents of the first virtual extension 31 or by switching between displaying and hiding the parts that do not have a physical body, which improves the efficiency and effectiveness of learning, while also reducing the introduction (procurement) and operating costs associated with owning multiple human body models, and requires less storage space when not in use.

Furthermore, the first virtual augmentation may use not only images of standard organs, etc., but also images of affected areas of individual patients collected during prior examinations, etc. In this case, before performing treatment or surgery on a patient with a specific condition, it is possible to conduct simulated training (learning) that is more in line with the case and incorporates visual and tactile sensations, compared to training using general models. Furthermore, by installing or updating data related to the first virtual augmentation, it is possible to learn about the examination of new or special cases, and to practice becoming proficient in new treatment methods.

Second Embodiment
(Learning System 1a)
Please refer to Fig. 3. The learning system 1a is another embodiment (second embodiment) of the learning system 1, and includes a human body model 2a, a mixed reality display 3a, and a learning target instrument 4a. The learning system 1a has a structure and effects in common with the learning system 1 of the first embodiment, so the common structure and effects will be omitted, and the different structure and effects will be described later. In the description of this embodiment, the mixed reality display 3a is not shown, but is referred to as the mixed reality display "3a" for the convenience of explaining the difference from the mixed reality display 3.

(Human body model 2a)
The human body model 2a can be the projection target for the first virtual expansion and the second virtual expansion described below, and in this embodiment is a life-size training model doll having a head, torso, and the upper half of the thighs (see Figure 3 (a); no special mechanisms such as a chest drain insertion part are provided).

(Mixed reality display 3a)
The mixed reality display 3a is a head-mounted display (goggles type) capable of implementing MR technology, and is configured to be able to visualize, in addition to the first virtual augmentation 31 described above, a second virtual augmentation 32, which is a facial expression model of the patient, superimposed on the facial portion of the human body model 2a (see Figures 3(b) and (c)).

In this embodiment, the second virtual augmentation 32 is an image of a person's face, and the data of this image is installed in the memory function unit of the mixed reality display, and then constructed by the image generation function unit as an image in which the image is superimposed on the facial portion of the human body model 2a (superimposed mode), and this image is projected via the display unit.

(Learning target instrument 4a)
The learning object tool 4a is a tool for the learner to learn a procedure, and in this embodiment is an ultrasound diagnostic device and its probe (examination tool) (see FIGS. 3(a) to 3(c)).

According to the learning system 1a and the learning method using the same, in addition to the first virtual extension 31, the second virtual extension 32 is visualized (applied) superimposed on the facial portion of the human body model 2a (see Figures 3(b) and (c)). This allows the learner to study while appropriately viewing the facial expression model projected as the second virtual extension 32. Furthermore, according to the learning system 1a, in addition to directly acquiring skills, learners can also train by observing changes in the patient's (human body model 2a's) facial expression due to a sudden change in the patient's condition before the start of the procedure, and by having a conversation with the patient to ease their anxiety and tension.

Furthermore, with the learning system 1a and the learning method using it, it is possible to train in the procedure or treatment to be performed as if the student were dealing with a real patient, without having to prepare an actual patient or a role-playing patient. This provides the student with visual stimulation as if they were actually performing treatment on a patient in a real setting, and an even greater learning effect can be expected.

[Modification 1]
The learning system 1a according to the first modification is configured to capture facial images of the instructor or attendees taken by the mixed reality display 3a or an external terminal as a facial expression model of the patient role represented by the second virtual augmentation 32, and to display real-time facial expressions processed by image processing software. In addition, the facial expression model and the learner are set to be able to converse with each other, so that training in responding flexibly to situations can be carried out.

Note that, except for the above points, the learning system 1a according to Modification 1 has the same structure and effects as the learning system 1a according to the second embodiment, so a description of the same structure and effects will be omitted, and the same reference numerals will be used for the learning system and the names of each part in the description of the differences. Also, although the learning system 1a according to Modification 1 is not shown in the drawings, it will be described using the same reference numerals as the second embodiment.

According to the learning system 1a of the first modified example and the learning method using the same, the facial expression model of the person in attendance plays the role of a patient, which creates a sense of tension during learning and is expected to have a high learning effect (on the other hand, it is possible that a good learning effect can be expected in a relaxed atmosphere by providing humor). Furthermore, according to the learning method using the learning system 1a of the first modified example, learning can be carried out including training in which the facial expression model of the patient is appropriately viewed and conversation appropriate to the situation is also included (for example, explanations or casual conversation to ease the patient's anxiety and tension).

[Modification 2]
In the learning system 1a according to the second modification, a pressure sensor (not shown) is provided in the torso of the human body model 2a, and a position sensor (not shown) is provided in the tip of the learning target instrument 4a. In addition, in the learning system 1a, the mixed reality display 3a has a receiving function for receiving position information detected by the position sensor, a receiving function capable of receiving a pressure value detected by the pressure sensor, and a setting value storage function, and is configured such that when both or either one of the position information and the pressure value received by each receiving function exceeds a preset setting value, the second virtual augmentation 32 changes to an expression expressing discomfort or agony.

Note that, except for the above points, the learning system 1a according to Modification 2 has the same structure and effects as the learning system 1a according to the second embodiment (and Modification 1), so a description of the same structure and effects will be omitted, and the same reference numerals will be used for the learning system and the names of each part in the description of the differences. Also, although the learning system 1a according to Modification 2 is not shown in the drawings, it will be described using the same reference numerals as in the second embodiment.

When training in ultrasound examination (so-called echo examination) using the learning system 1a of the second modified example, learner H presses the probe against the abdomen or other part of the human body model 2a, a pressure value is transmitted by a pressure sensor provided in the human body model 2a. The mixed reality display 3a receives the transmitted pressure value, and if the pressure value is inappropriate, the facial expression model represented by the second virtual augmentation 32 displayed on the screen of the mixed reality display 3a changes to an expression expressing discomfort or distress.

In other words, according to the learning system 1a of the second modification and the learning method using it, when the learning target tool 4a is applied to the human body model 2a in the training simulating the above-mentioned examination, if a treatment that would cause pain in a human is performed, the facial expression model represented by the second virtual extension 32 changes to a facial expression expressing discomfort or agony. This allows the learner H to immediately determine that he or she has performed an inappropriate treatment, and allows the learner H to train in performing the treatment while observing the facial expression in the same way as when performing the treatment on a real human.

[Modification 3]
In the learning system 1a relating to variant example 3, in the depiction of the eyes (eye portion 321) in the facial expression model (Figures 3 (b) and (c)) visualized in the second virtual extension 32, the gaze is set so as to be able to move toward the learner H as needed, and the gaze is set so that the learner H can recognize it via the mixed reality display 3a.

The learning system 1a according to the modified example 3 has the same structure and effects as the learning system 1a according to the second embodiment, except for the above points, so a description of the same structure and effects will be omitted, and the same reference numerals will be used for the learning system and the names of each part in the description of the differences. Also, although the learning system 1a according to the modified example 3 is not shown in the figures, it will be described using the same reference numerals as the second embodiment.

　According to the learning system 1a of the modified example 3 and the learning method using the same, when the learner H studies while visually checking the facial expression model of the patient role projected by the second virtual extension 32 as appropriate, the line of sight of the patient role can be changed and the learner H can recognize this. Note that actual patients and those receiving medical treatment may look at the face of the practitioner when they feel anxious or in pain, but according to the learning system 1a of the modified example 3 and the learning method using the same, the line of sight of the visualized facial expression model moves in the direction of the learner H as appropriate, allowing the learner to experience a sense of tension and realism similar to that of an actual treatment, which is expected to further improve the learning effect.

The aforementioned timely movement of the gaze can be performed automatically or manually. Examples of automatic movements include a mode in which the gaze is a standard operation preset in the mixed reality display, and a mode in which the gaze is set to move toward the learner at the appropriate time when pressure is detected at a specific part of the human model by linking a pressure sensor or the like provided on the human model with the software of the mixed reality display, etc. An example of manual movements includes a mode in which a teaching instructor or the like intentionally moves the gaze.

[Modification 4]
In the learning system 1a according to the fourth modification, a camera (not shown) capable of photographing the learner H is provided at the position of the eye 321 on the head of the human body model 2a. The camera is a fixed wide-angle camera, and the head of the human body model 2a is structured so that it can move left and right and up and down. The image captured by the camera can be displayed in real time on the mixed reality display 3 worn by the learner H, and is also provided so as to be recordable on a hard disk or the like of a personal computer wirelessly connected to the mixed reality display 3.

The learning system 1a according to Modification 4 has the same structure and effects as the learning system 1a according to the second embodiment, except for the above points, so a description of the same structure and effects will be omitted, and the same reference numerals will be used for the learning system and the names of each part in the description of the differences. Also, although the learning system 1a according to Modification 4 is not shown in the drawings, it will be described using the same reference numerals as the second embodiment.

According to the learning system 1a of the fourth modification and the learning method using the same, video can be captured from the viewpoint of observing the learner H during training (learning) simulating examinations and treatments, and the video can be obtained in real time. In addition, the learner can check the video recorded on the hard disk of a personal computer on another monitor after training (learning). In other words, the learner H can objectively view the facial expressions and actions of the therapist as seen by the patient during actual treatment, and can also experience the psychology of the patient, such as how they are perceived by the patient, making it possible to learn about both the therapist and the treated in a single training session.

Furthermore, the captured images can be simultaneously output to a large monitor, in which case the images can be shared with other students waiting in line in addition to the student currently undergoing training (studying). This makes it easier for the next student to begin training to understand the patient's perspective and state of mind, which is expected to improve the learning efficiency of the entire group of students.

In this modified example, the "camera" is the structure described above, but is not limited to this, and may be, for example, a movable structure or the like, as long as it is capable of photographing the learner. In addition, if the human body model does not have a head, a camera capable of photographing the learner may be provided at a position equivalent to the eye. Furthermore, if a stereo camera is installed as the camera, another learner playing the role of a patient, etc. can wear a mixed reality display and observe the image, thereby immersing himself in the three-dimensional environment from the perspective of the patient, etc., and also experiencing the psychology of the patient, etc.

[Modification 5]
The learning system 1a' according to the fifth modification shown in Figures 4 to 7 uses a human body model 2a' (headless, torso only), and as a first virtual extension 31a' applied to the human body model 2a, a two-dimensional CT image 311 (corresponding to the above-mentioned "medical image"; the same applies below) of an organ set to an appropriate size to fit the human body model 2a and a three-dimensional anatomical image 312 are provided so as to be superimposed on the same screen. Figures 4 to 7 show the entire image displayed on the mixed reality display 3a in the learning system 1a', which is viewed by the learner H.

Refer to Figure 4. The first virtual extension 31a' shown in this figure is a two-dimensional CT image 311 of the chest (multiple overlapping chest CT cross-sectional images). A number of handles (three in this modified example) and a number of buttons (eight in this modified example) expressed by virtual extension are displayed around the human body model 2a'. Each of the aforementioned handles is used to change the display position of the image, etc., and is operated by learner H by holding it on the display screen. Each of the aforementioned buttons is used to switch the display of images, etc., and is operated by learner H by pressing it on the display screen.

The learning system 1a' allows learner H to grasp the first handle 313 in the displayed image and move it up and down in the image to display a chest CT transverse image of any location. For example, when learner H grasps the first handle 313 and moves it downward from the position of the first handle 313 shown in Figure 4, the position of the displayed chest CT transverse image will gradually decrease as shown in Figures 5(a)-(b). Conversely, when learner H raises the first handle 313 that he is grasping, the position of the chest CT transverse image will gradually increase (not shown). In other words, the raising and lowering operation of the first handle 313 is linked to the display position on the chest CT transverse image.

Furthermore, the learning system 1a' allows learner H to grasp the second handle 314 in the displayed image and move it in the forward and backward directions of the image to display a chest CT coronal section image of any location. For example, when learner H grasps the second handle 314 and pushes it from the position of the first handle 313 shown in FIG. 6 toward the back of the displayed image, the position of the displayed chest CT coronal section image will gradually move toward the back as shown in FIGS. 6(a)-(b). Conversely, if learner H pulls the grasped second handle 314 toward the front, the position of the chest CT coronal section image will gradually move toward the front (not shown). In other words, the operation of moving the second handle 314 forward and backward is linked to the display position on the chest CT coronal section image.

Furthermore, the learning system 1a' allows learner H to grasp the third handle 315 in the displayed image and move it left and right (left and right in Figs. 4 to 6, front and back in Fig. 7(a)) to display a chest CT sagittal image of any location. For example, when learner H grasps the third handle 315 and pushes it from the position of the third handle 315 shown in Fig. 7(a) toward the back of the displayed image, the position of the displayed chest CT sagittal image will gradually move toward the back. Conversely, if learner H pulls the grasped third handle 315 toward the viewer, the position of the chest CT sagittal image will gradually move toward the viewer (not shown). In other words, the forward and backward (left and right) movement of the third handle 315 is linked to the display position in the chest CT sagittal image.

The first button 316 is an ON/OFF switch for displaying the 2D CT image 311. When the learner H presses the first button 316 (turns the switch ON) within the displayed image, the learning system 1a' can display the 2D CT image 311 (a 2D CT image of the chest in Figures 4 to 7(a)) (projecting the 2D CT image 311 in a superimposed manner onto the human body model 2a').

The second button 317 is an ON/OFF switch for displaying the 3D anatomical image 312. When the learner H presses the second button 317 (turns the switch ON) within the displayed image, the learning system 1a' can display the 3D anatomical image 312 (in FIG. 7(b) a 3D anatomical image of the chest) (projecting the 3D anatomical image 312 superimposed on the human body model 2a').

By alternately pressing the first button 316 and the second button 317, the images shown in Figs. 7(a) and (b) can be alternately displayed. In other words, according to the learning system 1a' of modified example 5 and the learning method using the same, learner H can study while appropriately viewing the stereoscopic life-size 2D CT image 311 and 3D anatomical image 312 superimposed and projected by the first virtual extension 31a', and can ultimately learn the spatial positional relationship between the 2D CT image 311 and the 3D anatomical image 312 by associating the two dimensions with the three dimensions.

This will provide training in understanding the relationship between 2D CT images and 3D anatomical images, something that many students and inexperienced practitioners have struggled with, and will allow for a deeper understanding, which is expected to further improve learning outcomes. Furthermore, it is expected that by deepening Learner H's understanding of the relationship between 2D CT images and 3D anatomical images, he or she will ultimately be able to intuitively understand the position of organs, etc., simply by looking at the 2D CT images.

In the learning system 1a' of variant 5, the 2D CT image 311 and the 3D anatomical image 312 may be images of a healthy standard, or images of the organs, bones, etc. of a patient or individual with a lesion or specific characteristics. When images of the organs, bones, etc. of a patient with a lesion are used, they can be used not only by learners but also by medical professionals including doctors to consider treatment plans and for pre-operative meetings before carrying out actual surgery or treatment.

"Able to superimpose 2D CT images and 3D anatomical images on the same screen" means that 2D CT images and 3D anatomical images can be superimposed on the same screen, and includes both cases where 2D CT images and 3D anatomical images are alternately displayed in the same position by switching, and where they are simultaneously displayed overlapping each other.

Third Embodiment
(Learning System 1b)
Please refer to Fig. 4. The learning system 1b is another embodiment (third embodiment) of the learning system 1, and includes a human body model 2b, a mixed reality display 3, and a learning target instrument 4. Since the learning system 1b has the same structures and effects as the learning system 1 of the first embodiment in terms of the mixed reality display 3 and the learning target instrument 4, the description of the common structures and effects will be omitted, and the structure and effects of the human body model 2b, which are different, will be described later. In addition, in the description of this embodiment, the mixed reality display 3 is not illustrated, but for convenience of description, the mixed reality display is designated by the symbol "3".

(Human body model 2b)
The human body model 2b can be a projection target of the first virtual augmentation 31 and the second virtual augmentation 32 (see FIG. 4). In this embodiment, the human body model 2b is a life-size training model doll having a head, a torso, and the upper half of the thighs, and a cavity is formed in the chest, and a pseudo-structure 21 (see the dashed line in FIG. 4) that reproduces the arrangement, shape, and texture of the skin, muscles, bones, pleura, lungs, and blood vessels is applied to the cavity. Of the pseudo-structure 21, the skin, muscles, pleura, lungs, and blood vessels are made of soft materials, and the bones are made of hard materials. The pseudo-structure may be a purchased model (ready-made product) of an individual organ or the like and assembled, or may be manufactured by the user using a 3D printer as described later.

When the pseudo-structure is produced in-house, it can be produced using a 3D printer, a resin film material (wrap), a foamed resin material (sponge), various hard and soft resin materials, or a combination of these. Furthermore, the pseudo-structure produced in-house or outsourced can be not only a standard organ, but also a reproduction of an individual patient's organ collected during a prior examination. In this case, compared to training using a general model, it is possible to carry out simulated training that is more in line with the case and incorporates visual and tactile sensations before performing surgery on a patient with a specific condition.

According to the learning system 1b and the learning method using the same, when the learning tool 4 is applied to the human body model 2b in training, a pseudo structure 21 that reproduces the shape and hardness of the area to be treated, such as the skin, is applied, so that a tactile response closer to that of the human body can be obtained. This allows the learner H to receive tactile stimulation as if he or she were actually performing treatment on a patient in the field, and a higher learning effect can be expected.

Furthermore, in the learning system 1b, the human body model 2b has a structure with a pseudo structure 21 in which each element is arranged in a superimposed manner, so that in training for the procedure of puncturing a target organ as shown in FIG. 4 (i.e., using the learning target instrument 4), learner H can learn the techniques with a sense of touch close to that of an actual human body, such as palpation to find between the bones, the amount of force to use when puncturing the skin, membrane, and muscle while feeling their resistance, the procedure of removing the needle between the bones without damaging the muscle fibers or blood vessels, and the amount of force and depth to use when reaching the needle into the target organ.

The image projected in the first virtual extension 31 is set to be projected at a position overlapping with the equivalent in the pseudo structure 21. For example, the bone part of the pseudo structure 21 and the image of the bone projected in the first virtual extension 31 are projected at an overlapping position. This allows the learner to visually recognize the course of the projected blood vessels, etc., and the arrangement of the organs and bones (first virtual extension 31) via the mixed reality display 3, and also the projected facial expression model (second virtual extension 32), so it goes without saying that learning through vision is also possible (however, in FIG. 4, the configuration of the pseudo structure 21 is emphasized to make it easier to understand, and the image projected in the first virtual extension 31 as shown in FIG. 2 is omitted, and only the position is shown).

Fourth Embodiment
(Learning System 1c)
Please refer to Fig. 5. The learning system 1c is another embodiment (fourth embodiment) of the learning system 1, and includes a human body model 2, a mixed reality display 3c, and a learning target instrument 4c. Since the learning system 1c has the same structure and effects as the learning system 1 of the first embodiment in terms of the structure and effects of the human body model 2, the description of the common structure and effects will be omitted, and the structure and effects of the different mixed reality display 3c and learning target instrument 4c will be described later. In addition, in the description of this embodiment, the mixed reality display 3c is omitted from the illustration, but for the convenience of explaining the difference from the mixed reality display 3, the mixed reality display is marked with the symbol "3c".

(Mixed reality display 3c)
The mixed reality display 3c is a head-mounted display (goggles type) capable of implementing MR technology, and in addition to the first virtual augmentation 31 and second virtual augmentation 32 described above, it is configured to be able to visualize a third virtual augmentation 33, which is an examination device (in this embodiment, a CT examination device), superimposed on the classroom R2, which is the indoor space in which the human body model 2 is installed (see Figure 5).

In this embodiment, the third virtual augmentation 33 is an image of the inspection device 331, and the data of these images is installed in the memory function unit of the mixed reality display, and then constructed by the image generation function unit as an image in which the image is superimposed (superimposed) on the indoor space of classroom R2, and this image is projected via the display unit. Note that, of the image of the inspection device 331, the gantry portion is projected into empty space, and the cradle portion is projected and superimposed onto a general bed on which the human body model 2 is placed.

(Learning target instrument 4c)
The learning target device 4c is an AED (Automated External Defibrillator).

According to the learning system 1c and the learning method using the same, the learner can view the third virtual extension 33 projected onto the space in the classroom R2 via the mixed reality display 3c, and can study by regarding the classroom R2 as an examination room and the human body dummy 2 (on which the first virtual extension 31 and the second virtual extension 32 are projected) installed there as the subject to be examined (see FIG. 5). The learner can then study or train on the preparation of the examination device 331 and the check procedures before starting while viewing the image of the third virtual extension 33, and can also study or train on the learner's movements and other movements according to the positions of the examination device 331 and the human body dummy 2 after starting.

Incidentally, since a contrast agent is used in a CT scan, there is a possibility that the subject may develop drug-induced shock. However, with the learning system 1c and a learning method using it, it is possible to conduct resuscitation training using the learning subject device 4c (AED) on the human body model 2, assuming that such a shock occurs. In other words, the learning system 1c reproduces a special environment within the classroom R2, and makes it possible to perform simulations for dealing with situations that may arise in that environment.

In other words, learning system 1c and a learning method using it allow trainees to practice the procedures or treatments they need to perform as if they were treating a patient in a space with actual testing equipment, without having to prepare actual testing equipment. This provides the trainee with visual stimulation as if they were treating a patient in an actual setting, and is expected to have an even greater learning effect. Furthermore, learning system 1c and a learning method using it make it easy to have a simulated experience as if there were virtual testing equipment, even when actual testing equipment does not exist, and is expected to help trainees become accustomed to the surrounding environment (examination room or operating room).

According to each of the learning systems 1 (1a, 1a', 1b, 1c) described above, the external output of the mixed reality display can be used to project the image seen by learner H onto another display, allowing learner H's perspective, situation, and other experiences (successful or unsuccessful examples, etc.) to be shared with other students. Furthermore, instructors can also provide guidance and advice to students while watching the other displays.

Although conventional medical simulators are positioned as tools for learning and mastering medical procedures and procedures, the above-mentioned learning systems 1 (1a, 1a', 1b, 1c) and the learning methods using them place emphasis on learning about the risks and safety of procedures and medical safety, something that could not be achieved with conventional medical simulators.

Furthermore, the various virtual augmentations displayed on the mixed reality display 3 (3a, 3c) can be given a function to superimpose a message on the area to be treated or on the correctness of the selected instrument for a particular disease or symptom on that area or instrument. For example, when the appropriate puncture area or range is touched, a pop-up message such as "correct" can be displayed, improving the effectiveness of self-study.

Furthermore, the first virtual augmentation 31 and the second virtual augmentation 32 displayed on the mixed reality display 3 (3a, 3c) in each of the above-mentioned learning systems 1 (1a, 1a', 1b, 1c) can select and project facial expressions and body types of people of all ages and genders from preset data. This allows for more practical learning while visually recognizing differences in body types and organs due to age and gender. In addition, since there is no need to prepare human body models of different ages, genders, and body types when learning, it is possible to reduce the introduction (procurement) and operating costs associated with owning multiple human body models, and less storage space is required when not in use.

In addition, with each of the above-mentioned learning systems 1 (1a, 1a', 1b, 1c), students (including active technicians and doctors) can intuitively learn from three-dimensional images (and can also be used for learning that does not involve procedural training). Even for doctors, radiological technicians, and clinical laboratory technicians, it is not easy to replace two-dimensional images with three-dimensional images in the mind unless they have a lot of experience (if they do not see many images), but with each of the above-mentioned learning systems 1 (1a, 1a', 1b, 1c), it is also possible to learn physical anatomy through vision.

　Each of the learning systems 1 (1a, 1a', 1b, 1c) described above also makes it possible to learn about rare anatomical or clinical cases. During surgical procedures in clinical settings, medical accidents can occur due to encountering rare anatomical or clinical cases, and there have been reported cases leading to the death of the patient. For example, there are cases where the course of blood vessels differs from that of general human anatomy, or where the existence of blood vessels themselves is unique. In such cases, the surgeon's lack of knowledge or experience can cause damage to blood vessels during treatment, leading to serious medical accidents.

However, with each learning system 1 (1a, 1a', 1b, 1c), it is possible to select whether to display 3D anatomical images, making it possible to reproduce and learn about rare anatomical or clinical cases, and this can also be used to learn about the risks, safety, and medical safety of procedures in rare cases such as those mentioned above.

In addition, with each of the learning systems 1a, 1a', 1b, and 1c described above, it is possible to reproduce and display the change in the patient's facial gaze as a second virtual extension, so that the patient's gaze turns toward the surgeon, which increases the sense of realism (especially the sense of tension) of the learner playing the role of the surgeon, and further improvements in the learning effect can be expected.

Furthermore, in daily clinical practice, 2D CT images are not observed at life-size, and currently there is no environment in which 2D CT images and 3D anatomical images can be studied in a life-size, stereoscopic manner. There have been no simulators to date that allow life-size 2D CT images and 3D anatomical images to be superimposed simultaneously and that allow learning to associate the spatial positional relationship between the 2D and 3D dimensions. However, with each of the learning systems 1a, 1a', 1b, and 1c described above, it is possible to simultaneously superimpose life-size 2D CT images and 3D anatomical images on a human body model (simulator), and this function can provide strong support for understanding the relationship between 2D CT images and 3D anatomical images, which many learners struggle with, and for training to do so.

　And, according to each of the learning systems described above, a human body model (simulator), which is a machine or tool, is used in a more anthropomorphic way, and in particular, each of learning systems 1a, 1a', 1b, and 1c adds interactivity based on the above-mentioned effects. Conventionally, there have been no learning systems that use a human body model or the like that is anthropomorphized and has interactivity, but according to the learning method using each of learning systems 1a, 1a', 1b, and 1c, it is expected that the learning effect will be more efficient and superior than learning using a conventional human body model or the like.

The terms and expressions used in this specification and claims are merely explanatory and are not limiting in any way, and are not intended to exclude terms and expressions equivalent to the features described in this specification and claims or parts thereof. It goes without saying that various modifications are possible within the scope of the technical concept of the present invention. Furthermore, the terms first, second, etc. are used to distinguish one element from another element, and do not imply rank or importance.

1, 1a, 1a', 1b, 1c Learning system 2, 2a, 2a', 2b Human body model 21 Pseudo structure 3, 3a, 3c Mixed reality display 31, 31a' First virtual augmentation 311 Two-dimensional CT image 312 Three-dimensional anatomical image 313 First handle 314 Second handle 315 Third handle 316 First button 317 Second button 32 Second virtual augmentation 321 Eye 33 Third virtual augmentation 331 Inspection device 4, 4a, 4c Learning target instrument H Learner R1, R2 Classroom

Claims (18)

The method is carried out using a human body mannequin as a projection target, a mixed reality display capable of visualizing a first virtual augmentation, which is a physical anatomical model, in a superimposed manner on a part or all of the human body mannequin, and a learning target instrument, which is a medical instrument or an examination instrument as a subject of procedure learning,
a first step in which a learner wears the mixed reality display and views the mannequin and the first virtual augmentation projected thereon through the mixed reality display;
a second step in which the learner, having completed the first step, picks up the learning target tool and applies it to the human body model while visually checking the human body model and the first virtual augmentation, thereby learning a procedure or treatment of a procedure to be performed;
A learning method that: the mixed reality display is capable of visualizing a second virtual augmentation, the second virtual augmentation being a facial expression model of a patient character, superimposed on a facial portion of the mannequin;
In the first and second steps, the second virtual augmentation is applied to the mannequin;
2. The learning method according to claim 1, wherein, at least in the second step, the learner studies while appropriately viewing the facial expression model of the patient role projected by the second virtual augmentation, or studies while appropriately viewing the facial expression model of the patient role and also engaging in conversation according to the situation. The learning method described in claim 2, wherein the facial expression model visualized in the second virtual augmentation is set so that the gaze can be moved toward the learner at any time in at least the eye depiction, and the gaze can be recognized by the learner via the mixed reality display. 3. The learning method according to claim 1 or 2, wherein the first virtual augmentation is provided such that a medical image and a three-dimensional anatomical image set to an appropriate size to fit the human body model can be superimposed on the same screen. 3. The learning method according to claim 1 or 2, wherein the human body model has applied to it, at least to a specific part to be studied, a pseudo structure reproducing any one of skin, muscle, bone, blood vessel, membrane, or organ, or a combination of a plurality of such pseudo structures. The learning method according to claim 5 , wherein the simulated structures are made of soft materials for skin, muscles, blood vessels, membranes, and organs, and are made of semi-hard or hard materials for bones. The learning method according to claim 2, wherein the human body model and the learning target instrument are provided with a position sensor and a pressure sensor or a pressure sensor, and the second virtual augmentation is configured to change to an expression expressing discomfort or agony when the position information detected by the position sensor and/or the pressure value detected by the pressure sensor or the pressure sensor exceed a preset value. 8. The learning method according to claim 1, wherein the human body mannequin has a camera capable of photographing the learner at an eye position if the human body mannequin has a head, or at a position equivalent to an eye position if the human body mannequin does not have a head. the mannequin is installed in an indoor space that can be used as a classroom, and the mixed reality display is capable of visualizing a third virtual augmentation, which is a model of an examination device and/or a treatment device, superimposed on the indoor space;
A learning method as described in claim 1, claim 2, claim 3 or claim 7, wherein in both or either of the first step and the second step, a learner learns the procedure or steps to be performed while visually viewing the third virtual augmentation projected into the indoor space via the mixed reality display. A human body model to be projected,
a mixed reality display capable of visualizing a first virtual augmentation, which is a physical anatomical model, superimposed on a part or all of the mannequin and which is wearable on a head of a learner;
A learning subject instrument which is a medical instrument or an examination instrument that is a subject of procedure learning;
A learning system. The learning system of claim 10 , wherein the mixed reality display is configured to be capable of visualizing a second virtual augmentation, which is a facial expression model of a patient, superimposed on a facial portion of the mannequin. The learning system described in claim 11, wherein the facial expression model visualized in the second virtual augmentation is configured so that at least the eye depiction has a gaze that can be moved toward the learner at any time, and the gaze is configured so that the learner can recognize the gaze via the mixed reality display. 12. The learning system according to claim 10 or 11, wherein the first virtual augmentation is configured to allow medical images and three-dimensional anatomical images, each set to an appropriate size to fit the human body model, to be superimposed on the same screen. The human body model and the learning object tool are provided with a position sensor and a pressure sensor or a pressure sensor;
The learning system described in claim 11, wherein the mixed reality display has a receiving function capable of receiving the position information detected by the position sensor and the pressure value detected by the pressure sensor or the pressure sensor, and a set value storage function, and is configured so that when both or either one of the position information and the pressure value received by the receiving function exceeds a predetermined set value, the second virtual augmentation changes to an expression expressing discomfort or distress. The learning system according to claim 10, claim 11, claim 12 or claim 14, wherein the human body model has applied to it, at least to a specific part to be studied, a pseudo structure reproducing any one of skin, muscle, bone, blood vessel, membrane or organ, or a combination of a plurality of such pseudo structures. The learning system according to claim 15 , wherein the simulated structures are made of soft materials for skin, muscles, blood vessels, membranes, and organs, and are made of semi-hard or hard materials for bones. The learning system according to any one of claims 10, 11, 12 and 14, wherein the human body model is provided with a camera capable of photographing the learner at an eye position if the human body model has a head, or at a position equivalent to an eye if the human body model does not have a head. The learning system of claim 10, claim 11, claim 12 or claim 14, wherein the mixed reality display is configured to be capable of visualizing a third virtual augmentation, which is a model of an examination device and/or a treatment device, superimposed on an indoor space in which the human body model is placed.

Images