FR3160785A1 - EXTENDED REALITY SYSTEM - Google Patents

Abstract

The invention relates to an extended reality system comprising an onboard haptic interface device (3) adapted to be integrated into an item of clothing intended to be worn by a user, the haptic interface device (3) comprising a set of ultrasonic transducers (13) controllable by a control module (5) to generate a haptic interaction space in at least a portion of the environment of the user wearing said onboard haptic interface device. Figure 1A for abstract.

Description

EXTENDED REALITY SYSTEM

The present invention relates to the field of extended reality and more particularly, that of the haptic devices which can be associated with it.

STATE OF THE PRIOR ART

An extended reality system is a real-time interaction interface between the real world and artificially created physical effects that augment our perception of or action on our environment. Physical effects can be visual cues to create images, audio signals, or ultrasonic pulses to create haptic sensations.

Haptic systems are known in the prior art for creating, for a user, a sensation of touch in a predefined fixed space such as inside a glove. Such a system comprises a set of controllable ultrasonic transducers to generate ultrasonic pulses. A control circuit controls the transmission of the transducers with phase shifts between the different transducers so as to focus the emitted waves at a given focal point in the predefined space. This makes it possible to generate, in the vicinity of the focal point, a pressure strong enough to be felt by the user. Different focal points can be successively scanned at a relatively high speed so as to generate in the predefined space a distribution of pressures perceptible by the user.

However, it is necessary to first define a fixed zone corresponding to all the focal points accessible by the transducers so that the user can benefit from the haptic sensations.

The object of the present invention is to propose an extended reality system comprising a haptic interface remedying the aforementioned drawbacks, in particular by not limiting the haptic sensations in a delimited space while increasing the haptic simulations felt by the user.

This objective is achieved with an extended reality system comprising an on-board haptic interface device adapted to be integrated into an item of clothing intended to be worn by a user, the haptic interface device comprising a set of ultrasonic transducers controllable by a control module to generate in the air a haptic interaction space in at least part of the environment of the user wearing said on-board haptic interface device.

This system allows for the creation of a haptic configuration space regardless of the location of use. The system also allows for a large acoustic emission surface emanating from the user wearing the haptic device.

According to one embodiment, the system comprises an extended reality device comprising a display module configured to display an image representative of a virtual optical object, and in that the on-board haptic interface device is configured to be connectable (for example wirelessly) to said extended reality device.

Advantageously, the extended reality device is a portable device intended to be worn on the user's head, the display module of the portable extended reality device comprising at least one display screen intended to be placed in front of at least one eye of the user, the display screen being controllable by the control module.

The use of the haptic interface in association with the wearing of a portable extended reality device makes it possible to combine haptic sensation with optical vision, allowing the user greater immersion in their extended, virtual, augmented or mixed reality experience. In addition, vision and touch sensation are allowed regardless of the location of use of the system.

According to a particular embodiment, the control module is integrated into the on-board haptic interface and/or into the extended reality device.

According to one variant, the control module is included in a remote device configured to remotely control the ultrasonic transducers and/or the display module. via control signals.

The remote device may be a mobile phone with a haptic control and/or display control application.

Advantageously, the extended reality device comprises a detection module adapted to detect at least one hand of the user.

Advantageously, the control module is configured to read the output data of the detection module and modify the control of the display module and the ultrasonic transducers accordingly.

Advantageously, the ultrasonic transducers are arranged in a set of haptic matrices or sub-matrices, the control module being configured to control the emission of ultrasonic pulses from the haptic sub-matrices with phase shifts between the different haptic sub-matrices so as to focus the emitted waves at a given point in the haptic space.

According to one embodiment, the ultrasonic transducers are PMUT-type piezoelectric transducers arranged in a set of haptic matrices. Each PMUT ultrasonic transducer comprises a flexible membrane suspended on a rigid support and a piezoelectric conversion element fixed on the flexible membrane.

Advantageously, a set of haptic interface devices is adapted to be worn by a corresponding set of users, thus generating at least one haptic interaction space in the environment of said set of users.

This allows users to interact with each other, with their environment, or with virtual optical objects.

The present invention will be better understood by reading the description of exemplary embodiments given purely for informational purposes and in no way limiting, with reference to the appended drawings in which:

THE FIG. 1 And FIG. 1 schematically illustrate an extended reality system comprising an on-board haptic interface device, according to one embodiment of the invention;

There FIG. 2 schematically illustrates an ultrasonic transducer of the haptic interface of the FIG. 1 ;

There FIG. 3 illustrates a pressure curve in Pa as a function of distance in cm for a silicon PMUT transducer;

There FIG. 4 illustrates very schematically the control of the ultrasonic transducers by the control module;

There FIG. 5 very schematically illustrates an extended reality system, according to another preferred embodiment of the invention;

There FIG. 6 schematically illustrates an extended reality system according to the FIG. 5 , comprising a portable extended reality device according to a preferred embodiment of the invention; and

There FIG. 7 schematically illustrates an application of the extended reality system, according to the embodiment of the FIG. 5 .

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

The concept behind the invention is to propose a haptic interface device embedded on the user so that the acoustic source emanates from the user himself.

Figs. 1A and 1B very schematically illustrate an extended reality system comprising an on-board haptic interface device, according to one embodiment of the invention.

The extended reality system 1 comprises an on-board haptic interface device 3, a control module or circuit 5, and a power source 7.

The on-board haptic interface device 3 is adapted to be integrated into an article of clothing 9 (see FIG. 1 ) intended to be worn by a user 11. An article of clothing is understood to mean anything that can be worn by a user, including any element (for example, straps, breastplate, etc.) added over the clothing worn by the user. The on-board haptic interface device 3 comprises a set of ultrasonic transducers 13.

According to one aspect of an embodiment, the ultrasonic transducers 13 are arranged in a set of haptic matrices or sub-matrices 15 arranged on a support 17 made of fabric or any other material suitable for being integrated into an article of clothing 9 which can be worn by the user 11.

The ultrasonic transducers 13 are controllable by the control module 5 to generate ultrasonic pulses in the air forming a haptic interaction space 19 in at least part of the environment of the user 11 wearing the on-board haptic interface device 3. The frequency of the ultrasonic pulses may be between 20 kHz and 10 THz, preferably between 20 kHz and 100 MHz.

The haptic interaction space 19 thus forms a three-dimensional variety (in the geometric sense of the term) with which are associated directional pressures with a haptic effect emanating from the user 11 wearing the on-board haptic interface device 3.

The control module 5 is an electronic circuit which can be integrated into the haptic interface device 3. Alternatively, the control module 5 can be a software application (or also a circuit) included in an external device linked or in communication with the haptic interface device 3.

According to one aspect of an embodiment, the ultrasonic transducers 13 are micromachined piezoelectric ultrasonic transducers, called PMUTs (Piezoelectric Micromachined Ultrasonic Transducers). The PMUT transducers may be made of ceramic or silicon. Other types of transducers may be used.

There FIG. 2 schematically illustrates an ultrasonic transducer of the haptic interface of the FIG. 1 .

This example concerns a PMUT transducer 13 comprising a flexible membrane 23 suspended by its periphery on a rigid support 25. The rigid support 25 is for example made of silicon or ceramic. The membrane 23 is fixed, by its lower face, on the upper face of the support 25. The membrane 23 can have a circular, rectangular, or square shape.

The PMUT transducer 13 further comprises a piezoelectric conversion element 27 fixed to the membrane, on its upper face side. The piezoelectric conversion element comprises two electrodes 27a, 27b.

The application of a voltage between the electrodes 27a, 27b of the piezoelectric conversion element 27 causes a deformation of the membrane 23 making it possible to generate an ultrasonic acoustic wave. Conversely, a deformation of the membrane 23 generates a voltage which can be used to measure an ultrasonic acoustic wave received by the transducer 13. This type of transducer is described in detail in the applicant's patent application FR3092680.

A PMUT 13 transducer can operate at different ultrasonic frequencies depending on its size or the technology used (silicon, ceramic or other). As a non-limiting example, we consider here the case of a PMUT 13 transducer of the silicon type, operating at 100kHz.

In fact, the FIG. 3 illustrates the pressure curve in Pa as a function of distance in cm for a silicon PMUT transducer.

More specifically, this curve is derived from measurements taken for a single PMUT 13 transducer at 100kHz operating under a driving voltage of 5V. For example, the curve shows that the PMUT 13 transducer generates an acoustic pressure of the order of 0.15 Pa at a distance of 30 cm.

It should be noted that the acoustic pressure increases almost linearly with increasing voltage. Thus, for a control voltage of 48 V, the PMUT 13 transducer generates an acoustic pressure of around 1.45 Pa at a distance of 30 cm.

In general, the threshold for feeling a haptic effect is around 200 Pa. Thus, to obtain a haptic effect at 30 cm, we can use around 140 PMUT transducers knowing that the pressures add up almost linearly.

As an indication, the diameter of a circular PMUT transducer is of the order of 800 µm for a silicon type transducer and is of the order of 5 mm for a ceramic type transducer. It should be noted that the diameter depends on the stiffness of the membrane 23, linked to its thickness and its constituent materials. The diameters considered above are indicative and taken as an example in the dimensioning of a haptic matrix 15.

Thus, using silicon type PMUT transducers and for a gap of the order of 300 µm between the membranes 23 of the neighboring transducers 13, a haptic matrix 15 of the order of 1.5 x 1.5 cm ² results, having a haptic effect (i.e. 200 Pa) at a distance of 30 cm. It will be noted that the gap of 300 µm between membranes is an indicative gap which is sufficient to allow solidity of the matrix while keeping the membranes close to each other.

The dimensions of the haptic matrices or sub-matrices 15 can be determined according to the emission frequencies of the PMUT transducers, the technology used (silicon or ceramic), the driving voltage thereof, and the specificities of the applications.

The table below shows the sizing of some configurations. The first column indicates the technology used (silicon or ceramic), the second column indicates the surface area of the haptic matrix, the third column indicates the driving voltage, the fourth column indicates the distance of the haptic effect (P=200 Pa), the fifth and sixth columns indicate the pressures felt at 30 cm and 50 cm respectively, the seventh column indicates the number of transducers forming the haptic matrix, and the eighth column indicates the diameter of each transducer.

Thus, depending on the desired application, a haptic matrix 15 of small size, light weight, and low driving voltage can be provided. In addition, the haptic interface device 3 can be sized to emit a haptic sensation from a few centimeters to several meters depending on the driving voltage and the surface area of the haptic matrices used.

There FIG. 4 illustrates very schematically the control of the ultrasonic transducers by the control module.

The control module 5 is configured to control the transducer matrices or sub-matrices 13 with phase shifts between the different sub-matrices 15 so as to focus the emitted waves at a focusing point 21 at a distance from the user. This distance can for example be between 5 cm and 200 cm.

This makes it possible to generate, in the vicinity of the focal point 21, a pressure strong enough to be felt by the user and/or by a collaborator in his environment. Different focal points can be successively scanned at a relatively high speed so as to dynamically generate a distribution of perceptible pressures in the haptic interaction space 19. The haptic interaction space 19 can thus be likened to a virtual and dynamic haptic object.

Furthermore, the power source 7 is configured to provide the transducers 13 with a driving voltage that can be between 5V and 50V. Advantageously, a driving track is used for each haptic matrix or sub-matrix 5. Thus, each driving track groups together a plurality of transducers 13, which facilitates the feasibility of the haptic interface device 3. The power source 7 can be integrated into the haptic interface device 3 and can optionally provide a voltage to the control module 5 when the latter is also integrated into the haptic interface device 3.

It is possible to integrate into a garment 9 a haptic interface device 3 comprising a relatively large surface area of haptic matrices or sub-matrices 15. Thus, a large acoustic emission surface area can emanate from a user 11 wearing a garment 9 in which such a haptic interface device is integrated. This induces an increase in the emitted acoustic power and therefore an increase in the sensation of touch felt by the user and/or his colleagues. This also induces an increase in the volume of a zone of use and the size of the contours of the haptic simulations, thus involving larger volumes of interaction zones on the surface and in depth.

It should be noted that the extended reality system 1 can be used by several users 11, each being equipped with a haptic interface device 3, thus collectively forming an enriched haptic interaction space within their common environment. When several users are equipped with these haptic interface devices 3, they benefit from the possibility of dynamic interactions between themselves, with their immediate environment, or even with virtual entities such as optical objects. This multimodal configuration promotes a collaborative and immersive experience where tactile interaction, either between users or with virtual or real elements is made possible.

A first application of the extended reality system 1 may be the generation of a unidirectional touch sensation without contact from a user 11 wearing an on-board haptic interface device 3, towards another person or user. This touch sensation may be bidirectional in case the other user also wears an on-board haptic interface device 3.

For example, a member of a medical team wearing clothing 9 incorporating an onboard haptic interface device 3 can generate a sensation of touch on a patient without this being done with direct contact. Another example is a bedridden patient who cannot move and who is wearing an onboard haptic interface device 3 can remotely activate a specific function on an object that he cannot reach or touch.

There FIG. 5 very schematically illustrates an extended reality system, according to another preferred embodiment of the invention.

According to this embodiment, the extended reality system 1 comprises an extended reality device 31, which may be portable or non-portable, in addition to the on-board haptic interface device 3. The extended reality device 31 comprises a display module 33 configured to display a 2D or 3D image representative of a virtual optical object.

According to this embodiment, the on-board haptic interface device 3 is configured to be connectable (for example wirelessly) to the extended reality device 31. For example, the extended reality device 31 and the on-board haptic interface device 3 may each comprise a communication module 35a, 35b allowing them to communicate with each other.

Furthermore, the control module 5 can be integrated into one and/or the other of the extended reality devices 33 and the on-board haptic interface 3.

Alternatively, the control module 5 may be included in a remote device (not shown) configured to remotely control via control signals the ultrasonic transducers 9 of the on-board haptic interface device 3 and/or the extended reality device 31. By way of example, the remote device may be a mobile phone comprising haptic control and display applications configured to control the on-board haptic interface device 3 and the extended reality device 31.

Furthermore, one or more power sources 7a, 7b is (are) configured to provide the power required for the on-board haptic interface device 3, extended reality device 31, and the control module 5 (which may be included in one and/or the other of the devices 3 and 31).

The extended reality device 31 may further comprise a detection module 37 adapted to detect elements of the environment. The control module 5 is configured to read the output data of the detection module 37 and accordingly modify the control of the display module 33 and the ultrasonic transducers 13.

By way of example, the detection module 37 is configured to detect the position of the hand of the user 11 so as to detect possible interactions of the user with virtual optical objects generated by the extended reality device 31, and to modify accordingly the virtual optical objects and/or the haptic effects generated.

The detection module 37 may comprise a camera, an infrared emission/reception detection element.

In a preferred embodiment, the extended reality device 31 may also comprise a plurality of ultrasonic transducers 13. In this case, the detection module 37 uses one or more of the ultrasonic transducers 13 to detect elements of the environment. For example, during a detection phase, certain ultrasonic transducers 13 may be controlled to generate an acoustic signal adapted to be reflected on the hands of the user, and other ultrasonic transducers 13 may be activated in reception to read the reflected acoustic signal. Thus, the hand of the user can be located and imaged by the extended reality system 31.

The ultrasonic transducers 13 used for detection may be the same as those used for generating the haptic effect. In this case, the detection phases and the haptic effect generation phases may be sequential. Alternatively, some ultrasonic transducers 13 may be dedicated to detecting the environment and other transducers may be dedicated to generating haptic effects, in which case the detection phases and the virtual object generation phases may be simultaneous.

There FIG. 6 illustrates in a very schematic way an extended reality system according to the FIG. 5 comprising a portable extended reality device according to a preferred embodiment of the invention.

According to this example, the extended reality device 31 is a portable device of the glasses type or of the virtual, augmented or mixed reality headset type.

The extended reality glasses 131 comprise a frame 133 intended to be worn on the user's head. The frame 133 is provided with one or two display screens 135 intended to be placed in front of at least one eye of the user.

As a non-limiting example, we consider here the case of a reality device 31 of the glasses 131 type like that described in the applicant's application FR3092680.

The display screens 135 are partially transparent screens, transmitting to the user's eyes all or part of the visible rays coming from the real scene located in front of the user. The user then sees images displayed by the screens 135 superimposed on the real scene in his field of vision.

According to this embodiment, the control module 5 is configured to control the display of 2D or 3D images on the screens 135 in addition to controlling the emission of ultrasonic pulses by the transducers 13 of the haptic interface device 3 according to the description relating to the FIG. 1 and possibly those integrated on the glasses 131. In particular, the control module 5 in cooperation with the detection module 37 can locate the user's hands.

More particularly, the control module 5 is configured to control the display of an optical virtual object and a haptic virtual object so that the two objects overlap at least at the location where the user's hands are located. Indeed, the relative position of the ultrasonic transducers 13 integrated in the clothing 9 or the glasses 131 of the user 11 relative to the display screens 135 can be easily predetermined. This results in an easy determination to be made of the concordance between the optical virtual object and a dynamic haptic virtual object as a function of the position of the user's hands.

Thus, the vision and the sensation of touching virtual objects can be achieved in harmony regardless of where the extended reality system is used.

There FIG. 7 illustrates very schematically an application of the extended reality system, according to the embodiment of the FIG. 6 .

This application relates to the projection of an optical object (hologram) 19 by the extended reality glasses 131 worn by the user 11 as well as the corresponding projection of a dynamic haptic object 191 by the haptic interface device 3 integrated into the clothing 9 of the user 11. For example, a caregiver wearing the haptic interface device 3 and the extended reality glasses 131 can project the hologram of a medical simulation mannequin and its corresponding haptic according to the positions of the caregiver's hands. This allows the caregiver to perform a virtual operation on this simulation mannequin.

According to another embodiment, the extended reality device 31 is an external device. In this case, the on-board haptic interface device 3 can be connected wirelessly (Wifi, Bluetooth, etc.) by means of the communication module 35a with the extended reality device 31 external. An application of the extended reality system 1 according to this embodiment may be the projection of a hologram by the external extended reality device 31 and the generation by the onboard haptic interface device 3 worn by a user of a haptic sensation on the user's fingers. For example, the external extended reality device 31 projects a 3D holographic plane onto a table or a room. The onboard haptic interface device 3 integrated into the clothing of a user 11 generates a haptic sensation on the user's fingers who will then have the sensation of actually touching the holographic plane.

Claims (11)

Extended reality system comprising an on-board haptic interface device (3) adapted to be integrated into an item of clothing (9) intended to be worn by a user (11), the haptic interface device (3) comprising a set of ultrasonic transducers (13) controllable by a control module (5) to generate a haptic interaction space in at least part of the environment of the user wearing said on-board haptic interface device. System according to claim 1, characterized in that it comprises an extended reality device (31) comprising a display module (33) configured to display an image representative of a virtual optical object, and in that the on-board haptic interface device (3) is configured to be connectable to said extended reality device (31). System according to claim 2, characterized in that the extended reality device (31) is a portable device intended to be worn on the user's head, the display module (33) of the extended reality portable device (31) comprising at least one display screen intended to be placed in front of at least one eye of the user, the display screen being controllable by the control module (5). System according to claim 2 or 3, characterized in that the control module (5) is integrated into the on-board haptic interface (3) and/or into the extended reality device (31). System according to any one of claims 2 to 4, characterized in that the control module (5) is included in a remote device configured to remotely control the ultrasonic transducers (13) and/or the display module. (33) via control signals. System according to any one of claims 2 to 5, characterized in that the extended reality device (31) comprises a detection module (37) adapted to detect at least one hand of the user. System according to claim 6, characterized in that the control module (5) is configured to read the output data of the detection module (37) and accordingly modify the control of the display module (33) and the ultrasonic transducers (13). System according to any one of the preceding claims, characterized in that the ultrasonic transducers (13) are arranged in a set of haptic matrices or sub-matrices (15), and in that the control module (5) is configured to control the emission of ultrasonic pulses from the haptic sub-matrices (15) with phase shifts between the different haptic sub-matrices (15) so as to focus the emitted waves at a given point in the haptic space. System according to any one of the preceding claims, characterized in that the ultrasonic transducers (13) are PMUT type piezoelectric transducers arranged in a set of haptic matrices. System according to any one of the preceding claims, characterized in that each ultrasonic transducer (13) comprises a flexible membrane (23) suspended on a rigid support (25) and a piezoelectric conversion element (27) fixed on the flexible membrane. System according to any one of the preceding claims, characterized in that a set of haptic interface devices is adapted to be worn by a corresponding set of users, thus generating at least one haptic interaction space in the environment of said set of users.