CN116367394A - Handheld controller light source control method, system, device, equipment and medium - Google Patents

Abstract

The application discloses a control method, a system, a device, equipment and a medium of a light source of a handheld controller, and belongs to the technical field of electronics. The control method of the handheld controller light source provided by the embodiment of the application is applied to the head-mounted equipment associated with the handheld controller, and the method comprises the following steps: determining a first light source in the shooting range of a camera of the head-mounted device and a second light source out of the shooting range from N light sources contained in the handheld controller, wherein N is a positive integer; and sending a control signal to the handheld controller, wherein the control signal is used for controlling the first light source to be in a light-emitting state and controlling the second light source to be in a non-light-emitting state.

Description

Handheld controller light source control method, system, device, equipment and medium

technical field

The present application belongs to the field of electronic technology, and in particular relates to a control method, system, device, equipment and medium of a handheld controller light source.

Background technique

At present, the head-mounted device associated with the hand-held controller can obtain the six-degree-of-freedom pose of the hand-held controller based on the camera of the head-mounted device and the light source of the hand-held controller within the shooting range of the camera, so as to Control the working state of the handheld controller.

In the process of obtaining the above-mentioned six-degree-of-freedom pose, the above-mentioned light source needs to be in a light-emitting state when the above-mentioned camera is exposed. In order to ensure that the light source is in a light-emitting state when the camera is exposed, all light sources of the above-mentioned hand-held controller can be made to work in a constant light state, or all light sources of the hand-held controller can be turned on before the camera is exposed is turned off after exposure.

However, according to the above method, since all the light sources of the above-mentioned hand-held controller will be in a light-emitting state when the above-mentioned camera is exposed, the power consumption of the hand-held controller will be relatively large.

Contents of the invention

The purpose of the embodiments of the present application is to provide a control method, system, device, equipment and medium for a light source of a hand-held controller, which can solve the problem of high power consumption of the hand-held controller.

In the first aspect, an embodiment of the present application provides a method for controlling a light source of a hand-held controller, the method is applied to a head-mounted device associated with the hand-held controller, and the method includes: selecting from the N light sources included in the hand-held controller, Determine the first light source within the shooting range of the camera of the head-mounted device, and the second light source outside the shooting range, N is a positive integer; send a control signal to the handheld controller, and the control signal is used to control the first light source to emit light state, and control the second light source to be in a non-luminous state.

In the second aspect, the embodiment of the present application provides a hand-held controller light source control system, the system includes a hand-held controller, and a head-mounted device associated with the hand-held controller; Among the N light sources included in the controller, determine the first light source within the shooting range of the camera of the head-mounted device and the second light source outside the shooting range, where N is a positive integer; the head-mounted device is also used to send The hand-held controller sends a control signal, and the control signal is used to control the first light source to be in a light-emitting state, and to control the second light source to be in a non-light-emitting state; the hand-held controller is used to control the second light source based on the control signal when receiving the control signal. A light source is in a light-emitting state, and the second light source is controlled to be in a non-light-emitting state.

In the third aspect, the embodiment of the present application provides a device for controlling the light source of the hand-held controller, the device is associated with the hand-held controller, and the device includes: a determination module and a sending module; Among the light sources, determine the first light source within the shooting range of the camera of the device, and the second light source outside the shooting range, N is a positive integer; the sending module is used to send a control signal to the hand-held controller, and the control signal is used for The first light source is controlled to be in a light-emitting state, and the second light source is controlled to be in a non-light-emitting state.

In a fourth aspect, an embodiment of the present application provides a head-mounted device, the head-mounted device includes a processor and a memory, the memory stores programs or instructions that can run on the processor, and the programs or instructions are executed by the processor When realizing the steps of the method as described in the first aspect.

In a fifth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

In the sixth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to realize the method.

In a seventh aspect, an embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the method described in the first aspect.

In the embodiment of the present application, the head-mounted device associated with the hand-held controller can determine the first light source within the shooting range of the camera of the head-mounted device from the N light sources included in the hand-held controller, and the For the second light source outside the shooting range, N is a positive integer; and send a control signal to the handheld controller, the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state. Through this solution, since the head-mounted device can send a control signal to the hand-held controller associated with it to control the light source of the hand-held controller, only the first light source within the shooting range of the camera of the head-mounted device is emitting light. state, so in the process of obtaining the six-degree-of-freedom pose of the handheld controller, it is not necessary for all the light sources of the handheld controller to be in a light-emitting state at each exposure of the camera, thereby reducing the power consumption of the handheld controller .

Description of drawings

Fig. 1 is a schematic diagram of light source distribution of a 6 degree of freedom (DOF) handle;

Fig. 2 is a flow chart of a control method for a hand-held controller light source provided in an embodiment of the present application;

Fig. 3 is a schematic diagram of a control method for a hand-held controller light source provided in an embodiment of the present application;

Fig. 4 is a schematic diagram of a control device for a handheld controller light source provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of a head-mounted device provided by an embodiment of the present application;

Fig. 6 is a schematic diagram of the hardware of the head-mounted device provided by the embodiment of the present application.

Detailed ways

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

The method, system, device, equipment and medium for controlling the light source of the handheld controller provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

At present, the 6DOF handles that are often used in the field of extended reality generally use infrared light-emitting diodes (Infrared Light-Emitting Diode, IR-LED) to form a set of light rings on the hardware as mark points, thus forming a set of unique Pattern (Pattern), such as a group of patterns composed of 21 IR-LEDs, where the power consumption of each IR-LED is 0.036w. Then, the algorithm uses the camera of the head-mounted device associated with the 6DOF handle to track the pose state of the Pattern, and fuses the inertial measurement unit data on the 6DOF handle to finally obtain the 6DOF pose of the 6DOF handle.

It is worth noting that among all the IR-LEDs of the 6DOF handle, the IR-LEDs within the shooting range of the camera of the head-mounted device must be in a glowing state when the camera is exposed. Currently, two solutions exist:

1. All the IR-LEDs of the 6DOF handle work in a constant light state, so that the camera can capture a pattern composed of a group of IR-LEDs every time it is exposed. However, in this solution, the camera is in a non-exposed state most of the time, while all the IR-LEDs are always on and not extinguished, which leads to a large power consumption of the IR-LEDs.

2. All the IR-LEDs of the 6DOF handle are turned on before the camera exposure, and are turned off after the camera exposure, so as to ensure that the camera can capture a pattern composed of a group of IR-LEDs every time it is exposed. Although this solution can reduce the power consumption of IR-LED compared with solution 1, there are still some problems:

A. As shown in Figure 1, in order to enable all the IR-LEDs of the 6DOF handle 10 to cover any handle pose, the layout of the IR-LEDs on the 6DOF handle 10 is 360° visible, and the camera of the head-mounted device During exposure, the IR-LED11 within the shooting range of the camera will be simultaneously lit.

B. The camera of the head-mounted device can only capture a part of the IR-LEDs in all the IR-LEDs, and the IR-LEDs outside the shooting range of the camera do not need to be lit when the camera is exposed.

C. In a scene with reflective surfaces (such as mirrors, smooth tabletops, etc.), a connected area (blob) in the image that was not in the "field of view" when the IR-LED is fully bright can enter the "field of view" through reflection. This will interfere with the head-mounted device's detection and identification of the blob, and further interfere with the positioning and tracking of the controller, reducing the tracking accuracy and stability of the controller.

In order to solve the above problems, in the method for controlling the light source of the hand-held controller provided in the embodiment of the present application, the head-mounted device associated with the above-mentioned 6DOF handle (such as the hand-held controller in the embodiment of the application) can be obtained from the Among the N IR-LEDs (such as the light source in the embodiment of the present application), determine the first IR-LED within the shooting range of the camera of the head-mounted device, and the second IR-LED outside the shooting range, N is a positive integer; and a control signal is sent to the 6DOF handle, the control signal is used to control the first IR-LED to be in a light-emitting state, and control the second IR-LED to be in a non-light-emitting state. Through this solution, since the head-mounted device can send a control signal to its associated 6DOF handle to control the first IR-LED that is only within the shooting range of the camera of the head-mounted device among the IR-LEDs of the 6DOF handle It is in the light-emitting state, so in the process of obtaining the six-degree-of-freedom pose of the 6DOF handle, it is not necessary for all the IR-LEDs of the 6DOF handle to be in the light-emitting state at each exposure of the camera, so that the IR-LED light emission can be reduced The duty cycle of the state can reduce the power consumption of the 6DOF handle and increase the use time of the 6DOF handle for one charge.

Furthermore, the service life and failure rate of the IR-LED can be optimized, and in the scene with reflective surfaces, the IR-LED outside the "field of view" can be controlled to turn off, so as to reduce the interference to the blob detection and improve the robustness of the controller tracking. Stickiness and precision.

An embodiment of the present application provides a method for controlling a light source of a handheld controller, and FIG. 2 shows a flow chart of the method for controlling a light source of a handheld controller provided by the embodiment of the present application. As shown in FIG. 2 , the method for controlling the light source of the handheld controller provided in the embodiment of the present application may include the following steps 201 and 202 . The method is exemplarily described below by taking the head-mounted device associated with the handheld controller as an example to execute the method.

Step 201 , the head-mounted device determines, from the N light sources included in the handheld controller, a first light source within the shooting range of a camera of the head-mounted device and a second light source outside the shooting range.

Wherein, N is a positive integer.

Optionally, in this embodiment of the present application, the aforementioned handheld controller may be a virtual reality (Virtual Reality, VR) handle (for example, a 6DOF VR handle) or an extended reality (Extended Reality, XR) handle, or the like.

Optionally, in the embodiment of the present application, the above-mentioned head-mounted device may be a head-mounted display device associated with the above-mentioned handheld controller, for example, the head-mounted device may be a VR helmet, VR glasses, XR helmet or XR glasses wait.

Optionally, in the embodiment of the present application, the foregoing light source may be an IR-LED.

Optionally, in the embodiment of the present application, the aforementioned shooting range may be a spatial range within which the aforementioned camera can capture images of objects.

It can be understood that the first light source within the shooting range can be captured by the camera corresponding to the image; the second light source outside the shooting range cannot be captured by the camera.

Optionally, in the embodiment of the present application, the above step 201 may be specifically implemented through the following steps 201a to 201d.

Step 201a, the head-mounted device acquires the first pose information of the handheld controller.

Optionally, in this embodiment of the present application, the first pose information may indicate the position of the handheld controller in three-dimensional space.

The specific method for obtaining the first pose information by the aforementioned head-mounted device will be described in detail below.

Optionally, in the embodiment of the present application, the aforementioned head-mounted device may obtain the first pose information through the following method 1 or method 2.

method one

Optionally, in the embodiment of the present application, the above step 201a may specifically be implemented through the following steps 201a1 to 201a3.

Step 201a1, the head-mounted device aligns the attitude reference coordinate system of the handheld controller with the attitude reference coordinate system of the head-mounted device.

Optionally, in this embodiment of the application, the above-mentioned head-mounted device can realize the posture reference coordinate system of the above-mentioned handheld controller and the posture reference of the head-mounted device based on the 6DOF of the above-mentioned camera and the magnetometer of the head-mounted device. The coordinate system is aligned.

For a specific description of aligning the attitude reference coordinate system of the handheld controller with the attitude reference coordinate system of the head-mounted device, reference may be made to relevant descriptions in related technologies, and details are not repeated here to avoid repetition.

Step 201a2, the head-mounted device obtains the third pose information of the handheld controller in the last time frame.

Optionally, in this embodiment of the present application, the third pose information may indicate the position of the handheld controller in the three-dimensional space in the last time frame.

Optionally, in this embodiment of the present application, the above time frame is an image frame.

Step 201a3, based on the third pose information, the head-mounted device acquires the predicted first pose information of the handheld controller in the current time frame.

Optionally, in this embodiment of the present application, the above-mentioned head-mounted device may predict the position of the above-mentioned handheld controller in the current time frame through the inertial measurement unit (Inertial Measurement Unit, IMU) of the head-mounted device based on the third pose information. First pose information to get the first pose information.

Optionally, in this embodiment of the present application, the foregoing IMU may include a gyroscope or an accelerometer.

It should be noted that the head-mounted device aligns the attitude reference coordinate system of the handheld controller with the attitude reference coordinate system of the head-mounted device, which can ensure that the obtained first pose information is consistent with the position of the head-mounted device. The posture information is in the matching posture reference coordinate system, so as to ensure the accuracy of the head-mounted device controlling the working state of the handheld controller.

In the embodiment of the present application, since the head-mounted device can obtain the predicted first pose information of the handheld controller in the current time frame based on the obtained third pose information of the handheld controller in the previous time frame, it can Improve the flexibility of obtaining first pose information.

way two

Optionally, in the embodiment of the present application, the above step 201a may be specifically implemented through the following steps 201a4 and 201a5.

Step 201a4, the head-mounted device acquires the attitude angle of the handheld controller.

Step 201a5: The head-mounted device determines the first pose information of the handheld controller based on the pose angle.

For the specific description of obtaining the attitude angle by the head-mounted device and determining the first attitude information based on the attitude angle, reference may be made to relevant descriptions in related technologies, and details are not repeated here to avoid repetition.

In the embodiment of the present application, since the head-mounted device can determine the first pose information of the handheld controller based on the acquired attitude angle of the handheld controller, the flexibility of obtaining the first pose information can be further improved.

Step 201b, the head-mounted device acquires second pose information of the head-mounted device.

Optionally, in this embodiment of the present application, the second pose information may indicate the position of the aforementioned head-mounted device in three-dimensional space.

The specific method for obtaining the second pose information by the head-mounted device will be described in detail below.

Optionally, in the embodiment of the present application, the aforementioned head-mounted device may obtain the second pose information through the following method I or method II.

Method Ⅰ

Optionally, in the embodiment of the present application, the above-mentioned step 201a can be specifically implemented through the following step 201a6; then the above-mentioned step 201b can be specifically implemented through the following step 201b1.

Step 201a6, the head-mounted device acquires the first pose information of the handheld controller in the relative reference coordinate system between the handheld controller and the head-mounted device.

Optionally, in this embodiment of the present application, the head-mounted device may acquire the first pose information of the handheld controller in the relative reference coordinate system based on the data obtained by the IMU and the magnetometer of the head-mounted device.

Step 201b1, the head-mounted device acquires the second pose information of the head-mounted device in the relative reference coordinate system.

Optionally, in this embodiment of the present application, the head-mounted device may obtain the 6DOF of the head-mounted device, and combine the data obtained by the magnetometer to calculate the second pose of the head-mounted device in the above-mentioned relative reference coordinate system information.

In the embodiment of the present application, since the head-mounted device can obtain the first pose information of the hand-held controller under the above-mentioned relative reference coordinate system, and the second pose information of the head-mounted device under the relative reference coordinate system Therefore, in the case where IMU prediction cannot be performed, the first pose information and the second pose information can still be obtained, thereby improving the flexibility of obtaining pose information.

Mode II

Optionally, in the embodiment of the present application, before the above step 201a, the method for controlling the light source of the handheld controller provided in the embodiment of the present application may also include the following steps A and B; then the above step 201b may specifically be performed through the following Step 201b2 is realized.

Step A, the head-mounted device acquires 6DOF information of the head-mounted device.

Step B. The head-mounted device calculates an initial deviation value between the attitude of the head-mounted device and the attitude of the hand-held controller based on the 6DOF information and the attitude angle of the hand-held controller.

Optionally, in this embodiment of the present application, the foregoing attitude angle may be calculated based on data acquired by the IMU and the magnetometer of the foregoing head-mounted device.

Optionally, in the embodiment of the present application, after the above-mentioned initial deviation value is calculated, the above-mentioned head-mounted device may be aligned with the attitude of each subsequent time frame.

Step 201b2, the head-mounted device acquires the second pose information of the head-mounted device based on the first pose information and the initial deviation value.

In the embodiment of the present application, since the head-mounted device can first calculate the initial distance between the attitude of the head-mounted device and the attitude of the hand-held controller based on the acquired 6DOF information of the head-mounted device and the attitude angle of the hand-held controller The deviation value, and then based on the first attitude information and the initial deviation value, the second attitude information is obtained, so the second attitude information can be obtained without the magnetometer of the head-mounted device, thereby improving the acquisition of the second attitude information flexibility.

Step 201c, based on the first pose information and the second pose information, the head-mounted device calculates the angle between the normal direction of each light source in the handheld controller and the shooting direction of the camera.

Optionally, in the embodiment of the present application, the above-mentioned normal direction is the normal positive direction, that is, the direction from the inside of the light source to the outside (ie, the above-mentioned camera side). It can be understood that after the head-mounted device acquires the above-mentioned first pose information and the second pose information, it can determine the positions of the above-mentioned hand-held controller and the head-mounted device in three-dimensional space, and then can calculate the position of the hand-held controller. The angle between the normal direction of each light source and the shooting direction of the camera above.

Step 201d: The head-mounted device determines a light source with a corresponding included angle greater than or equal to 90 degrees among the N light sources as the first light source, and determines a light source with a corresponding included angle smaller than 90 degrees among the N light sources as the second light source.

It can be understood that the corresponding images of the light sources whose corresponding included angles are greater than or equal to 90 degrees can be captured by the above-mentioned camera; the corresponding images of the light sources whose corresponding included angles are smaller than 90 degrees cannot be collected by the above-mentioned cameras.

The method for controlling the light source of the handheld controller provided by the embodiment of the present application will be exemplarily described below with reference to the accompanying drawings.

Exemplarily, as shown in FIG. 3 , after the head-mounted device obtains the above-mentioned first pose information and the second pose information, it can calculate the normal direction of the light source 31 among the light sources of the handheld controller and the normal direction of the head-mounted device. If the angle between the shooting directions of the cameras 32 is greater than or equal to 90 degrees, the light source 31 is determined as the first light source, and if the angle is less than 90 degrees, the light source 31 is determined as the second light source.

In the embodiment of the present application, since the head-mounted device can calculate the normal direction of each light source in the hand-held controller and the above-mentioned camera based on the acquired pose information of the handheld controller and the pose information of the head-mounted device The included angle between shooting directions, and the light source whose corresponding included angle is greater than or equal to 90 degrees is determined as the first light source, and the light source whose corresponding included angle is less than 90 degrees is determined as the second light source, so the determination of the first light source and the second light source can be improved. The accuracy of the light source.

Step 202, the head-mounted device sends a control signal to the handheld controller.

Wherein, the above-mentioned control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state.

Optionally, in the embodiment of the present application, the aforementioned head-mounted device may send the aforementioned control signal to the aforementioned handheld controller through wireless communication with the aforementioned handheld controller.

In the method for controlling the light source of the hand-held controller provided in the embodiment of the present application, since the head-mounted device can send a control signal to the hand-held controller associated with it to control the light source of the hand-held controller, only the head-mounted device The first light source within the shooting range of the camera is in a light-emitting state, so in the process of obtaining the six-degree-of-freedom pose of the hand-held controller, it is not necessary for all light sources of the hand-held controller to be in the light-emitting state, which can reduce the power consumption of the handheld controller.

An embodiment of the present application also provides a control system for a light source of a handheld controller, and the system includes a handheld controller and a head-mounted device associated with the handheld controller.

Wherein, the above-mentioned head-mounted device can be used to determine the first light source within the shooting range of the camera of the head-mounted device and the second light source outside the shooting range from the N light sources included in the above-mentioned handheld controller. Light source, N is a positive integer.

The above-mentioned head-mounted device can also be used to send a control signal to the above-mentioned handheld controller, and the control signal is used to control the above-mentioned first light source to be in a light-emitting state, and control the above-mentioned second light source to be in a non-light-emitting state.

The above-mentioned handheld controller can be used for controlling the above-mentioned first light source to be in a light-emitting state and controlling the above-mentioned second light source to be in a non-light-emitting state based on the control signal upon receiving the above-mentioned control signal.

Optionally, in the embodiment of the present application, the above-mentioned head-mounted device may specifically be used to obtain the first pose information of the above-mentioned handheld controller; and obtain the second pose information of the head-mounted device; and based on the first Pose information and second pose information, calculating the angle between the normal direction of each light source in the above N light sources and the shooting direction of the above camera; and the light source corresponding to the angle greater than or equal to 90 degrees among the N light sources , determine as the above-mentioned first light source, and determine a light source with a corresponding included angle smaller than 90 degrees among the N light sources as the above-mentioned second light source.

Optionally, in the embodiment of the present application, the above-mentioned head-mounted device may specifically be used to align the posture reference coordinate system of the above-mentioned hand-held controller with the posture reference coordinate system of the head-mounted device; The third pose information of the last time frame; and based on the third pose information, obtain the predicted first pose information of the handheld controller in the current time frame.

Optionally, in the embodiment of the present application, the above-mentioned head-mounted device may specifically be used to obtain the first pose information of the above-mentioned hand-held controller in a relative reference coordinate system between the hand-held controller and the head-mounted device; And obtain the second pose information of the head-mounted device under the relative reference coordinate system.

Optionally, in the embodiment of the present application, the above-mentioned head-mounted device may also be used to obtain the 6DOF information of the head-mounted device before obtaining the first pose information of the above-mentioned handheld controller; and based on the 6DOF information and The attitude angle of the hand-held controller is to calculate the initial deviation value between the attitude of the head-mounted device and the attitude of the hand-held controller. The head-mounted device may specifically be configured to acquire second pose information of the head-mounted device based on the first pose information and the initial deviation value.

Optionally, in the embodiment of the present application, the aforementioned head-mounted device may be specifically configured to obtain the attitude angle of the aforementioned handheld controller; and based on the attitude angle, acquire the first pose information of the handheld controller.

For the specific description of the embodiments of the present application and the technical effects that can be achieved by each device, you can refer to the relevant descriptions in the above embodiment of the method for controlling the light source of the hand-held controller. In order to avoid repetition, details are not repeated here.

In the light source control system of the hand-held controller provided in the embodiment of the present application, since the head-mounted device can send a control signal to the hand-held controller associated with it to control the light source of the hand-held controller, only the head-mounted device The first light source within the shooting range of the camera is in a light-emitting state; and the hand-held controller can control the first light source to be in a light-emitting state based on the control signal after receiving the control signal, and control the light source of the hand-held controller to be in a light-emitting state. The second light source outside the shooting range is in a non-luminous state; therefore, in the process of obtaining the six-degree-of-freedom pose of the handheld controller, when the camera is exposed every time, it is not necessary for all light sources of the handheld controller to be illuminated state, thereby reducing the power consumption of the handheld controller.

In the method for controlling the light source of the hand-held controller provided in the embodiment of the present application, the execution subject may be a control device of the light source of the hand-held controller. In the embodiment of the present application, the light source control device of the handheld controller provided in the embodiment of the present application is described by taking the control device of the light source of the handheld controller as an example to execute the control method of the light source of the handheld controller.

The embodiment of the present application provides a light source control device of a handheld controller, which is associated with the handheld controller. As shown in FIG. 4 , the light source control device 40 of the handheld controller includes: a determination module 41 and a sending module 42 . The determination module 41 can be used to determine the first light source within the shooting range of the camera of the device and the second light source outside the shooting range from the N light sources included in the handheld controller, where N is a positive integer. The sending module 42 can be used to send a control signal to the handheld controller, and the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state.

In a possible implementation manner, the determination module 41 may be specifically configured to obtain the first pose information of the handheld controller; and obtain the second pose information of the device; and based on the first pose information and the second Pose information, calculate the angle between the normal direction of each light source in the above N light sources and the shooting direction of the above camera; and determine the light source with a corresponding angle greater than or equal to 90 degrees among the N light sources as the above first light source, and a light source whose corresponding included angle is smaller than 90 degrees among the N light sources is determined as the second light source.

In a possible implementation manner, the determination module 41 may be specifically configured to align the posture reference coordinate system of the handheld controller with the posture reference coordinate system of the device; and obtain the first position of the handheld controller in the last time frame Three pose information; and based on the third pose information, obtain the predicted first pose information of the handheld controller in the current time frame.

In a possible implementation manner, the determination module 41 may be specifically configured to: acquire the first pose information of the hand-held controller in a relative reference coordinate system between the hand-held controller and the above-mentioned device; The second pose information in the relative reference coordinate system.

In a possible implementation manner, the above device may further include an acquisition module and a calculation module. The acquiring module may be configured to acquire the 6DOF information of the device before the determining module 41 acquires the first pose information of the handheld controller. The calculation module can be used to calculate an initial deviation value between the posture of the device and the posture of the handheld controller based on the 6DOF information obtained by the obtaining module and the posture angle of the handheld controller. The determining module 41 may specifically be configured to acquire second pose information of the device based on the first pose information and the initial deviation value.

In a possible implementation manner, the determination module 41 may be specifically configured to obtain the attitude angle of the hand controller; and determine the first pose information of the hand controller based on the attitude angle.

In the control device of the hand-held controller light source provided in the embodiment of the present application, since the control device of the hand-held controller light source can send a control signal to the hand-held controller associated with it to control the light source of the hand-held controller, only in the The first light source within the shooting range of the camera of the device is in a light-emitting state, so in the process of obtaining the six-degree-of-freedom pose of the handheld controller, when the camera is exposed every time, all light sources of the handheld controller may not need to be In the light-emitting state, which can reduce the power consumption of the handheld controller.

The device for controlling the light source of the handheld controller in the embodiment of the present application may be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or other devices other than the terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR ) equipment, robots, wearable devices, ultra-mobile personal computer (ultra-mobilepersonal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc. ), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which are not specifically limited in this embodiment of the present application.

The device for controlling the light source of the handheld controller in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The device for controlling the light source of the hand-held controller provided in the embodiment of the present application can realize various processes realized by the method embodiments in FIG. 2 to FIG. 3 , and details are not repeated here to avoid repetition.

Optionally, as shown in FIG. 5 , the embodiment of the present application also provides a head-mounted device 500, including a processor 501 and a memory 502, and the memory 502 stores programs or instructions that can run on the processor 501. , when the program or instruction is executed by the processor 501, each step of the above embodiment of the method for controlling the light source of the handheld controller can be realized, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

FIG. 6 is a schematic diagram of a hardware structure of a head-mounted device implementing an embodiment of the present application.

The head mounted device 1000 includes but not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.

Those skilled in the art can understand that the head-mounted device 1000 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1010 through the power management system, so that the management of charging, discharging, and power management functions. The structure of the head-mounted device shown in FIG. 6 does not constitute a limitation to the head-mounted device. The head-mounted device may include more or less components than shown in the illustration, or combine some components, or arrange different components. , which will not be repeated here.

Wherein, the above-mentioned head-mounted device 1000 is associated with a hand-held controller, and the above-mentioned processor 1010 can be used to determine the first light within the shooting range of the camera of the head-mounted device 1000 from the N light sources included in the hand-held controller. A light source, and a second light source outside the shooting range, N is a positive integer. The above-mentioned radio frequency unit 1001 can be used to send a control signal to the handheld controller, and the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state.

In a possible implementation manner, the above-mentioned processor 1010 may be specifically configured to obtain the first pose information of the above-mentioned handheld controller; and obtain the second pose information of the above-mentioned head-mounted device 1000; and based on the first pose Information and the second pose information, calculate the angle between the normal direction of each light source in the above N light sources and the shooting direction of the above camera; and determine the light source with a corresponding angle greater than or equal to 90 degrees among the N light sources is the above-mentioned first light source, and the light source whose corresponding included angle is smaller than 90 degrees among the N light sources is determined as the above-mentioned second light source.

In a possible implementation manner, the above-mentioned processor 1010 may specifically be configured to align the posture reference coordinate system of the above-mentioned handheld controller with the posture reference coordinate system of the above-mentioned head-mounted device 1000; The third pose information of the time frame; and based on the third pose information, obtain the predicted first pose information of the handheld controller in the current time frame.

In a possible implementation manner, the above-mentioned processor 1010 may be specifically configured to: acquire the first pose information of the above-mentioned handheld controller in a relative reference coordinate system between the handheld controller and the above-mentioned head-mounted device 1000; and Obtain second pose information of the head-mounted device 1000 in the relative reference coordinate system.

In a possible implementation manner, the above-mentioned processor 1010 may also be configured to obtain the 6DOF information of the above-mentioned head-mounted device 1000 before obtaining the first pose information of the above-mentioned handheld controller; and based on the obtained 6DOF information, and the attitude angle of the hand-held controller to calculate an initial deviation value between the attitude of the head-mounted device 1000 and the attitude of the hand-held controller. The processor 1010 may specifically be configured to acquire the second pose information of the head mounted device 1000 based on the first pose information and the initial deviation value.

In a possible implementation manner, the processor 1010 may specifically be configured to acquire an attitude angle of the handheld controller; and determine first pose information of the handheld controller based on the attitude angle.

In the head-mounted device provided in the embodiment of the present application, since the head-mounted device can send a control signal to the hand-held controller associated with it to control the light source of the hand-held controller, only the camera of the head-mounted device The first light source within the shooting range of the handheld controller is in a light-emitting state, so in the process of obtaining the six-degree-of-freedom pose of the handheld controller, it is not necessary for all light sources of the handheld controller to be in a light-emitting state when the camera is exposed every time , so that the power consumption of the handheld controller can be reduced.

For the beneficial effects of the various implementations in this embodiment, refer to the beneficial effects of the corresponding implementations in the foregoing method embodiments. To avoid repetition, details are not repeated here.

It should be understood that, in this embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 is used for an image capture device (such as Camera) to process the image data of still pictures or videos. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here.

The memory 1009 can be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required by at least one function (such as a sound playing function, image playback function, etc.), etc. Furthermore, memory 1009 may include volatile memory or nonvolatile memory, or, memory 1009 may include both volatile and nonvolatile memory. Wherein, the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM , SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by the processor, the various processes in the above embodiment of the method for controlling the light source of the hand-held controller are implemented , and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.

Wherein, the processor is the processor in the head-mounted device described in the above embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and the like.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize the light source of the hand-held controller as described above Each process of the embodiment of the control method can achieve the same technical effect, so in order to avoid repetition, details are not repeated here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

An embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the various processes in the above embodiment of the method for controlling the light source of the hand-held controller, and can achieve The same technical effects are not repeated here to avoid repetition.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims (15)

1. A method for controlling a light source of a hand-held controller, which is applied to a head-mounted device associated with the hand-held controller, wherein the method comprises: From the N light sources included in the handheld controller, determine a first light source within the shooting range of the camera of the head-mounted device, and a second light source outside the shooting range, where N is a positive integer; Sending a control signal to the handheld controller, the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state. 2. The method according to claim 1, wherein the first light source within the shooting range of the camera of the head-mounted device is determined from among the N light sources included in the handheld controller, and The second light source outside the shooting range includes: Acquiring the first pose information of the handheld controller; Obtaining second pose information of the head-mounted device; Based on the first pose information and the second pose information, calculate the angle between the normal direction of each of the N light sources and the shooting direction of the camera; Determining a light source with a corresponding included angle greater than or equal to 90 degrees among the N light sources as the first light source, and determining a light source with a corresponding included angle smaller than 90 degrees among the N light sources as the second light source . 3. The method according to claim 2, wherein the acquiring the first pose information of the handheld controller comprises: aligning the attitude reference frame of the handheld controller with the attitude reference frame of the head mounted device; Obtain the third pose information of the handheld controller in the last time frame; Based on the third pose information, the predicted first pose information of the handheld controller in the current time frame is acquired. 4. The method according to claim 2, wherein the acquiring the first pose information of the handheld controller comprises: Obtaining the first pose information of the handheld controller in a relative reference coordinate system between the handheld controller and the head-mounted device; The acquisition of the second pose information of the head-mounted device includes: Acquiring the second pose information of the head-mounted device in the relative reference coordinate system. 5. The method according to claim 2, wherein, before acquiring the first pose information of the handheld controller, the method further comprises: Acquiring 6 degrees of freedom DOF information of the head-mounted device; Based on the 6DOF information and the attitude angle of the hand-held controller, calculate an initial deviation value between the attitude of the head-mounted device and the attitude of the hand-held controller; The acquisition of the second pose information of the head-mounted device includes: Acquiring second pose information of the head-mounted device based on the first pose information and the initial deviation value. 6. The method according to claim 2, wherein the acquiring the first pose information of the handheld controller comprises: Obtain the attitude angle of the handheld controller; Based on the attitude angle, first attitude information of the handheld controller is acquired. 7. A control system for a handheld controller light source, characterized in that the system includes a handheld controller, and a head-mounted device associated with the handheld controller; Wherein, the head-mounted device is configured to determine, from the N light sources included in the hand-held controller, the first light source within the shooting range of the camera of the head-mounted device and the first light source outside the shooting range The second light source, N is a positive integer; The head-mounted device is further configured to send a control signal to the handheld controller, the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state; The handheld controller is configured to control the first light source to be in a light-emitting state and control the second light source to be in a non-light-emitting state based on the control signal when the control signal is received. 8. A control device for a light source of a hand-held controller, the device is associated with the hand-held controller, characterized in that the device includes: a determination module and a sending module; The determining module is configured to determine, from the N light sources included in the handheld controller, a first light source within the shooting range of the camera of the device and a second light source outside the shooting range, where N is positive integer; The sending module is configured to send a control signal to the handheld controller, the control signal is used to control the first light source to be in a light-emitting state, and control the second light source to be in a non-light-emitting state. 9. The device of claim 8, wherein: The determining module is specifically configured to obtain first pose information of the handheld controller; and obtain second pose information of the device; and based on the first pose information and the second pose information , calculating the angle between the normal direction of each of the N light sources and the shooting direction of the camera; and determining the light source with a corresponding angle greater than or equal to 90 degrees among the N light sources as the first A light source, and a light source whose corresponding included angle is smaller than 90 degrees among the N light sources is determined as the second light source. 10. The apparatus of claim 9, wherein: The determining module is specifically configured to align the posture reference coordinate system of the handheld controller with the posture reference coordinate system of the device; and obtain third pose information of the handheld controller in a previous time frame; and Based on the third pose information, the predicted first pose information of the handheld controller in the current time frame is acquired. 11. The device of claim 9, wherein: The determination module is specifically used for: Acquiring the first pose information of the handheld controller in a relative reference coordinate system between the handheld controller and the device; Acquiring the second pose information of the device in the relative reference coordinate system. 12. The device according to claim 9, further comprising an acquisition module and a calculation module; The acquiring module is configured to acquire the 6DOF information of the device before the determining module acquires the first pose information of the handheld controller; The calculation module is configured to calculate an initial deviation value between the posture of the device and the posture of the handheld controller based on the 6DOF information acquired by the acquisition module and the attitude angle of the handheld controller; The determining module is specifically configured to acquire second pose information of the device based on the first pose information and the initial deviation value. 13. The apparatus of claim 9, wherein: The determining module is specifically configured to acquire an attitude angle of the handheld controller; and determine first attitude information of the handheld controller based on the attitude angle. 14. A head-mounted device, characterized in that it includes a processor and a memory, the memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, the following The steps of the control method of the hand-held controller light source according to any one of claims 1-6. 15. A readable storage medium, characterized in that the readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the handheld control according to any one of claims 1-6 is realized The steps of the control method of the device light source.

Images