TW202005600A - Interpupillary distance computation apparatus and method - Google Patents

Abstract

A interpupillary distance (IPD) computation apparatus and method are provided. In the method, an image including barcode and human face is obtained. An actual object length included in the barcode is read from the image. An object corresponding to the actual object length included and the human face are recognized from the image, and image lengths of the object and IPD of the human face in the image are calculated. Then, an actual IPD can be determined according to the image lengths of the object and IPD, and the actual barcode length. Accordingly, the efficiency of IPD measure procedure can be improved.

Description

Interpupillary distance calculation device and method

The invention relates to an artificial intelligence (AI) technology, and in particular to an interpupillary distance (IPD) computing device and method using artificial intelligence technology.

In recent years, augmented reality (Augmented Reality, AR), virtual reality (VR) and mixed reality (Mixed Reality, MR) and other simulation environments or technologies that generate virtual objects have become increasingly popular. The aforementioned technologies can be applied in various fields such as games, remote conferences, and remote medical care. Among them, the head-mounted display (Head-Mounted Display, HMD) is commonly used in VR or MR products, so that the user has a better visual experience, and the user may have a personal experience. After wearing the HMD helmet, users will adjust the interpupillary distance setting of the helmet to improve the experience and reduce dizziness. However, the interpupillary distance of different users is not the same, and most users do not know the value of their interpupillary distance, so it is impossible to adjust the interpupillary distance setting to an appropriate value simply and quickly.

In the prior art, existing software can calculate the interpupillary distance through a webcam (Webcam) with scale items. The steps are: in front of the webcam, the user places the scale item above or below the eyes to take a selfie; the software obtains The length of the scale item entered by the user (it can be omitted if it is a common and uniform size object (for example, a credit card)); the user points the eye at the specified position according to the instructions of the software; the software calculates the pupil distance between the scale item and the eye position . In other words, the user needs to move the scale item and his eyes to the designated position by himself. Such a measurement process is obviously not simple and convenient. In addition, the use of credit cards has more worries about capital leakage.

In view of this, the present invention provides an interpupillary distance computing device and method, which allows the user to adjust the position of the item or eyes without reading the barcode and image recognition of the binocular features.

The pupil distance calculation method of the present invention includes the following steps. Obtain an image, and this image includes barcode and face. Read the actual length of the object included in the barcode from the image. Identify the object and face corresponding to the actual length of the object from the image, and calculate the frame length of the interpupillary distance in the object and face in the image. The actual interpupillary distance is calculated based on the object and the screen length of the interpupillary distance and the actual length of the object.

On the other hand, the interpupillary distance computing device of the present invention includes an image capturing device, a memory, and a processor. The image capturing device captures images. The memory records images and the modules. The processor is coupled to the image capturing device and the memory, and accesses and loads those modules recorded in the memory. Those modules include image access module, barcode reading module, image recognition module and interpupillary distance calculation module. The image access module obtains an image, and the image includes a barcode and a human face. The barcode reading module reads the actual length of the objects included in the barcode from this image. The image recognition module recognizes the object and the human face corresponding to the actual length of the object from this image, and calculates the frame length of the object and the interpupillary distance in the image. The interpupillary distance calculation module calculates the actual interpupillary distance according to the object and the screen length of the interpupillary distance and the actual length of the object.

Based on the above, the embodiment of the present invention uses barcodes to entrain its actual length, and recognizes eye features and object features based on image recognition technology to obtain corresponding frame lengths, and then based on the actual length of the object, interpupillary distance, and barcode in the image To calculate the actual interpupillary distance. The user does not need to move his eyes or barcode to the designated position, as long as he takes a selfie (for example, paper, card, eye mask or screen, etc.) with a printed or rendered barcode at the image capturing device, the embodiment of the present invention can automatically Calculate the actual interpupillary distance. And these vehicles can replace credit cards with personal capital, so as to solve the doubt of personal capital leakage. In addition, these actual pupil distances can be directly applied to the pupil distance setting on the head-mounted display, thereby improving the user's visual experience.

In order to make the above-mentioned features and advantages of the present invention more obvious and understandable, the embodiments are specifically described below in conjunction with the accompanying drawings for detailed description as follows.

FIG. 1 is a block diagram of components of an interpupillary distance computing device 100 according to an embodiment of the invention. Referring to FIG. 1, the interpupillary distance computing device 100 includes at least but not limited to the storage 110, the image capturing device 130 and the processor 150. The pupillary distance computing device 100 may be a desktop computer, a notebook computer, a workstation, a server, a smartphone, a tablet computer, or other electronic devices.

The memory 110 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (Flash memory), or the like Or a combination of the above components, the storage 110 is used to store buffered or permanent data, software modules (image access module 111, barcode reading module 112, image recognition module 113, operation guide module Group 114 and interpupillary distance calculation module 115, etc.), application programs, operation interface, images, feature points, actual length of objects, screen length, pixel density and other data, and the detailed contents will be described in detail in subsequent embodiments.

The image capturing device 130 may be a device with a dynamic or static image capturing function such as a camera, a camera, a webcam (Webcam), etc. The image capturing device 130 may include a lens, an image sensor, and an image processing chip/circuit to perform Image capture operation.

The processor 150 is coupled to the storage 110 and the image capturing device 130, and may be a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphic Processing Unit, GPU), or other programmable general purpose Or special purpose microprocessor (Microprocessor), digital signal processor (Digital Signal Processor, DSP), programmable controller, special application integrated circuit (Application Specific Integrated Circuit, ASIC) or other similar components or the above components combination. In the embodiment of the present invention, the processor 150 is used to perform all operations of the interpupillary distance computing device 100, and can load and execute various software modules, files, and data recorded in the storage 110.

In order to facilitate understanding of the operation flow of the embodiment of the present invention, the following will describe the operation flow of the interpupillary distance computing device 100 in the embodiment of the present invention in detail with many embodiments. Hereinafter, the methods described in the embodiments of the present invention will be described in conjunction with various components and devices of the interpupillary distance computing device 100. The various processes of the method can be adjusted according to the implementation situation, and it is not limited to this.

FIG. 2 is a flowchart illustrating an interpupillary distance calculation method according to an embodiment of the invention. Referring to FIG. 2, the image access module 111 directly obtains the image captured by the image capturing device 130 from the storage 110 or directly (step S210). It is worth noting that this image includes barcodes and faces. The barcode may be a one-dimensional or two-dimensional barcode (for example, Universal Product Code (UPC), QR code, PDF417 barcode, etc.). The human face is the human face.

The barcode reading module 112 then reads the actual length of the object included in the barcode from this image (step S230). In this embodiment, the data included in the barcode design includes the actual length of the object. This barcode may be printed on paper, cards, HMD helmets, anti-fouling eye masks for helmets and other items. The actual length of an object is the length measured by a ruler or other length measuring instruments in real life on objects that are used as scales (for example, barcodes, cards, mobile phones, and carriers that print barcodes, etc.). The barcode reading module 112 can perform preprocessing on the image (for example, reading color information, converting grayscale, binarization, edge detection, etc.), barcode positioning and barcode recognition, or other barcode recognition algorithms to Read the information included in the barcode.

In one embodiment, the barcode can be transmitted through a screen (eg, Liquid Crystal Display (LCD), light emitting diode (LED), etc.) on an electronic device (eg, smartphone, tablet, etc.) Display), at this time, the barcode will additionally carry the pixel density of the screen (or pixels per inch (PPI)), and the actual length of the object will be the length of the screen display barcode at the specified pixel density (for example, 300PPI The actual length of the barcode is displayed on the screen). The barcode reading module 112 reads the pixel density included in the barcode from the image.

In another embodiment, the barcode further includes a guide link. This navigation link can be a link to a website, a specific application ID, or a file path. After the barcode reading module 112 reads the guidance link included in the barcode from the image, the operation guidance module 114 connects to a designated website, calls a designated program, or opens a specific file according to the guidance link to activate the guidance screen . The guide screen is used to indicate the process of measuring the interpupillary distance (for example, instructing the shooting of images, warning the face to face the angle, and presenting the actual interpupillary distance). In some embodiments, these designated websites, programs, or files are executed by the processor 150 of the embodiment of the present invention and used to perform the pupillary distance calculation method of the embodiment of the present invention.

。Next, the image recognition module 113 recognizes the object and the face corresponding to the actual length of the object from the image, and calculates the frame length of the object and the pupil distance in the image (step S250). Specifically, before recognition, the image recognition module 113 first obtains several training images, and each training image includes a human face, and different training images may have different human faces. The image recognition module 113 obtains image information (such as hue, length, position, etc.) of key facial features such as eyes, nose, mouth, ears, and eyebrows in the human face, and uses machine learning techniques of artificial intelligence algorithms ( For example, artificial neural network (Artificial Neural Network, ANN), decision tree, support vector machine (Support Vector Machine, SVM, etc.) establishes the comparison model. Afterwards, the image recognition module 113 can input the image obtained in step S210 into the comparison model, and search and match the feature points and their neighborhood positions to adjust the position of the key points of the face to determine from the image Binocular features, thus completing feature point detection. After detecting the binocular features, the image recognition module 113 can extract the position information of the binocular features, and obtain the pixel positions Le and Re of the left and right eyes according to the position information of the binocular features (may be the pupil center point and pupil The position of the left outer edge or the right outer edge of the pupil) or the relative position with other feature points, and then calculate the screen length of the interpupillary distance

.

It should be noted that, in other embodiments, the image recognition module 113 may also use other image recognition algorithms other than the artificial intelligence algorithm to recognize the binocular features. In addition, the image recognition module 113 may simultaneously detect key facial features such as eyebrows and corners of the mouth. The image recognition module 113 determines whether the user's head is facing the image capturing device 130 or whether the eyes are facing the image capturing device 130 according to these key facial features. If the position and angle of the head or the eye orientation in the image do not meet the preset conditions, the operation guide module 114 will prompt the user through the guide screen or the prompt sound to ensure that the user is looking directly at the image capturing device 130, thereby Reduce the error of interpupillary distance estimation.

On the other hand, for object recognition, if the object uses a barcode, the image recognition module 113 can learn the position of the edge pixels of the barcode during the reading of the barcode in step S230, and then obtain the distance between the edges of the barcode on both sides as a The screen length of the barcode. If the object is not a barcode (for example, a card, helmet, mobile phone, etc., the object must be included in the image at this time), the image recognition module 113 can create a comparison model through the aforementioned eye recognition method, and use this comparison model to determine the object The position information of the edge on both sides or the outermost edge of the feature is used to calculate the screen length of the object. Alternatively, the image recognition module 113 may use other image recognition algorithms to recognize objects.

…(1) 實際瞳距為IPD，物件實際長度為Dc，物件的畫面長度為Pc，畫面比例即為Dc/Pc，瞳距的畫面長度為Pe。Next, the interpupillary distance calculation module 115 may calculate the actual interpupillary distance according to the object and the screen length of the interpupillary distance and the actual length of the object (step S270). In this embodiment, the interpupillary distance calculation module 115 calculates the screen ratio between the screen length of the object and the screen length of the interpupillary distance, and calculates the actual interpupillary distance according to the screen ratio. Equation 1 to calculate the actual pupil distance is:

…(1) The actual interpupillary distance is IPD, the actual length of the object is Dc, the screen length of the object is Pc, the screen ratio is Dc/Pc, and the screen length of the interpupillary distance is Pe.

For example, if a barcode is used as a scale object, the actual length of the object is the actual length of the barcode; the screen length of the object is the distance between the edges of the barcode on the opposite sides of the screen; the interpupillary distance calculation module 115 substitutes the aforementioned value into Equation 1. Or, if the card printed with barcodes is used as a scale object, the actual length of the object is the actual length of the card; the screen length of the object is the distance between the edges of the card on the opposite sides of the screen; the interpupillary distance calculation module 115 substitutes the aforementioned value Equation 1.

It is worth noting that in the embodiment where the barcode is presented through the screen of the electronic device, the actual length of the object carried by the barcode may be the actual length of the barcode. However, different screens support different resolutions and sizes, so that the actual length of the barcode may be different from the actual length of the barcode displayed on the screen. The pupil distance computing module 115 further adjusts the actual length of the object according to the ratio between the pixel density carried by the barcode and the pixel density corresponding to the actual length of the object. The pupil distance computing module 115 converts the adjusted actual length of the object The true length of the screen) is substituted into Equation 1.

In another embodiment, the actual length of the object carried by the barcode can be the screen resolution, then the image recognition module 113 will estimate the screen length of the screen in the image, and the interpupillary distance calculation module 115 depends on the pixel density and the screen resolution After calculating the actual size of the screen, it can be substituted into Equation 1 in combination with the screen length of the screen.

Then, the operation guidance module 114 can present the actual pupil distance through the display or play through the speaker, allowing the user to set the pupil distance setting of the HMD. Or, the processor 150 transmits the actual interpupillary distance to the HMD for the HMD to adjust the interpupillary distance setting by itself. It should be noted that in the foregoing multiple embodiments, the use of a barcode as a scale object is the preferred embodiment, and the user only needs to use a carrier that prints or presents the barcode.

To help the reader understand the operation process of the embodiment of the present invention, the following two application scenarios are described.

3A-3D are schematic diagrams of an interpupillary distance measurement process according to an embodiment of the invention. Please refer to FIG. 3A first. In this embodiment, the smartphone 200 is an implementation of the pupil distance computing device 100, and the image capturing device 130 of the smartphone 200 is a front lens, and the smartphone 200 is additionally provided with The display 170 displays a guide screen. The barcode QR1 is a QR code, and the actual length of the object with the barcode QR1 (that is, the object as a scale is a barcode). This barcode QR1 can be printed on the card C or the anti-fouling eye mask M used in the HMD helmet. The operation guide module 114 can instruct the user to take the card C to a position where the image capture device 130 can take a picture on the guide screen through the display 170 or wear the anti-fouling eye mask M and point the head toward the image capture device 130. Referring to FIG. 3B, the guide screen may present the image S captured by the image capturing device 130 in real time, so as to facilitate the user to confirm that the barcode QR1 is captured by the image capturing device 130. The barcode reading module 112 reads the actual length of the object contained in the barcode from the image S acquired by the image capturing device 130 and the guide link. The processor 150 records the actual length of the object and connects it to the guide link. Website. The webpage presented by the linked website will instruct the user to face the image capturing device 130 (face F shown in FIG. 3B). Referring to FIG. 3C, the image recognition module 113 will recognize the eye position e from the image S, and calculate the frame length of the interpupillary distance accordingly. In addition, the image recognition module 113 also calculates the screen length of the barcode QR1 on the image S. The interpupillary distance calculation module 115 can then substitute the actual length of the object of the barcode QR1, the screen length of the barcode QR1, and the screen length of the interpupillary distance into Formula 1, to calculate the actual interpupillary distance (for example, 68 millimeters (mm)). The operation guidance module 114 may present the actual interpupillary distance to notify N. Please refer to FIG. 3D, this actual pupil distance can help the user to set the pupil distance setting of the head mounted display HMD.

4A-4D are schematic diagrams of an interpupillary distance measurement process according to another embodiment of the invention. Please refer to FIGS. 4A and 4B first. In this embodiment, the notebook computer 300 is an implementation of the pupil distance computing device 100, and the image capturing device 130 of the notebook computer 300 is a video camera, and the notebook computer 300 is provided separately. There is a display 190 to present a guide picture. The smartphone P presents a barcode QR2 (as shown in FIG. 4B) through its display D. The barcode QR2 is a QR code with the actual length of the object represented by the barcode QR2 at a pixel density of a five-inch and 1080P resolution display. The operation guidance module 114 can instruct the user to take the smartphone P to a position where the image capturing device 130 can shoot on the guidance screen through the display 190. The guide screen may present the image S2 captured by the image capturing device 130 in real time, so as to facilitate the user to confirm that the barcode QR2 is captured by the image capturing device 130. The barcode reading module 112 reads the actual length of the object carried by the barcode from the image S2 obtained by the image capturing device 130 and the guide link. The processor 150 will record the actual length of the object and open the program specified by the guide link . The guidance screen presented by the guidance linking program will instruct the user to face the image capturing device 130 (face F2 shown in FIG. 3B). Referring to FIG. 4C, the image recognition module 113 will recognize the eye position e2 from the image S2, and calculate the frame length of the interpupillary distance accordingly. In addition, the image recognition module 113 also calculates the screen length of the barcode QR2 on the image S2. The interpupillary distance calculation module 115 can then calculate the actual interpupillary distance (for example, 68 millimeters (mm)) according to the actual length of the object of the barcode QR2, the length of the barcode QR2, and the length of the pupil distance. The operation guidance module 114 can present the actual interpupillary distance to notify N2. Referring to FIG. 4D, the interpupillary distance computing device 100 transmits this actual interpupillary distance to the head-mounted display HMD, and the head-mounted display HMD can adjust the interpupillary distance setting accordingly.

It should be noted that, in another application scenario, the user can also hold a printed barcode carrier and take a self-portrait through the camera, and then record the self-portrait photo through the network or write the file in the interpupillary distance computing device 100 Then, the pupil distance computing device 100 can calculate the actual pupil distance based on the self-photograph.

In summary, the embodiments of the present invention recognize the barcode and binocular features presented by the vehicle through image recognition technology, and enclose the actual length of the information through the barcode, so that the user only needs to use the barcode on the face and the vehicle. When the object as a scale is oriented toward the image capturing device, the actual interpupillary distance can be obtained. The interpupillary distance measurement process of the embodiment of the present invention is quite simple and convenient. Users can also print the barcode on any convenient carrier or display it on the screen, without the need to manually enter the length of the carrier, nor need to limit it to credit cards and other objects with suspicion of leakage.

Although the present invention has been disclosed as above with examples, it is not intended to limit the present invention. Any person with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to the scope defined in the appended patent application.

100‧‧‧Pupillary distance computing device 110‧‧‧Storage 111‧‧‧Image access module 112‧‧‧ Barcode reading module 113‧‧‧Image recognition module 114‧‧‧Operation guide module 115 ‧‧‧Pupillary distance calculation module 130‧‧‧Image capture device 150‧‧‧ processor 170, 190, D‧‧‧ display 200, P‧‧‧smartphone 300‧‧‧note PC C‧‧ ‧Card M‧‧‧Anti-fouling eye mask QR1, QR2‧‧‧Barcode N, N2‧‧‧Notification S, S2‧‧‧Image e, e2‧‧‧Eye F, F2‧‧‧Face HMD‧‧‧Head S210~S270‧‧‧Steps

FIG. 1 is a block diagram of components of an interpupillary distance computing device according to an embodiment of the invention. 2 is a flowchart of an interpupillary distance calculation method according to an embodiment of the invention. 3A~3D are schematic diagrams of an interpupillary distance measurement process according to an embodiment of the invention. 4A~4D are schematic diagrams of a pupillary distance measurement process according to another embodiment of the invention.

S210~S270‧‧‧Step

Claims (10)

An interpupillary distance (IPD) calculation method includes: obtaining an image, wherein the image includes a barcode and a human face; reading the actual length of an object included in the barcode from the image; identifying the image from the image An object corresponding to the actual length of the object and the face, and calculating the frame length of the pupil distance in the image and the object; and calculating an actual pupil based on the object and the screen length of the pupil distance and the actual length distance. The interpupillary distance calculation method as described in item 1 of the patent application, wherein the step of calculating the actual interpupillary distance according to the object and the screen length of the interpupillary distance and the actual length of the object includes: calculating the screen length of the object and the pupil The picture ratio between the picture lengths of the distance; and the actual pupil distance is calculated according to the picture ratio. The interpupillary distance calculation method as described in item 1 of the patent scope, wherein the object is the barcode, and the step of reading the actual length of the object included in the barcode from the image includes: reading the barcode from the image Included is a pixel density, where the actual length of the object is the length of the second pixel density screen presenting the barcode; and the actual length of the object is adjusted according to the ratio between the pixel density and the second pixel density. The interpupillary distance calculation method as described in item 1 of the patent scope, wherein the steps of recognizing the object and the human face from the image and calculating the frame length of the barcode and the interpupillary distance in the image of the image include: The artificial intelligence algorithm determines a binocular feature from the image; and calculates the frame length of the interpupillary distance based on the position information of the binocular feature. The pupil distance calculation method as described in item 1 of the patent application scope, wherein the step of reading the actual length of the object included in the barcode from the image includes: reading a guide link included in the barcode from the image ; And starting a guidance screen according to the guidance link, wherein the guidance screen is used to indicate the interpupillary distance measurement process. An interpupillary distance computing device includes: an image capturing device that captures an image, wherein the image includes a code and a human face; a memory that records the image and multiple modules; and a processor coupled to the An image capturing device and the storage, and access and load the modules recorded in the storage, and the modules include: an image access module to obtain the image; a barcode reading module , Read the actual length of an object included in the barcode from the image; an image recognition module, identify the object and the face corresponding to the actual length of the object from the image, and calculate the object and the person in the image A picture length of a pupil distance in the face; and a pupil distance calculation module, calculating an actual pupil distance based on the object and the picture length of the pupil distance and the actual length of the object. The pupil distance computing device as described in item 6 of the patent application range, wherein the pupil distance computing module calculates the picture ratio between the picture length of the object and the picture length of the pupil distance, and calculates the actual pupil according to the picture ratio distance. The interpupillary distance computing device as described in item 6 of the patent application range, wherein the barcode reading module reads a pixel density included in the barcode from the image, the object is the barcode, and the actual length of the object is a The screen of the second pixel density presents the length of the barcode, and the pupil distance calculation module adjusts the actual length of the object according to the ratio between the pixel density and the second pixel density. An interpupillary distance computing device as described in item 6 of the patent scope, wherein the image recognition module determines a binocular feature from the image through an artificial intelligence algorithm, and calculates the interpupillary distance based on the position information of the binocular feature Picture length. The pupil distance computing device as described in item 6 of the patent application scope, wherein the barcode reading module reads a guide link included in the barcode from the image, and the modules further include: an operation guide The module activates a guide screen according to the guide link, and presents the actual interpupillary distance through the guide screen, wherein the guide screen is used to indicate the interpupillary distance measurement process.

Images