TWI512641B - Identification system and identification method - Google Patents

Description

Identification system and identification method

The present invention relates to an identification system, and more particularly to a system and method for identifying and searching for information about a person or character in a movie.

With the advancement of multimedia technology, the playback technology of movies has become more and more popular. Users can easily and conveniently watch videos without using the electronic equipment such as tablet computers and smart phones. However, when the user wants to inquire about the information of the person or character in the movie, there is still much room for improvement in the existing technology.

For example, the user may need to pause the movie on a certain screen, and then capture images of the person or character of interest in the screen, and use the identification software or image search server to search for relevant information. . Moreover, the existing technology in the search for information about people or characters, the probability of its search success needs to be strengthened.

Accordingly, one aspect of the present invention is to provide an identification system including a storage server, at least one search server, and an identification device. The identification device includes a detection module, an image processing module, and an execution module. The detection module is configured to perform face detection on the first image in the movie according to a user instruction to generate at least one facial image data. The image processing module is coupled to the detection module and communicates with the storage server. The image processing module is configured to obtain a weight value of the facial image data according to the weight calculation formula, and select and transmit the facial image data to the storage server according to the weight value in the facial image data, and store the server backhaul. Corresponding to the storage URL of the storage location where the facial image data is to be recognized. The execution module communication connection stores the server and the search server. The execution module is configured to receive the storage URL and transmit the storage URL to the search server, so that the search server reads the facial image data to be recognized according to the storage URL, and generates and returns the target information search according to the facial image data to be recognized. As a result, the execution module outputs the object related information based on the object information search result.

According to an embodiment of the invention, the execution module is configured to transmit the storage URL to the plurality of search servers, and determine whether the plurality of object information search results returned by the search server include the image data corresponding to the facial image to be recognized. Object related information.

According to another embodiment of the present invention, the execution module outputs object related information through a browser.

According to still another embodiment of the present invention, the detecting module is further configured to calculate a corresponding face detection confidence value for the facial image data.

According to still another embodiment of the present invention, the identification device further includes an image. The optimization module is configured to determine whether the face detection confidence value is less than a threshold value, and if so, the detection module is further configured to perform face detection on at least one second image in the movie to generate a facial image Face image data with similar data and higher facial detection confidence value to replace facial image data.

According to still another embodiment of the present invention, the weight calculation formula is determined by the following mathematical formula: W = D × C × B × dmin, where W is a weight value; D is a distance between two eyes in the face image data; The confidence value is detected for the face; B is the brightness value of the facial image data; and dmin is the smallest of the distance between the middle point of the two eyes and the five fixed points on the facial image data in the facial image data, wherein The five fixed points are the intersection of the center point of the facial image data and the well-word line of the equal-part image on the facial image data.

According to still another embodiment of the present invention, the image processing module is configured to select facial image data having the highest weight value as the facial image data to be recognized in the facial image data.

According to still another embodiment of the present invention, the image processing module is configured to output the facial image data to the display component according to the weight value, and select the facial image data to be recognized from the facial image data according to the user selection instruction. .

Another aspect of the present invention provides an identification method comprising the steps of: (a) performing face detection on a first image in a movie according to a user instruction to generate at least one facial image data; (b) Obtaining a weight value of the facial image data according to the weight calculation formula; (c) selecting and storing the facial image data to be recognized in the facial image data according to the weight value; (d) according to the facial image data corresponding to the facial image to be recognized Storage address of the storage location, read to identify Face image data; (e) generating object information search results according to the facial image data to be recognized; and (f) outputting object related information according to the object information search result.

According to an embodiment of the invention, the identifying method further comprises: determining whether the target information search result includes object related information corresponding to the facial image data to be recognized, and if not, repeating step (d) and step (e).

According to another embodiment of the present invention, the identification method further includes: calculating a corresponding face detection confidence value for the facial image data.

According to still another embodiment of the present invention, the identifying method further includes: determining whether the face detection confidence value is less than a threshold; and if so, performing face detection on the at least one second image in the movie to generate the at least Face image data with similar facial image data and higher facial detection confidence value replaces facial image data.

According to still another embodiment of the present invention, the weight calculation formula is determined by the following mathematical formula: W = D × C × B × dmin, where W is a weight value; D is a distance between two eyes in the face image data; C is Face detection confidence value; B is the brightness value of the facial image data; and dmin is the smallest of the distance between the middle point of the two eyes and the five fixed points on the facial image data in the facial image data, five of which The fixed points are the intersection of the center point of the facial image data and the well-word line of the equal-part image on the facial image data.

According to still another embodiment of the present invention, the step (c) further includes: selecting facial image data having the highest weight value as the facial image data to be recognized in the facial image data.

According to still another embodiment of the present invention, the step (c) further includes: The facial image data is output to the display component according to the weight value, and the facial image data to be recognized is selected from the facial image data according to the user selection instruction.

The advantage of applying the invention is that the user can query the relevant information of the person or character in the movie by simply issuing an instruction. In addition, the system automatically recognizes and determines the characters or characters that the user may be interested in. The user does not need to pause the movie or manually select the person or character to be queried. In addition, by using a plurality of search servers for searching, the probability that the object related information is successfully searched can be increased.

100, 100a‧‧‧ identification system

110, 110a‧‧‧ Identification device

120, 120a‧‧‧Detection module

122, 122a, 222‧‧‧ Face image data

130‧‧‧Image Processing Module

132‧‧‧To identify facial image data

140‧‧‧Execution module

150‧‧‧Storage server

152‧‧‧Save URL

160‧‧‧Search server

162‧‧‧Target information search results

End points of 170, 174, 180, 182, 184, 186, 188‧‧

190, 192, 194, 196, 198‧‧ ‧ line segments

210‧‧‧Image Optimization Module

302, 304, 306, 308, 310, 312, 402, 502, 602, 604‧ ‧ steps

FIG. 1 is a block diagram of an identification system according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a facial image data according to an embodiment of the invention.

FIG. 3 is a block diagram of an identification system according to an embodiment of the present invention.

FIG. 4 is a schematic flow chart of an identification method according to an embodiment of the present invention.

FIG. 5 is a schematic flow chart of an identification method according to an embodiment of the present invention.

FIG. 6 is a schematic flow chart of an identification method according to an embodiment of the present invention.

FIG. 7 is a schematic flow chart of an identification method according to an embodiment of the present invention.

The embodiments are described in detail below with reference to the accompanying drawings, but the embodiments are not intended to limit the scope of the invention, and the description of the structure operation is not intended to limit the order of execution, any component recombination The structure, which produces equal devices, is within the scope of the present invention. In addition, the drawings are for illustrative purposes only and are not drawn to the original dimensions. For ease of understanding, the same elements in the following description will be denoted by the same reference numerals.

In this document, "one" and "the" can be used to mean one or more, unless the article specifically defines the article. It will be further understood that the terms "comprising", "comprising", "having", and <RTIgt; One or more of its other features, regions, integers, steps, operations, elements, components, and/or groups thereof.

In addition, the words "first, second, third, etc." are used herein to describe various elements, components, regions, layers and/or blocks. However, these elements, components, regions, layers and/or blocks should not be limited by these terms. These terms are only used to identify a single element, component, region, layer, and/or block. Thus, a singular element, component, region, layer and/or block may be referred to as a second element, component, region, layer and/or block, without departing from the spirit of the invention.

Please refer to Figure 1. 1 is a block diagram of an identification system 100 in accordance with an embodiment of the present invention. The identification system 100 includes a storage server 150, at least one search server 160, and an identification device 110.

The identification system 100 can be used to identify and search for information about a person or character in a movie and provide it to the user. In one embodiment, the storage server 150 and the search server 160 are connected to the network and the identification device. The identification device 110 can be a personal computer, a notebook computer, a tablet computer, a smart phone, a multimedia player or a smart glasses.

The identification device 110 includes a detection module 120, an image processing module 130, and an execution module 140. The image processing module 130 is coupled to the detection module 120 and communicatively coupled to the storage server 150. The execution module 140 communicatively connects the storage server 150 with the search server 160. In one embodiment, the detection module 120 is a face detection chip, the image processing module 130 is an image processing chip, and the execution module 140 is a system wafer. In another embodiment, the identification device 110 includes at least one processor and a memory, and the detection module 120, the image processing module 130, and the execution module 140 are stored in the memory, and the processor is Perform its functions.

The detection module 120 is configured to perform face detection on a first image of a movie according to a user instruction to generate at least one facial image data 122. The user command can be a command issued by the user through a keyboard, a mouse or a remote controller, or a voice command. The facial image data 122 can be a facial image of a human face. In one embodiment, the detection module 120 is further configured to calculate a corresponding facial detection confidence value for the facial image data 122. The face detection confidence value can be used as a measure of whether the detected facial image data 122 does include a reliability indicator of the face.

The image processing module 130 is configured to obtain a weight value of the facial image data 122 according to a weight calculation formula. Please refer to FIG. 2 at the same time. FIG. 2 is a schematic diagram showing a facial image data 122a according to an embodiment of the present invention. In this embodiment, the weight value of the facial image data 122a is calculated. The weight calculation formula is determined by the following mathematical formula: W = D × C × B × dmin, where W is the weight value; D is the distance between the two eyes in the facial image data 122a (that is, the length of the line segment 170 in the second figure) C is the facial detection confidence value of the facial image data 122a; B is the luminance value of the facial image data 122a; and dmin is the intermediate point between the two eyes in the facial image data 122a (ie, the second image) The smallest of the distance between the endpoint 174) and the five fixed points on the facial image data 122a. The above five fixed points are respectively the intersection point of the center point of the facial image data 122a (ie, the end point 180 in FIG. 2) and the ticks of the halved picture on the facial image data 122a (ie, in FIG. 2) End points 182, 184, 186, and 188). In the present embodiment, the distance between the intermediate point 174 between the two eyes and the above five fixed points 180, 182, 184, 186 and 188 is the length of the line segments 190, 192, 194, 196 and 198 in Fig. 2, respectively. As can be seen from Fig. 2, in the present embodiment, dmin is the length of the line segment 194.

The image processing module 130 is further configured to select and transmit a facial image data 132 to the storage server 150 in the facial image data 122 according to the weight value, so that the storage server 150 returns the image data corresponding to the facial image to be recognized. A storage URL 152 of the storage location of 132.

In one embodiment, the image processing module 130 is configured to select a facial image data having the highest weight value from the facial image data 122 as the facial image data 132 to be recognized. Here, the face image data having the highest weight value may refer to the image data occupying the largest proportion of the face image in the picture, and may also refer to the face image data having the highest weight value after being calculated by the weight calculation formula.

In another embodiment, the image processing module 130 is configured to output the facial image data 122 to a display component according to the weight value, and select the facial surface to be recognized from the facial image data 122 according to a user selection command. Image data 132.

The execution module 140 is configured to receive the storage URL 152 and transmit the storage URL 152 to the search server 160, so that the search server 160 reads the face to be recognized from the storage location where the facial image data 132 is to be recognized according to the storage URL 152. The image data 132 is generated and a target information search result 162 is generated and returned according to the facial image data 132 to be recognized. The execution module 140 is further configured to output an object related information according to the object information search result 162.

In one embodiment, the search server 160 searches for information related to the facial image data 132 to be recognized on a database or the Internet according to the facial image data 132 to be recognized, for example, the facial image data to be recognized 132. The name, age, gender, photo, marital status, nationality, place of residence, occupation, etc. of the person in the face of the face are used, and the searched information is used as the target information search result 162.

In another embodiment, the execution module 140 outputs the object related information on the display component through a browser (for example, Internet Explorer, Firefox, Chrome, or Safari).

In still another embodiment, the execution module 140 is configured to transmit the storage URL 152 to the plurality of search servers 160, and determine whether the plurality of object information search results 162 returned by the search server 160 include the face to be recognized. Information about the object of the image data 132.

In this embodiment, the execution module 140 first transmits the storage URL 152. And sending to a first search server, and determining whether the first object information search result returned by the first search server includes object related information corresponding to the facial image data 132 to be recognized. If so, the execution module 140 outputs object related information corresponding to the facial image data 132 to be recognized included in the first object information search result. If not, the execution module 140 transmits the storage URL 152 to a second search server, and determines whether the second target information search result returned by the second search server includes the image data corresponding to the facial image to be recognized. 132 object related information. If yes, the execution module 140 outputs object related information corresponding to the facial image data 132 to be recognized included in the second object information search result. If not, the execution module 140 transmits the storage web address 152 to a third search server and repeats the above operations. In this embodiment, by using a plurality of search servers to sequentially search for object related information of the facial image data 132 to be recognized, the probability that the object related information is successfully searched can be increased.

Please refer to Figure 3. FIG. 3 is a block diagram of an identification system 100a according to an embodiment of the present invention. Compared with the identification system 100 shown in FIG. 1 , in the embodiment, the identification device 110 a further includes an image optimization module 210 . In one embodiment, the image optimization module 210 is an image processing wafer. In another embodiment, the identification device 110a includes at least one processor and a memory, and the image optimization module 210 is stored in the memory and performs its functions by the processor. The detection module 120a can be the detection module 120 shown in FIG. 1 , and its function and operation are similar, and therefore will not be described herein.

The image optimization module 210 is configured to determine that the facial image data 122 is correct. Whether the face detection confidence value is less than a critical value. If yes, the detecting module 120a is further configured to perform face detection on at least one second image in the movie to generate a face similar to the facial image data 122 and having a higher facial detection confidence value. The image data 222 is substituted for the facial image data 122. In this embodiment, the face image data with a lower face detection confidence value is replaced with a similar face image data with a higher face detection confidence value, so that the face image data does include the face. The probability is higher, and thus the reliability of the information related to the objects found by the identification system is improved.

Please refer to Figure 4. FIG. 4 is a flow chart showing an identification method according to an embodiment of the invention. The identification method can be implemented as a computer program product (such as an application) and stored in a computer readable recording medium, and the computer can perform the identification method after reading the recording medium. The computer readable recording medium can be a read-only memory, a flash memory, a hard disk, a compact disc, a flash drive, a magnetic tape, a database accessible by the network, or a computer familiar to the skilled person who can easily think of the same function. The recording medium can be read.

This identification method can be applied to the identification system 100 as shown in FIG. 1 , but is not limited thereto. For convenience and clarity of description, the following description of the identification method is described in conjunction with the identification system 100 shown in FIG.

In step 302, the detection module 120 performs face detection on the first image in the movie according to the user instruction to generate at least one facial image data 122.

Then, in step 304, the image processing module 130 obtains the weight value of the facial image data 122 according to the weight calculation formula.

Next, in step 306, the image processing module 130 is based on the weight For example, the facial image data 132 is selected from the facial image data 122, and the facial image data 132 to be recognized is transmitted to the storage server 150 for storage.

In step 308, the search server 160 reads the facial image data 132 to be recognized according to the storage URL 152 corresponding to the storage location of the facial image data 132 to be recognized by the execution module 140.

Then, in step 310, the search server 160 generates an object information search result 162 based on the facial image data 132 to be recognized.

Next, in step 312, the execution module 140 outputs the object related information according to the object information search result 162 returned by the search server 160.

Please refer to Figure 5. FIG. 5 is a schematic flow chart of an identification method according to an embodiment of the invention. Compared with the identification method shown in FIG. 4, in the embodiment, the identification method further includes step 402. This identification method can be applied to the identification system 100 as shown in FIG. 1 , but is not limited thereto. For convenience and clarity of description, the following description of the identification method is described in conjunction with the identification system 100 shown in FIG.

In step 402, the execution module 140 determines whether the target information search result 162 includes object related information corresponding to the facial image data 132 to be recognized. If no, step 308 and step 310 are repeated.

In this embodiment, by repeatedly searching for the object related information of the facial image data 132 to be recognized, the probability that the object related information is successfully searched can be increased.

Please refer to Figure 6. FIG. 6 is a flow chart showing an identification method according to an embodiment of the invention. Compared to the identification shown in Figure 4 In this embodiment, the identification method further includes step 502. This identification method can be applied to the identification system 100 as shown in FIG. 1 , but is not limited thereto. For convenience and clarity of description, the following description of the identification method is described in conjunction with the identification system 100 shown in FIG.

In step 502, the detection module 120 calculates a corresponding face detection confidence value for the facial image data 122. The face detection confidence value can be used as a measure of whether the detected facial image data 122 does include a reliability indicator of the face.

Please refer to Figure 7. FIG. 7 is a flow chart showing an identification method according to an embodiment of the invention. Compared with the identification method shown in FIG. 6, in the embodiment, the identification method further includes step 602 and step 604. This identification method can be applied to the identification system 100a as shown in FIG. 3, but is not limited thereto. For convenience and clarity of description, the following description of the identification method is described in conjunction with the identification system 100a shown in FIG.

In step 602, the image optimization module 210 determines whether the face detection confidence value corresponding to the facial image data 122a is less than a critical value.

If the result of the above determination is yes, then in step 604, the detection module 120a performs face detection on at least one second image in the movie to generate a higher facial similar to the facial image data 122a. The face image data of the confidence value is detected instead of the face image data 122a.

In this embodiment, the face image data with a lower face detection confidence value is replaced with a similar face image data with a higher face detection confidence value, so that the face image data does include the face. The probability is higher, and thus the reliability of the information about the object being searched is improved.

It should be understood that the steps mentioned in the present embodiment can be adjusted according to actual needs, and can be performed simultaneously or partially simultaneously, unless the order is specifically stated.

In summary, with the technical means of the present invention, the user can query the related information of the person or character in the movie by simply issuing an instruction. In addition, the system automatically recognizes and determines the characters or characters that the user may be interested in. The user does not need to pause the movie or manually select the person or character to be queried. In addition, when the identification device needs to use the search server to search for object information, the identification device does not need to upload the image data to be identified to the search server. The identification device only needs to provide the storage URL generated by the storage server to the search server, and the search server can search for the object information. Since some search servers may not be able to provide direct uploading of data services, or users may not be able to directly upload files, the search server may use the storage URL to read facial image data to perform object information search. . In addition, by using a plurality of search servers for searching, the probability that the object related information is successfully searched can be increased.

The present disclosure has been disclosed in the above embodiments, but it is not intended to limit the disclosure, and any person skilled in the art can make various changes and refinements without departing from the spirit and scope of the disclosure. The scope of protection of the disclosure is subject to the definition of the scope of the patent application.

100‧‧‧ Identification System

110‧‧‧ Identification device

120‧‧‧Detection module

122‧‧‧Face image data

130‧‧‧Image Processing Module

132‧‧‧To identify facial image data

140‧‧‧Execution module

150‧‧‧Storage server

152‧‧‧Save URL

160‧‧‧Search server

162‧‧‧Target information search results

Claims (15)

An identification system comprising: a storage server; at least one search server for searching for information related to the facial image data to be recognized in a database or an internet network according to a facial image data to be recognized; And an identification device comprising: a detection module configured to perform face detection on a first image of a movie according to a user command to generate at least one facial image data; The module is coupled to the detection module and communicatively coupled to the storage server, and the image processing module is configured to obtain a weight value of the at least one facial image data according to a weight calculation formula, and according to the weight value Selecting and transmitting the image data to be recognized to the storage server in the at least one facial image data, and the storage server returns a storage URL corresponding to the storage location of the facial image data to be recognized; and executing a module that communicates with the storage server and the at least one search server, the execution module is configured to receive the storage URL, and transmit the storage URL to the at least one search The server, the at least one search server reads the image data to be recognized according to the storage URL, and generates and returns an object information search result according to the image data to be recognized, the execution module according to the object The information search result outputs an object related information. The identification system of claim 1, wherein the execution module is configured to transmit the storage URL to a plurality of search servers, and determine whether the plurality of object information search results returned by the search servers include The information related to the object for which the facial image data is to be recognized. The identification system of claim 1, wherein the execution module outputs the object related information through a browser. The identification system of claim 1, wherein the detecting module is further configured to calculate a corresponding facial detection confidence value for the at least one facial image data. The identification system of claim 4, wherein the identification device further comprises: an image optimization module, configured to determine whether the face detection confidence value is less than a threshold value, and if so, the detection module is further used Performing face detection on at least one second image in the movie to generate a facial image data similar to the at least one facial image data and having a higher facial detection confidence value instead of the at least one face Department of video data. The identification system of claim 4, wherein the weight calculation formula is determined by the following mathematical formula: W = D × C × B × dmin, where W is the weight value; D is between the two eyes of the facial image data of Distance; C is the confidence value of the face detection; B is the brightness value of the facial image data; and dmin is the intermediate point between the two eyes and the five fixed points on the facial image data in the facial image data. The smallest of the distances, wherein the five fixed points are the intersection of the center point of the facial image data and the well-word connection of the aliquoted image on the facial image data. The identification system of claim 1, wherein the image processing module is configured to select a facial image data having the highest weight value as the facial image data to be recognized in the at least one facial image data. The identification system of claim 1, wherein the image processing module is configured to output the at least one facial image data to a display component according to a weight value, and output the facial image according to a user selection instruction. Select the facial image data to be recognized in the data. An identification method includes: (a) performing face detection on a first image of a movie according to a user instruction to generate at least one facial image data; and (b) obtaining the at least one weight calculation formula a weight value of one of the facial image data; (c) selecting and storing a facial image data to be recognized in the at least one facial image data according to the weighting value; (d) according to the corresponding facial image a storage URL of the storage location of the image data, and reading the facial image data to be recognized; (e) searching for information related to the facial image data in a database or an internet network to generate an object information search result according to the image data to be recognized; and (f) searching for information based on the object information The result outputs an object related information. The identification method of claim 9, further comprising: determining whether the target information search result includes the object related information corresponding to the facial image data to be recognized, and if not, repeating step (d) and step (e) . The identification method of claim 9, further comprising: calculating a corresponding face detection confidence value for the at least one facial image data. The method of claim 11, further comprising: determining whether the face detection confidence value is less than a threshold; and if so, performing face detection on the at least one second image in the movie to generate a The facial image data similar to the at least one facial image data and having a higher facial detection confidence value is substituted for the at least one facial image data. The identification method according to claim 11, wherein the weight calculation formula is determined by the following mathematical formula: W=D×C×B×dmin, Where W is the weight value; D is the distance between the two eyes in the facial image data; C is the face detection confidence value; B is the brightness value of the facial image data; and dmin is the facial image data The smallest of the distance between the middle point of the two eyes and the five fixed points on the facial image data, wherein the five fixed points are respectively the center point of the facial image data and the upper part of the facial image data The intersection of the well word line. The identification method of claim 9, wherein the step (c) further comprises: selecting a facial image data having the highest weight value as the facial image data to be recognized in the at least one facial image data. The identification method of claim 9, wherein the step (c) further comprises: outputting the at least one facial image data to a display component according to the weight value, and extracting the facial image data according to a user selection instruction. Select this to identify facial image data.