TWI891171B - Object identification method and system and computer program product - Google Patents

Abstract

An object recognition method implemented by an object recognition system comprises: using an appearance feature matching model to analyze the appearance of an object to be recognized in a photographic result according to a feature data set to generate a feature recognition result corresponding to the object to be recognized; and using an optical character recognition model to recognize a key word presented on the object to be recognized in the photographic result. After the data is obtained, a text relevance analysis model is used to analyze the key text data based on a product name data set to generate a product name recognition result corresponding to the object to be identified; based on at least one of the feature recognition result and the product name recognition result, one of the multiple object name data is used as a target object name data, and the target object name data is output.

Description

Object identification method and system and computer program product

The present invention relates to an identification method, and more particularly to an object identification method for identifying objects in an image. The present invention also relates to an object identification system for identifying objects in an image, and a computer program product that enables an electronic device to identify objects in an image.

In recent years, image recognition technology has been widely used in object recognition, which is to use computer vision to identify the type of object.

One common approach in existing technologies is to allow computers to learn features such as the shape and outline of objects through machine learning technology, thereby achieving the function of identifying specific objects from images.

However, if a pile of objects actually contains multiple items with similar appearances (for example, mixing together multiple products of the same brand with similar packaging), existing technologies may have difficulty accurately identifying the type of each object. Therefore, how to improve existing technologies for identifying objects with similar appearances has become an issue worth exploring.

In order to improve the existing technology, one of the objectives of the present invention is to provide an object recognition method that helps to accurately identify multiple objects with similar appearances.

The object recognition method of the present invention is implemented by an object recognition system, which includes a processing unit and a storage unit. The storage unit stores a feature data set, a product name data set, an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities. The object identification method comprises: (A) the processing unit obtains a photographic result showing at least one object to be identified, and uses the appearance feature matching model to match the object name data with the feature data; The processing unit analyzes the appearance of the object to be identified in the photographic result to generate a feature recognition result corresponding to the object to be identified and related to the object name data; (B) when the processing unit uses the optical character recognition model to recognize a key text data presented on the object to be identified in the photographic result, the processing unit uses the text relevance analysis model to The key text data is analyzed based on the product name data set to generate a product name recognition result corresponding to the object to be recognized and related to the object name data; (C) the processing unit uses one of the object name data as a target object name data based on at least one of the feature recognition result and the product name recognition result, and outputs the target object name data.

In some embodiments of the object identification method of the present invention, in step (A), the processing unit first selects one or more matching object feature data with the highest degree of match with the appearance of the object to be identified from the object feature data, and then generates the feature identification result based on the object name data (or items) corresponding to the matching object feature data (or items). In step (B), the processing unit first selects one or more matching object name data with the highest degree of match with the keyword text data from the object name data, and then generates the item name identification result based on the matching object name data (or items).

In some embodiments of the object recognition method of the present invention, in step (A), the feature recognition result includes the object name data corresponding to the matching object feature data, and one or more confidence scores corresponding to the matching object feature data; in step (B), the product name recognition result includes the matching object name data, and one or more confidence scores corresponding to the matching object name data; in step (C), the processing unit determines The target object name data is determined by determining whether one of the object name data is included in the feature recognition result and the product name recognition result. If the determination result is yes, the one of the object name data is used as the target object name data. If the determination result is no, the object name data with the highest confidence score among the object name data included in the feature recognition result or the product name recognition result is used as the target object name data.

In some embodiments of the object recognition method of the present invention, in step (A), the processing unit selects a single matching object feature data that has the highest degree of matching with the appearance of the object to be recognized from the object feature data, and then generates the feature recognition result, and the feature recognition result includes the object name data corresponding to the matching object feature data; in step (B), the processing unit first uses the optical character recognition model to determine whether there is text information on the object to be recognized in the photographic result, and if the determination result is yes, obtains the key text data from the text information, and, in the case where the processing unit has recognized the key text data In the following, the processing unit first selects a single matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name recognition result, and the product name recognition result includes the matching object name data; in step (C), if the processing unit determines in step (B) that the text information exists on the object to be recognized, the processing unit uses the object name data included in the product name recognition result as the target object name data; if the processing unit determines in step (B) that the text information does not exist on the object to be recognized, the processing unit uses the object name data included in the feature recognition result as the target object name data.

In some embodiments of the object recognition method of the present invention, in step (B), the processing unit first uses the optical character recognition model to determine whether text information consisting of multiple characters exists on the object to be identified in the photographic result. If the determination result is yes, at least a portion of the text information is extracted from the text information as the key text data based on the size and/or position of the characters in the photographic result.

Another object of the present invention is to provide an object recognition system that helps to accurately identify multiple objects of similar appearance.

The object recognition system of the present invention includes a processing unit and a storage unit electrically connected to the processing unit. The storage unit stores a feature data set, a product name data set, an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities. The feature data set includes multiple pieces of object feature data corresponding to multiple object types, and the product name data set includes multiple pieces of object name data corresponding to the object types and corresponding to the object feature data. The processing unit is used to: obtain a photographic result showing at least one object to be identified; analyze the appearance of the object to be identified in the photographic result according to the feature data set using the appearance feature matching model to generate a feature recognition result corresponding to the object to be identified and related to the object name data; and recognize a key text displayed on the object to be identified in the photographic result using the optical character recognition model. In the case of word data, the processing unit uses the word relevance analysis model to analyze the key word data according to the product name data set to generate a product name recognition result corresponding to the object to be identified and related to the object name data; based on at least one of the feature recognition result and the product name recognition result, one of the object name data is used as a target object name data, and the target object name data is output.

In some embodiments of the object identification system of the present invention, the processing unit first selects one or more matching object feature data from the object feature data that has the highest degree of match with the appearance of the object to be identified, and then generates the feature identification result based on the object name data corresponding to the matching object feature data. In addition, the processing unit first selects one or more matching object name data from the object name data that has the highest degree of match with the keyword text data, and then generates the name identification result based on the matching object name data.

In some embodiments of the object recognition system of the present invention, the feature recognition result includes the object name data corresponding to the matching object feature data, and one or more confidence scores corresponding to the matching object feature data; the product name recognition result includes the matching object name data, and one or more confidence scores corresponding to the matching object name data; the processing unit determines the target object name data. The method is to determine whether one of the object name data is included in the feature recognition result and the product name recognition result. If the determination result is yes, the one of the object name data is used as the target object name data. If the determination result is no, the object name data with the highest corresponding confidence score among the object name data included in the feature recognition result or the product name recognition result is used as the target object name data.

In some embodiments of the object recognition system of the present invention, the processing unit selects a single matching object feature data that has the highest degree of match with the appearance of the object to be recognized from the object feature data, and then generates the feature recognition result, and the feature recognition result includes the object name data corresponding to the matching object feature data; the processing unit first uses the optical character recognition model to determine whether there is text information on the object to be recognized in the photographic result, and if the determination result is yes, obtains the key text data from the text information, and, after the processing unit recognizes the key text data In this case, the processing unit first selects a single matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name recognition result, and the product name recognition result includes the matching object name data; if the processing unit determines that the text information exists on the object to be recognized, the processing unit will use the object name data included in the product name recognition result as the target object name data; if the processing unit determines that the text information does not exist on the object to be recognized, the processing unit will use the object name data included in the feature recognition result as the target object name data.

In some embodiments of the object recognition system of the present invention, the processing unit first uses the optical character recognition model to determine whether text information consisting of multiple characters exists on the object to be identified in the photographic result. If the determination result is yes, the processing unit extracts at least a portion of the text information as the key text data based on the size and/or position of the characters in the photographic result.

Another object of the present invention is to provide a computer program product that enables an electronic device to correctly identify multiple objects of similar appearance.

The computer program product of the present invention includes an application program, which includes an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities. When a computer device obtains a feature data set and a product name data set, when the electronic device loads and runs the application program, the application program enables the computer device to implement the object identification method described in any of the aforementioned embodiments, wherein the feature data set includes multiple object feature data corresponding to multiple object types, and the product name data set includes multiple object name data corresponding to the object types and corresponding to the object feature data.

The object recognition system of the present invention is effective in that it can simultaneously consider the appearance of the object to be identified and the text on it to comprehensively determine the object type to be identified. In this way, the object recognition system helps avoid recognition errors caused by the similar appearance of objects of different types to be identified, and can more accurately identify the object type of each object to be identified.

Before the present invention is described in detail, it should be noted that, unless otherwise specified, the term "electrically connected" as used in this patent specification is used to describe the "coupled" relationship between computer hardware (e.g., electronic systems, devices, apparatuses, units, components), and generally refers to "wired electrical connections" achieved by physically connecting multiple computer hardware components to each other through conductive/semiconductor materials, as well as "radio connections" achieved by wireless data transmission using wireless communication technologies (such as, but not limited to, wireless networks, Bluetooth, and electromagnetic induction). On the other hand, unless otherwise specified, the "electrical connection" described in this patent specification also generally refers to a "direct electrical connection" achieved by directly coupling multiple computer hardware components to each other, and an "indirect electrical connection" achieved by indirectly coupling multiple computer hardware components through other computer hardware components.

This patent specification provides multiple embodiments of the same invention. Therefore, in the following description, similar elements between different embodiments are represented by the same numbers.

Referring to FIG. 1 , a first embodiment of the object recognition system 1 of the present invention is used to identify objects in an image to identify the type of the object in the image.

The object identification system 1 includes a processing unit 11 and a storage unit 12 electrically connected to the processing unit 11. Specifically, in this embodiment, the object identification system 1 is a computer device. The processing unit 11 is a processor implemented as an integrated circuit and having data processing and instruction transmission and reception functions. The storage unit 12 is a data storage device for storing digital data (such as a hard drive, but it can also be other types of computer-readable recording media).

However, in similar implementations of this embodiment, the processing unit 11 may also be a circuit assembly including a processor and a circuit board, and the storage unit 12 may also be a collection of multiple storage devices of the same or different types. Furthermore, in other embodiments, the object identification system 1 may also be implemented as multiple electrically connected computer devices (e.g., multiple servers). Moreover, in such embodiments, the processing unit 11 may be implemented as a collection of multiple processors/circuit assemblies respectively possessed by these computer devices, and the storage unit 12 may be implemented as a collection of multiple storage devices respectively possessed by these computer devices. Based on the above, the actual implementation of the object identification system 1 in terms of computer hardware is not limited to this embodiment.

In this embodiment, the processing unit 11 is adapted to be electrically connected to a camera 2, and the camera 2 is, for example, a camera or camcorder configured to shoot downward. Thus, the processing unit 11 can instantly obtain images captured by the camera 2 and analyze them to identify objects therein. In some embodiments, the camera 2 can be included in the object recognition system 1 as a camera unit. However, in other embodiments, if the processing unit 11 does not need to instantly analyze the images captured by the camera 2, the processing unit 11 does not need to be electrically connected to the camera 2.

The storage unit 12 stores a feature data set D1, a product name data set D2, an appearance feature matching model M1, an optical character recognition model M2, and a text relevance analysis model M3.

The feature data set D1 includes multiple pieces of object feature data D11, each corresponding to a plurality of object categories. In this embodiment, each object category corresponds to a specific product. For example, three of the object categories may be health foods such as Brand A's Vitamin B complex, Brand A's Vitamin C, and Brand A's Vitamin E. Furthermore, in this embodiment, each piece of object feature data D11 is an image or photograph depicting the packaging appearance of the corresponding product of that object category, such as the front or entire side of a product packaging box, but not limited thereto. It is worth noting that, in this embodiment, if the object recognition system 1 wishes to increase the types of objects that can be recognized, this can be achieved by simply adding product photos related to the object type to be recognized to the feature data set D1, without the need to spend time performing deep learning on the photos of the relevant products. Therefore, this embodiment has excellent scalability in terms of the types of objects that can be recognized. However, it should be noted that in other embodiments, each object feature data D11 can also be the result of deep learning of photos through machine learning. In other words, each object feature data D11 can also be appearance feature data learned through machine learning. Therefore, the actual form of the object feature data D11 is not limited to this embodiment.

In addition, in another embodiment, the camera device 2 is a depth camera. Therefore, the image captured by the camera device 2 includes not only the image but also depth information corresponding to the image. Furthermore, in this embodiment, each object feature data D11 includes an image (or photo) and size data, where the size data indicates the object size (e.g., length, width, and height) of the corresponding object type.

The product name data set D2 includes a plurality of object name data D21, wherein the object name data D21 respectively correspond to the object categories, and also respectively correspond to the object feature data D11 of the feature data set D1. More specifically, in this embodiment, each object name data D21 is a string, and indicates the product name of the corresponding object category, for example, in the form of text in natural language. Continuing with the aforementioned example of the three object categories of Brand A, namely, Vitamin B complex, Vitamin C, and Vitamin E, the three object name data D21 corresponding to the three object categories in the product name data set D2 can be implemented as "Vitality B complex", "Vitality C", and "Vitality E", for example, respectively, but is not limited to this. It should be noted that, in other embodiments, each object name data D21 is not limited to a string presented in natural language form, but can also be implemented as a product number or identification code of the relevant product. Therefore, the actual form of the object name data D21 is not limited to this embodiment.

The appearance feature matching model M1 is a software module implemented based on image recognition technology and can be loaded and run by the processing unit 11. The image recognition technology can be, for example, image segmentation technology based on deep learning. Specifically, the appearance feature matching model M1 can be implemented using semantic segmentation technology, instance segmentation technology, or panoramic segmentation technology within image segmentation technology. Since the appearance feature matching model M1 can be implemented using a variety of existing technologies and its details are not the focus of this specification, they will not be described in detail here.

The optical character recognition model M2 is a software module that can be loaded and run by the processing unit 11 and is implemented using existing optical character recognition (OCR) technology.

The text relevance analysis model M3 is a software module with text processing capabilities, and it can be loaded and run by the processing unit 11. More specifically, in this embodiment, the text relevance analysis model M3 is a large language model (LLM) implemented with neural network-like technology, such as but not limited to a generative pre-trained transformer (GPT) model based on a transformer, a language model for dialogue applications (LaMDA), or LLaMA (Large Language Model Meta AI). However, in another embodiment, the text relevance analysis model M3 can also be implemented using a CLIP (Contrastive Language-Image Pre-Training) model. Therefore, the text relevance analysis model M3 is not limited to being implemented with a large language model. It should be understood that the training and operation methods of both the Large Language Model and the CLIP model are existing technologies, so we will not go into too much detail here.

Referring to FIG. 2 , the following exemplarily illustrates in detail how the object identification system 1 of this embodiment implements an object identification method.

First, in step S1, the processing unit 11 obtains a photographic result showing at least one object to be identified. Specifically, in this embodiment, the processing unit 11 controls the camera 2 to capture images, thereby receiving the photographic result generated by the camera 2 from the camera 2. The photographic result is, for example, a photograph. It should be noted that in embodiments where the processing unit 11 is not electrically connected to the camera 2, the photographic result may be pre-stored in the storage unit 12, for example. Therefore, the processing unit 11 may also obtain the photographic result by reading from the storage unit 12. Alternatively, the processing unit 11 may also receive the shooting result from an external electronic device (such as a computer, a mobile phone or a flash drive) not shown in the figure.

In addition, in actual implementation, the shooting result may show multiple objects to be identified, but it may also show only a single object to be identified. For the sake of ease of description, it is assumed here that the shooting result shows only a single object to be identified.

After the processing unit 11 obtains the shooting result, the process proceeds to step S2.

In step S2, the processing unit 11 uses the appearance feature matching model M1 (i.e., loads and runs the appearance feature matching model M1) to analyze the appearance of the object to be identified in the photographic result according to the feature data set D1 to generate a feature recognition result corresponding to the object to be identified and related to the object name data D21.

Specifically, the processing unit 11 utilizes the appearance feature matching model M1 to compare the appearance of the object to be identified in the photographic result with the object feature data D11 for appearance similarity, and selects one or more matching object feature data D11′ (one of which is shown as an example in FIG1 ) that has the highest degree of match with the appearance of the object to be identified (i.e., the highest degree of appearance similarity) from the object feature data D11, and then generates the feature recognition result based on the object name data D21 corresponding to each of the matching object feature data D11′.

Furthermore, in this embodiment, the processing unit 11 selects a plurality of matching object feature data D11' that best matches the appearance of the object to be identified and that meets a predetermined number (e.g., but not limited to, two pieces), and then generates the feature recognition result based on the object name data D21 corresponding to each of the matching object feature data D11'. Furthermore, in this embodiment, the feature recognition result includes the object name data D21 corresponding to each of the matching object feature data D11' (e.g., "Vitality Group B," "Vitality C," etc., as mentioned above) and a plurality of confidence scores corresponding to each of the matching object feature data D11'. Here, each confidence score may be, for example, between 0 and 1, and, for each matching object feature data D11' and its corresponding confidence score, the confidence score may be understood as the degree of confidence of the processing unit 11 that "the appearance of the object to be identified in the photographic result is consistent with the matching object feature data D11'".

After the processing unit 11 generates the feature recognition result, the process proceeds to step S3.

In step S3, the processing unit 11 uses the optical character recognition model M2 (i.e., loads and runs the optical character recognition model M2) to determine whether text information consisting of multiple characters is present on the object to be recognized in the photographed result. If the determination is yes, the process proceeds to step S4. If not, the process proceeds to step S7.

In step S4 following step S3, once it is determined that the text information exists on the object to be identified in the photographic result, the processing unit 11 performs a text pre-processing on the text information to extract a portion of the text information and uses the extracted portion as a key text data.

Specifically, the processing unit 11 performs text pre-processing by, for example, segmenting the text information into sentences and words based on the arrangement of the characters in the text information (e.g., order, direction, and position), thereby breaking the text information into multiple text parts. The processing unit 11 then selects one or more of the text parts as the key text data based on, for example, the number of characters, semantics, font size in the image, and/or their relative positional relationships. In other words, the key text data is one or more text parts that appear on the object to be recognized in the image and are recognized by the processing unit 11 using the optical character recognition model M2.

For example, the processing unit 11 may select, as the key text data, one or two text portions whose word count is less than a threshold, whose semantics indicate a product name or brand, whose font size is the largest or second largest among all text portions, and whose location is relatively central to the image area occupied by the text information. It should be noted that the selection of text portions by the processing unit 11 as the key text data may also be determined based on the results of supervised machine learning, and is not limited to the above example.

After the processing unit 11 extracts the key text data from the text information, the process proceeds to step S5.

In step S5, the processing unit 11 uses the text relevance analysis model M3 (i.e., loads and runs the text relevance analysis model M3) to analyze the key text data according to the product name data set D2 to generate a product name recognition result corresponding to the object to be recognized and related to the object name data D21.

Specifically, the processing unit 11 utilizes the text relevance analysis model M3 to compare the key text data with the object name data D21 for text similarity, and selects one or more matching object name data D21' (one of which is shown as an example in FIG1 ) from the object name data D21 that has the highest degree of match with the key text data (i.e., the highest degree of similarity in text composition), and then generates the product name recognition result based on the matching object name data D21'.

Furthermore, in this embodiment, the processing unit 11 selects a plurality of matching object name data D21′ that best matches the key text data and meets a predetermined number (e.g., but not limited to, two) and then generates a product name recognition result based on these matching object name data D21′. Furthermore, in this embodiment, the product name recognition result includes the matching object name data D21′ (e.g., "Vitality Group B" and "Vitality C" in the aforementioned examples) and a plurality of confidence scores corresponding to the matching object name data D21′. Each confidence score may be, for example, between 0 and 1, and for each matching object name data D21′ and its corresponding confidence score, the confidence score may be understood as the degree of confidence of the processing unit 11 that “the keyword text data on the object to be identified matches the matching object name data D21′”.

After the processing unit 11 generates the product name recognition result, the process proceeds to step S6.

In step S6, the processing unit 11 uses one of the object name data D21 as target object name data D21* (illustrated in FIG. 1 ) based on the feature recognition result and the product name recognition result, and outputs the target object name data D21*. The processing unit 11 determines that the target object name data D21* is equivalent to determining that the object to be identified belongs to the object type corresponding to the target object name data D21* (e.g., the aforementioned brand A vitamin B complex).

Specifically, in this embodiment, the processing unit 11 determines the target object name data D21* by determining whether one of the object name data D21 is included in both the feature recognition result and the product name recognition result. If the determination result of the processing unit 11 is yes, the processing unit 11 uses the one of the object name data D21 as the target object name data D21*. For example, assuming that the two object name data D21 included in the feature recognition result are "Vibrant Group B" and "Vibrant C", and the two object name data D21 included in the product name recognition result (i.e., the matching object name data D21') are "Vibrant Group B" and "Vibrant E", since both the feature recognition result and the product name recognition result include the object name data D21 of "Vibrant Group B", the processing unit 11 will use the object name data D21 of "Vibrant Group B" as the target object name data D21*. On the other hand, if the judgment result of the processing unit 11 is negative (i.e., the object name data D21 included in the feature recognition result and the product name recognition result are completely different), the processing unit 11 will use the object name data D21 with the highest corresponding confidence score among all the object name data D21 included in the feature recognition result or the product name recognition result as the target object name data D21*.

On the other hand, the processing unit 11 may output the target object name data D21* by, for example, displaying the target object name data D21* on a display device (not shown) to allow a user to confirm the object type identified by the object identification system 1. However, in other embodiments, the processing unit 11 may output the target object name data D21* by, for example, transmitting the target object name data D21* to the storage unit 12 for storage, or transmitting the target object name data D21* to an external electronic device, and the present embodiment is not limited thereto.

In step S7, following step S3, if it is determined that the text information exists on the object to be identified in the photographic result, the processing unit 11 is unable to obtain the key text data from which to generate the product name identification result. In this case, the processing unit 11 uses one of the object name data D21 as target object name data D21* based on the feature identification result and outputs the target object name data D21* (e.g., by displaying it on the display device). More specifically, when the processing unit 11 cannot obtain the key text data, the processing unit 11 uses the object name data D21 with the highest confidence score among the object name data D21 of the feature recognition result as the target object name data D21*.

It should be noted that, in other similar embodiments, the processing unit 11 may also select only a single matching object feature data D11’ that has the highest degree of match with the appearance of the object to be identified in step S2. Therefore, the feature identification result may also include only a single object name data D21 and a single confidence score corresponding to the matching object feature data D11’. Similarly, the processing unit 11 may also select only a single matching object name data D21’ that has the highest degree of match with the keyword text data in step S5. Therefore, the product name identification result may also include only a single matching object name data D21’ and another single confidence score corresponding to the matching object name data D21’.

The above is an example of how the object identification system 1 of this embodiment implements the object identification method.

The above example description assumes that the shooting result in step S1 only presents a single object to be identified. If in step S1, the shooting result presents multiple objects to be identified, then unlike the above example description, in step S2, the processing unit 11 uses the appearance feature matching model M1 to analyze the appearance of each object to be identified in the shooting result, and generates multiple feature identification results corresponding to the objects to be identified, and then continues to execute step S3 and other steps after step S3 for each object to be identified in the shooting result. It is particularly noted that by executing step S3 and subsequent steps for all objects to be identified, the processing unit 11 determines multiple target object name data D21* corresponding to the objects to be identified. Moreover, in a preferred practical implementation, the processing unit 11 outputs the target object name data D21* at one time, which is equivalent to generating and outputting an object name list.

By implementing the object recognition method, for each object to be identified presented in the photographic result, the present embodiment can first analyze the appearance of the object to be identified in the photographic result to generate the feature recognition result, then obtain the key text data from the object to be identified after determining that there is text information on the object to be identified, and generate the product name recognition result by analyzing the key text data. Finally, based on both the feature recognition results and the product name recognition results, a decision is made as to which object name data D21 to output as the target object name data D21*. In other words, this embodiment can simultaneously consider the appearance (e.g., shape, outline, color, etc.) of the object to be recognized as well as the text on it to comprehensively determine which object type the object to be recognized is. Therefore, if the image capture results show multiple objects to be identified that appear similar in appearance but are actually of different types (for example, multiple health supplements or other products of the same brand and similar packaging, or multiple boxes of similar shape and size), this embodiment helps avoid misidentification due to the similar appearance of the objects to be identified, and can more accurately identify the object type of each object to be identified. Based on this, this embodiment is particularly suitable for applications such as inventory counting of incoming and outgoing goods, or identifying items to be checked out at self-checkout machines in physical stores. On the other hand, the present embodiment can also be used in conjunction with a robotic arm device (not shown). For example, in step S1, the processing unit 11 can control the photographing device 2 to take pictures to obtain the photographing result when the object to be identified is held by the robotic arm device and moved into the photographing range of the photographing device 2. In addition, where the object to be identified will be subsequently moved by the robotic arm device (for example, into which container it will be placed) is determined based on the target object name data D21* selected by the processing unit 11.

In an advanced implementation of this embodiment, each object feature data D11 is a template image, and each template image is implemented as a photograph of an object belonging to the corresponding object type. N pieces of object feature data D11 (N is an integer greater than or equal to 1) are used as N pieces of marked object feature data. Each marked object feature data includes a key region pre-marked by the user, and the key region corresponds to a region of interest (ROI) in the marked object feature data to be used as an image recognition target. For example, the N object types corresponding to the N pieces of tagged object feature data may be, for example, N types of pharmaceuticals or health foods with similar packaging but different ingredients (or dosages of the ingredients). Furthermore, the key area of each piece of tagged object feature data may be, for example, the portion of the corresponding object type (e.g., the packaging box) where the product ingredients (or dosages of the ingredients) are displayed in text. In step S2 of the object recognition method of this embodiment, after the processing unit 11 selects the matching object feature data D11', for each piece of matching object feature data D11', the processing unit 11 further determines whether the matching object feature data D11' belongs to one of the N pieces of tagged object feature data. If the determination result of the processing unit 11 is no, the processing unit 11 continues to execute the subsequent steps S3 to S7. However, if the judgment result of the processing unit 11 is yes, it means that the object to be identified is an object that is particularly easy to be confused. In this case, the processing unit 11 uses the appearance feature matching model M1 again to compare the appearance similarity of a key part of the overall appearance of the object to be identified in the shooting result with the N key areas of the N marked object feature data, and selects from the N marked object feature data one of the matching marked object feature data whose included key area has the highest degree of matching with the key part of the object to be identified, and generates the feature identification result based on the object name data D21 corresponding to the matching marked object feature data. The key portion refers to the portion of the object to be identified in the overall appearance of the photographed result that corresponds to the key region of the N pieces of marked object feature data, such as a label on a packaging box that describes its ingredients or dosage in text. Furthermore, in subsequent steps S3 and S4, the processing unit 11 uses the optical character recognition model M2 to determine whether text information exists in the key portion of the object to be identified. If the determination is yes, the processing unit 11 performs text pre-processing on the text information in the key portion to obtain the key text data.

It should be noted that steps S1 through S7 of this embodiment and the flowchart in FIG2 are merely intended to illustrate one possible implementation of the object identification method of the present invention. It should be understood that even if steps S1 through S7 are combined, split, or reordered, if the resulting process achieves substantially the same effect as the present embodiment in a substantially identical manner, it still falls within the scope of the object identification method of the present invention. Therefore, steps S1 through S7 of this embodiment and the flowchart in FIG2 are not intended to limit the scope of implementation of the present invention.

The present invention also provides a second embodiment of the object identification system 1. The second embodiment differs from the first embodiment in the object identification method implemented by the object identification system 1.

Specifically, in step S2 of the object recognition method of the second embodiment, the processing unit 11 selects a single piece of matching object feature data D11' from the object feature data D11 that best matches the appearance of the object to be recognized. The processing unit 11 then generates the feature recognition result based on the object name data D21 corresponding to the matching object feature data D11'. More specifically, the feature recognition result includes the object name data D21 corresponding to the matching object feature data D11'.

Furthermore, in step S5 of the second embodiment, if the processing unit 11 determines that the text information exists on the object to be identified and obtains the key text data therefrom (i.e., recognizes the key text data), the processing unit 11 first selects a single matching object name data D21′ from the object name data D21 that has the highest degree of match with the key text data, and then generates the product name identification result based on the matching object name data D21′. More specifically, the product name identification result includes the matching object name data D21′.

Furthermore, in step S6 of the second embodiment (i.e., when the processing unit 11 has obtained the key text data), the processing unit 11 directly uses the matching object name data D21' included in the product name recognition result as the target object name data D21*, i.e., the product name recognition result is used as the basis without considering the feature recognition result. On the other hand, in step S7 of the second embodiment (i.e., when the processing unit 11 has not obtained the key text data), the processing unit 11 directly uses the object name data D21 included in the feature recognition result as the target object name data D21*.

The present invention also provides an embodiment of a computer program product, wherein the computer program product includes an application program that can be stored on a computer-readable recording medium and can be loaded and executed by a computer device (e.g., a laptop or desktop computer). The application program includes, for example, the appearance feature matching model M1, the optical character recognition model M2, and the text relevance analysis model M3. Furthermore, when the computer device obtains the feature data set D1 and the product name data set D2 (e.g., manually input by a user or received from an external electronic device), when the electronic device loads and executes the application program of the computer program product, the application program can cause the computer device to implement the object recognition method described in any of the aforementioned embodiments.

In summary, by implementing the object identification method, the object identification system 1 can simultaneously consider the appearance of the object to be identified and the text present thereon to comprehensively determine the object type to be identified as. In this way, the object identification system 1 helps to avoid identification errors caused by the similar appearances of the objects to be identified and can more accurately identify the object type to which each object to be identified belongs, thereby truly achieving the purpose of the present invention.

However, the above description is merely an example of the present invention and should not be used to limit the scope of the present invention. All simple equivalent changes and modifications made within the scope of the patent application and the contents of the patent specification of the present invention are still within the scope of the present patent.

1··········· Object recognition system 11········· Processing unit 12········· Storage unit D1········ Feature data set D11······· Object feature data D11’······ Matching object feature data D2········ Product name data set D21······· Object name data D21’······ Matching object name data D21*······ Target object name data M1········ Appearance feature matching model M2········ Optical character recognition model M3········· Text Relevance Analysis Model 2. Camera Equipment Steps S1-S7

Other features and functions of the present invention are clearly illustrated in the accompanying drawings, wherein: Figure 1 is a block diagram illustrating a first embodiment of the object identification system of the present invention; and Figure 2 is a flow chart illustrating how the first embodiment implements an object identification method.

S1~S7····· Steps

Claims (7)

An object recognition method is implemented by an object recognition system, the object recognition system including a processing unit and a storage unit, the storage unit storing a feature data set, a product name data set, an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities, the feature data set including a plurality of object feature data corresponding to a plurality of object categories, the product name data set including a plurality of object name data corresponding to the object categories and corresponding to the object feature data; the object recognition method includes: (A) the processing unit obtains a photographic result showing at least one object to be recognized, and uses the appearance feature matching model to process the image feature matching model; The appearance of the object to be identified in the photographed result is analyzed based on the feature data set to generate a feature identification result corresponding to the object to be identified and related to the object name data, wherein the processing unit first selects one or more matching object feature data with the highest degree of matching with the appearance of the object to be identified from the object feature data, and then generates the feature identification result based on the object name data to which the matching object feature data respectively corresponds, and the feature identification result includes the object name data to which the matching object feature data respectively corresponds, and one or more confidence scores corresponding to the matching object feature data respectively; (B) the processing unit uses the optical character recognition When the recognition model identifies a key text data presented on the object to be identified in the photographic result, the processing unit analyzes the key text data according to the product name data set using the text relevance analysis model to generate a product name recognition result corresponding to the object to be identified and related to the object name data, wherein the processing unit first selects one or more matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name recognition result based on the (such) matching object name data, the product name recognition result including the (such) matching object name data and one or more confidence scores corresponding to the (such) matching object name data; and (C) the processing unit The processing unit determines the target object name data by determining whether one of the object name data is included in the feature recognition result and the product name recognition result. In the identification result and the product name identification result, if the judgment result is yes, one of the object name data will be used as the target object name data. If the judgment result is no, the object name data with the highest corresponding confidence score among the object name data included in the feature identification result or the product name identification result will be used as the target object name data. The object identification method as described in claim 1, wherein: in step (A), the processing unit selects a single matching object feature data that has the highest degree of match with the appearance of the object to be identified from the object feature data, and then generates the feature identification result, and the feature identification result includes the object name data corresponding to the matching object feature data; in step (B), the processing unit first uses the optical character recognition model to determine whether there is text information on the object to be identified in the shooting result, and if the determination result is yes, obtains the key text data from the text information, and, if the processing unit has identified the key text data The processing unit first selects a single matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name recognition result, and the product name recognition result includes the matching object name data; and in step (C), if the processing unit determines in step (B) that the text information exists on the object to be recognized, the processing unit uses the object name data included in the product name recognition result as the target object name data; if the processing unit determines in step (B) that the text information does not exist on the object to be recognized, the processing unit uses the object name data included in the feature recognition result as the target object name data. The object recognition method as described in claim 1, wherein in step (B), the processing unit first uses the optical character recognition model to determine whether text information consisting of multiple characters exists on the object to be identified in the photographic result, and if the determination result is yes, at least a portion of the text information is extracted from the text information as the key text data based on the size and/or position of the characters in the photographic result. An object recognition system includes: a processing unit; and a storage unit electrically connected to the processing unit and storing a feature data set, a product name data set, an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities, wherein the feature data set includes multiple object feature data corresponding to multiple object categories, and the product name data set includes multiple object name data corresponding to the object categories and corresponding to the object feature data; wherein the processing unit is used to: obtain a photographic result showing at least one object to be recognized, and use the appearance feature matching model to identify the object according to the feature data set; The apparatus further comprises a processing unit for processing a feature recognition result of the object to be identified and the object name data. The processing unit first selects one or more matching object feature data with the highest degree of matching with the appearance of the object to be identified from the object feature data, and then generates the feature recognition result according to the object name data to which the matching object feature data corresponds. The feature recognition result includes the object name data to which the matching object feature data corresponds, and one or more confidence scores corresponding to the matching object feature data. The apparatus further comprises a processing unit for processing a feature recognition result of the object to be identified and the object name data to which the matching object feature data corresponds. In the case of a key text data presented on the object to be identified in the result, the processing unit uses the text relevance analysis model to analyze the key text data according to the product name data set to generate a product name identification result corresponding to the object to be identified and related to the object name data, wherein the processing unit first selects one or more matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name identification result based on the (such) matching object name data, the product name identification result including the (such) matching object name data and one or more confidence scores corresponding to the (such) matching object name data respectively; and according to the feature identification At least one of the feature recognition result and the product name recognition result is selected, and one of the object name data is used as a target object name data, and the target object name data is output. The processing unit determines the target object name data by judging whether one of the object name data is included in the feature recognition result and the product name recognition result. If the judgment result is yes, the one of the object name data is used as the target object name data. If the judgment result is no, the object name data with the highest confidence score among the object name data included in the feature recognition result or the product name recognition result is used as the target object name data. The object recognition system as described in claim 4, wherein: the processing unit selects a single matching object feature data that has the highest degree of match with the appearance of the object to be identified from the object feature data, and then generates the feature recognition result, and the feature recognition result includes the object name data corresponding to the matching object feature data; the processing unit first uses the optical character recognition model to determine whether there is text information on the object to be identified in the photographic result, and if the determination result is yes, obtains the key text data from the text information, and, after the processing unit has identified the key text data In this case, the processing unit first selects a single matching object name data with the highest degree of matching with the key text data from the object name data, and then generates the product name recognition result, and the product name recognition result includes the matching object name data; and if the processing unit determines that the text information exists on the object to be identified, the processing unit will use the object name data included in the product name recognition result as the target object name data; if the processing unit determines that the text information does not exist on the object to be identified, the processing unit will use the object name data included in the feature recognition result as the target object name data. As described in claim 4, the object recognition system, wherein the processing unit first uses the optical character recognition model to determine whether text information consisting of multiple characters exists on the object to be identified in the photographic result, and if the determination result is yes, extracts at least a portion of the text information as the key text data based on the size and/or position of the characters in the photographic result. A computer program product includes an application program, which includes an appearance feature matching model implemented based on image recognition technology, an optical character recognition model, and a text relevance analysis model with text processing capabilities. When a computer device obtains a feature data set and a product name data set, when the electronic device loads and runs the application program, the application program enables the computer device to implement the object identification method as described in any one of claims 1 to 3, wherein the feature data set includes multiple object feature data corresponding to multiple object types, and the product name data set includes multiple object name data corresponding to the object types and corresponding to the object feature data.