[go: up one dir, main page]

WO2022015236A1 - Procédé de traitement d'images pour l'identification d'objets et système associé - Google Patents

Procédé de traitement d'images pour l'identification d'objets et système associé Download PDF

Info

Publication number
WO2022015236A1
WO2022015236A1 PCT/SG2020/050416 SG2020050416W WO2022015236A1 WO 2022015236 A1 WO2022015236 A1 WO 2022015236A1 SG 2020050416 W SG2020050416 W SG 2020050416W WO 2022015236 A1 WO2022015236 A1 WO 2022015236A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
predefined
image portion
portions
identified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/SG2020/050416
Other languages
English (en)
Inventor
Shuyang Dou
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Ltd
Original Assignee
Hitachi Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Ltd filed Critical Hitachi Ltd
Priority to PCT/SG2020/050416 priority Critical patent/WO2022015236A1/fr
Publication of WO2022015236A1 publication Critical patent/WO2022015236A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/817—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level by voting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates

Definitions

  • the present invention generally relates to a method of image processing for object identification and a system thereof.
  • FIG. 1A depicts a diagram 100 illustrating an example scenario for vehicle recognition across multiple cameras (e.g., at different locations & time), which illustrates a challenging problem due to visual appearance difference caused by viewpoint change.
  • FIG. 1A depicts a diagram 100 illustrating an example scenario for vehicle recognition across multiple cameras (e.g., at different locations & time), which illustrates a challenging problem due to visual appearance difference caused by viewpoint change.
  • IB illustrates example images 120a, 120b and 120c of the same vehicle from a VeRi dataset which were obtained from a database, as described in Liu, Xinchen, et al. "Large-scale vehicle re-identification in urban surveillance videos.” 2016 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2016.
  • the images 120a, 120b and 120c may be taken by three different cameras with different viewpoints.
  • there is significant visual appearance difference for example, due to viewpoint change, and therefore it may be difficult to identify that the vehicle in images 120a, 120b and 120c is the same vehicle.
  • FIG. 1C illustrates example images 150a, 150b and 150c of different vehicles which were obtained from the database, as described in Liu, Xinchen, et al. "Large-scale vehicle re-identification in urban surveillance videos.” 2016 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2016. As illustrated, the respective vehicles in the images 150a, 150b and 150c, although are different vehicles, have similar appearance.
  • a conventional technique to address the problem above is to utilize affine transformation to eliminate the visual appearance difference.
  • U.S. Patent No. 8,295,604 entitled “Image search method and device using affine-invariant regions” filed on 23 Oct. 2012, which describes a “neighboring regions” approach to apply the affine transformation affine invariant regions are extracted from a query image and their neighboring regions, then each region is compared with every region in a pre-processed learning table to obtain votes (i.e., using neighboring regions and affine transformation).
  • an optimized learning table which contains neighboring regions information and affine transformation is generated during a learning stage, while a voting table is generated during a matching stage by comparing each region (and neighboring regions) of a query image with every region in the learning table.
  • the learning table may contain the neighborhood relationship for each region and corresponding affine transformation.
  • correspondences are decided by referring to the votes in voting table, e.g., highest votes in each row.
  • the conventional approach may compare each region of the query image with every region in the reference images (database images). In this regard, the processing associated with the comparison for all parts of the images for the identification may be inefficient and time consuming.
  • a method of image processing for object identification using at least one processor comprising: obtaining an image of an object; identifying at least one image portion of the image corresponding to at least one object portion of the object based on a plurality of predefined object portions related to the object, the at least one object portion corresponds to at least one predefined object portion of the plurality of predefined object portions; determining a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and determining whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • a system for image processing for object identification comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: obtain an image of an object; identify at least one image portion of the image corresponding to at least one object portion of the object based on a plurality of predefined object portions related to the object, the at least one object portion corresponds to at least one predefined object portion of the plurality of predefined object portions; determine a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and determine whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • a computer program product embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform a method of image processing for object identification, the method comprising: obtaining an image of an object; identifying at least one image portion of the image corresponding to at least one object portion of the object based on a plurality of predefined object portions related to the object, the at least one object portion corresponds to at least one predefined object portion of the plurality of predefined object portions; determining a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and determining whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • FIG. 1A depicts a diagram illustrating an example scenario for vehicle recognition across multiple cameras
  • FIG. IB illustrates images of the same vehicle which was taken by three different cameras with different viewpoints
  • FIG. 1C illustrates images of different vehicles which have similar appearance
  • FIG. 2 depicts a schematic flow diagram of a method (computer-implemented method) of image processing for object identification using at least one processor according to various embodiments of the present invention
  • FIG. 3 depicts a schematic block diagram of an image processing system for object identification according to various embodiments of the present invention, such as corresponding to the method shown in FIG.2;
  • FIG. 4 depicts an example computer system which the system according to various embodiments of the present invention may be embodied in;
  • FIG. 5A illustrates a schematic of a vehicle divided into four planar parts according to various example embodiments of the present invention
  • FIG. 5B illustrates an exemplary schematic of an object portions definition data set which may be defined in relation to the plurality of predefined object parts of the object (e.g., vehicle) in FIG. 5A according to various example embodiments of the present invention
  • FIG. 6A illustrates another exemplary schematic of a vehicle divided into planar parts to define a plurality of predefined object portions of the vehicle according to various example embodiments of the present invention
  • FIG. 6B illustrates another exemplary schematic of an object portions definition data set in relation to the plurality of predefined object parts related to the object (e.g., vehicle) in FIG. 6A according to various example embodiments of the present invention
  • FIGS. 7A and 7B illustrate exemplary frameworks of image processing for object identification according to various example embodiments of the present invention
  • FIG. 8 illustrates another exemplary schematic of image processing for object identification according to various example embodiments of the present invention.
  • FIG. 9 shows an exemplary process flow of image processing for object identification according to various example embodiments of the present invention.
  • FIG. 10 shows an exemplary schematic diagram of a process of image processing for object identification according to various example embodiments of the present invention
  • FIG. 11 shows two exemplary images of an object each comprising an image portion of the image corresponding to an object portion and transformed image portions according to various example embodiments of the present invention
  • FIG. 12 illustrates a schematic in relation to key point detection and affine rectification for two different vehicle parts according to various example embodiments of the present invention
  • FIG. 13 shows another exemplary process flow of image processing for object identification according to various example embodiments of the present invention.
  • FIG. 14 shows an exemplary process flow for determining a similarity of at least one transformed image portion corresponding to at least one object portion according to various example embodiments of the present invention.
  • FIG. 15 illustrate an exemplary framework of image processing for object identification combined with vehicle number plate recognition according to various example embodiments of the present invention.
  • FIG. 2 depicts a schematic flow diagram of a method 200 (computer- implemented method) of image processing for object identification using at least one processor according to various embodiments of the present invention.
  • the method 200 comprises obtaining (at 202) an image (e.g., a query image) of an object (target object); identifying (at 204) at least one image portion of the image corresponding to (e.g., thereof or in relation to) at least one object portion of the object (i.e., respectively) based on a plurality of predefined object portions related to the object, the at least one object portion corresponds to at least one predefined object portion of the plurality of predefined object portions; determining (at 206) a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and determining (at 208) whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • an image e.g., a query image
  • identifying at least one image portion of the image corresponding to (e.g., thereof or in relation to) at least one object portion of the object (i.e., respectively) based
  • the object may be an object(s) having (approximately) planar portions or parts which are discriminative that is desired to be identified (or recognized).
  • the object may be an artifact having at least approximately or substantially planar portions which are discriminative relative to one another and is useful for identification/recognition purposes.
  • the image of the object may be obtained from one or more image capturing devices.
  • the one or more image capturing devices may be of any type(s) known in the art, for example, may be a camera at a location or a plurality of cameras disposed at different locations of an area or region desired to be covered.
  • the one or more image capturing devices may produce an image of the object and at least one processor may then obtain (or receive) the image for processing to identify (or recognize) the object.
  • the image of the object may be obtained from a database and at least one processor may then obtain (or receive) the image for processing to identify (or recognize) the object.
  • the above-mentioned identifying at least one image portion of the image corresponding to at least one object portion of the object may include identifying multiple image portions of the image corresponding to multiple object portions of the object based on the plurality of predefined object portions related to the object. The multiple object portions may each correspond to a respective predefined object portion of the plurality of predefined object portions.
  • an object desired to be identified may be defined (e.g., divided or segmented) into a plurality of predefined object portions, each corresponding to a portion (e.g., a unique part) of the object which may be helpful for images thereof to be processed according to various embodiments for identifying (e.g., recognizing) the object (or the type of object).
  • Each of the plurality of predefined object portions (e.g., predefined object parts) related to the object may be an (approximately) planar portion according to a structure of the object and is unique to the object.
  • each predefined object portion may be a unique portion of the object to facilitate identification.
  • each of the plurality of predefined object portions may have meaning in relation to object structure (i.e., related to vehicle structure).
  • the predefined object portions may be defined by one or more users based on domain knowledge with identification purpose in mind prior to processing an image for object identification such that each predefined object portion may provide unique information for object identification or recognition (e.g., each predefined object portion is selected and pre defined by one or more users using domain knowledge with identification purpose in mind). Accordingly, the at least one image portion identified may be co-related to one predefined object portion.
  • each of the plurality of predefined object portions may include unique information (or unique features) related the object to facilitate subsequent comparison (or matching) of an image portion identified from the query image with a corresponding image portion identified from a reference image (i.e., comparison of corresponding image portion pairs from different images).
  • each of the plurality of predefined object portions may correspond to a corresponding vehicle part or structure, such as front face, tail face, left side, right side, and so on.
  • the at least one image portion identified may include unique information (or unique features) of the object for facilitating subsequent comparison (or matching).
  • the predefined object portions may overlap each other.
  • the predefined object portions related to the object may be defined such that the two or more object portions comprise overlapping area.
  • the overlap may be partial or complete.
  • one or more users may define “front face” and “number plate area at a front face” as two different object portions. In such case, the latter predefined object portion is completely overlapped in the former predefined object portion.
  • the at least one corresponding reference image portion of at least one corresponding object portion of a reference object may be identified based on the plurality of predefined object portions related to the object.
  • image portions of corresponding object portion pairs from different images e.g., query image and reference image
  • each identified image portion corresponding to each object portion (e.g., of a query image and reference image) is already correlated or corresponds to at least one predefined object portion of the plurality of predefined object portions, therefore only image portions of corresponding object portion pairs from different images are compared (e.g., image portion corresponding to front face of vehicle from query image compared to reference image portion corresponding to front face of vehicle from reference image).
  • the object being a vehicle and and each of the at least one object portion of the object corresponds to a part of the vehicle
  • the at least one image portion identified may be compared with the at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively, for example, to determine similarity values of corresponding object portion pairs from the image and the reference image.
  • the above-mentioned determining whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified may provide information whether the object and the reference object are the same objects, same type of objects, same class of objects, etc.
  • various embodiments provide a method of image processing for object identification based on the plurality of predefined object portions related to the object.
  • the plurality of predefined object portions may be predefined discriminative (approximately) planar portions.
  • an algorithm e.g., by detecting intensity change of image pixels
  • object structure e.g., vehicle structure
  • reference image e.g., one to all
  • various embodiments of the present invention compare image portions of corresponding object portion pairs from the query image and the reference image based on the plurality of predefined object portions (e.g., “one to one” comparisons), which improves the identification accuracy by avoiding mismatching.
  • various embodiments of the present invention increases processing speed in relation to determining the similarity of each image portion identified with corresponding reference image portions as only corresponding object portion pair may be compared (e.g., it is not necessary to compare the image portion corresponding to the front face part of a vehicle from a query image with the image portion corresponding to the left body part of a vehicle from a reference image).
  • the method 200 may further comprise performing transformation of the at least one image portion identified to obtain at least one transformed image portion corresponding to the at least one object portion (i.e., respectively) based on an object portions definition data set related to the object.
  • the object portions definition data set comprises a plurality of predefined data subsets defining the plurality of predefined object portions, respectively, related to the object.
  • the above-mentioned transformation is performed based on the predefined data subset defining the predefined object portion of the plurality of predefined object portions corresponding to the object portion corresponding to the image portion. Accordingly, the above-mentioned determining a similarity of the at least one image portion identified may be based on the at least one transformed image portion corresponding to the at least one object portion.
  • each predefined object portion may be defined by a set of key points.
  • a set of key points may include at least four key points.
  • the key points in a set of key points of a predefined object portion may be interconnected to form a planar polygon defining the object portion.
  • the above-mentioned identifying at least one image portion of the image comprises, for each image portion corresponding to the corresponding object portion of the object, detecting a set of key points in the image corresponding to a predefined object portion of the plurality of predefined object portions related to the object.
  • the set of key points in the image may be detected based on pattern recognition or a machine learning model trained based on the plurality of predefined object portions related to the object.
  • the machine learning model may comprise a neural network such as a plurality of convolutional neural networks (CNNs) each trained based on a respective predefined object portion of the plurality of predefined object portions related to the object.
  • CNNs convolutional neural networks
  • the plurality of CNNs may be trained separately for respective predefined object portions to detect a set of key points in the image corresponding to a predefined object portion.
  • each CNN may be trained based on images comprising an object portion corresponding to a respective predefined object portion for the CNN.
  • each CNN of the plurality of CNN may be trained for a particular predefined object portion and used to detect key points for that predefined object portion based on an image comprising an object portion corresponding to that predefined object portion.
  • a single CNN model may be used to detect all key points in relation to all predefined object portions.
  • the CNN model may be trained using all key points in relation to all the predefined object portions.
  • a CNN may be used to detect key points related to two or more predefined object portions. For example, in the case of a vehicle having five predefined vehicle portions, two CNNs may be used for key points detection.
  • one CNN may be used to detect key points related to two predefined vehicle portions, and another CNN may be used to detect key points related to the remaining three predefined vehicle portions.
  • the image of the object may be input into the machine learning model, and the machine learning model may output the location of each key point in the image.
  • the output of a CNN may be a set of key points (e.g., a set of coordinates of corresponding key points in the image).
  • each of the plurality of predefined data subsets comprises a set of rectified coordinates for defining a shape associated with the corresponding predefined object portion.
  • the set of rectified coordinates defines a planar polygon associated with the corresponding predefined object portion.
  • the above-mentioned performing transformation of the at least one image portion identified comprises, for each image portion identified: determining a correspondence relationship between the set of key points detected for the image portion and the set of rectified coordinates associated with the corresponding predefined object portion; and transforming the image portion based on the correspondence relationship determined to obtain the transformed image portion.
  • the correspondence relationship comprises an affine transformation matrix.
  • the image portion is affine transformed based on the affine transformation matrix determined to obtain the transformed image portion in the form of a rectified image portion.
  • the transformation of the at least one image portion identified to obtain at least one transformed image portion corresponding to the at least one object portion may be an affine transformation or rectification.
  • the method 200 may further comprise: obtaining a reference image of the reference object; identifying at least one reference image portion of the reference image corresponding to at least one reference object portion of the reference object based on the plurality of predefined object portions; and performing transformation of the at least one reference image portion identified to obtain at least one transformed reference image portion corresponding to the at least one reference object portion based on the object portions definition data set, wherein the at least one reference object portion corresponds to at least one predefined object portion of the plurality of predefined object portions, and wherein for each of the at least one reference image portion identified, the transformation is performed based on the predefined data subset defining the predefined object portion of the plurality of predefined object portions corresponding to the reference object portion corresponding to the reference image portion.
  • the transformation of the at least one reference image portion identified may be the same or similar to that of the transformation of the at least one image portion described above.
  • the reference image of the reference object may be a different frame captured by the same image capture device from which the image (or query image) of the object is obtained.
  • the image and the reference image may be obtained from different frames captured by the same image capture device.
  • the image and the reference image may be sequential frames in a predetermined time period.
  • the reference image of the reference object may be obtained from a frame of a different image capture device than an image capture device from which the image (or query image) of the object is obtained.
  • the method may then determine whether the object in the image (or query image) corresponds to (the same as) the reference object in the reference image based on the determined similarity of the at least one transformed image portion corresponding to the at least one object portion.
  • the object and the reference object may be the same object, same type of object same classification, etc. In other cases, the object and the reference object may be different.
  • the reference image of the reference object may be obtained in real time from one or more image capture devices. In other embodiments, the reference image of the reference object may be obtained from a database.
  • the method 200 may further comprise obtaining at least one transformed reference image portion corresponding to the at least one reference object portion from the database, wherein the at least one reference object portion corresponds to at least one predefined object portion of the plurality of predefined object portions.
  • the method 200 may further comprise obtaining image features extracted from at least one transformed reference image portion corresponding to the at least one reference object portion from the database, wherein the at least one reference object portion corresponds to at least one predefined object portion of the plurality of predefined object portions.
  • the above-mentioned determining a similarity of the at least one transformed image portion comprises determining, for each transformed image portion, a similarity of the transformed image portion with respect to the corresponding transformed reference image portion to output a corresponding similarity value.
  • the above-mentioned determining a similarity of the at least one image portion identified may be based on pixel intensity correlation, structure similarity or image feature distance.
  • the similarity for corresponding object portion pairs from different images may be determined.
  • the similarity may be determined based on intensity correlation, structure similarity, Normalized Crossed Correlation (NCC) between the transformed image portion from a query image and transformed reference image portion from a reference image, or combination thereof.
  • NCC Normalized Crossed Correlation
  • the NCC for each transformed image portion may be determined and the average of all NCCs may be computed to obtain a final similarity value.
  • the object in the query image may be determined to correspond to the reference image (e.g., the two objects in the query image and the reference image may be recognized as the same object).
  • Other techniques may also be used to determine the similarity such as extracting features from two images (e.g., query image and reference image), and compute the feature distance (e.g., the Euclidean distance) as similarity (or lack of similarity thereof).
  • the object comprises a vehicle, a building, or a suitcase, and each of the at least one object portion of the object corresponds to a planar part according to a structure of the vehicle, the building, or the suitcase.
  • the vehicle may be a car, a train, an airplane, in various non-limiting examples. It is understood that the present invention is not limited to the above-mentioned objects. For example, other types of objects having discriminative (approximately) planar parts may also be used in the image processing method as described according to various embodiments of the present invention for object identification (or recognition).
  • the object may be an artefact which may be defined or divided into multiple (approximately) planar portions.
  • the predefined object portion is an approximately planar portion.
  • each of the at least one transformed image portion may then be determined with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object to determine whether the object corresponds to the reference object.
  • vehicle identification or vehicle type classification e.g., deciding the type of the vehicle.
  • one highly demanded use case is to track a vehicle in a restricted area in real-time. It is quite often that the tracking may be broken, for example, due to occlusion, limitation of cameras’ field of view (FoV), etc. In this case, the tracking may be resumed by identifying the vehicle with previously tracked vehicles.
  • vehicle identification Another common usage of vehicle identification is to identify whether the query vehicle is one of the vehicles registered in a current database. Accordingly, a more stable vehicle tracking in outdoor environment or more reliable vehicle identification (which is free of viewpoint change) may be implemented.
  • vehicle type classification for example, some representative vehicle images of each vehicle type may be chosen, and when the query image matches one of the representative vehicle images, the vehicle type of the query image man be defined as the matched representative image.
  • the object comprises a vehicle and the reference object comprises a reference vehicle
  • the method 200 may further comprise identifying a number plate image portion of the image corresponding to a number plate portion of the vehicle; performing number plate recognition of the number plate image portion identified to obtain a number plate information; determining a similarity of the number plate image portion identified with respect to a corresponding reference number plate image portion of a corresponding reference number plate portion of the reference vehicle based on the number plate information; and determining whether the object corresponds to the reference object based on the determined similarity of the number plate image portion identified.
  • FIG. 3 depicts a schematic block diagram of an image processing system 300 for object identification according to various embodiments of the present invention, such as corresponding to the method 200 of image processing for object identification as described hereinbefore according to various embodiments of the present invention.
  • the system 300 comprises a memory 304, and at least one processor 306 communicatively coupled to the memory 304 and configured to: obtain an image of an object; identify at least one image portion of the image corresponding to at least one object portion of the object based on a plurality of predefined object portions related to the object, the at least one object portion corresponds to at least one predefined object portion of the plurality of predefined object portions; determine a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and determine whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • the at least one processor 306 may be configured to perform the required functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 306 to perform the required functions or operations. Accordingly, as shown in FIG.
  • the system 300 may further comprise an image obtaining module (or circuit) 308 configured to obtain an image of an object; an image processing module (or circuit) 310 configured to identify at least one image portion of the image corresponding to at least one object portion of the object; a similarity determining module (or circuit) 312 configured to determine a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object, respectively; and an object identification module (or circuit) 314 configured to determine whether the object corresponds to the reference object based on the determined similarity of the at least one image portion identified.
  • an image obtaining module or circuit 308 configured to obtain an image of an object
  • an image processing module or circuit
  • 310 configured to identify at least one image portion of the image corresponding to at least one object portion of the object
  • a similarity determining module (or circuit) 312 configured to determine a similarity of the at least one image portion identified with respect to at least one corresponding reference image portion of at least one corresponding object
  • modules are not necessarily separate modules, and two or more modules may be realized by or implemented as one functional module (e.g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention.
  • the image obtaining module 308, the image processing module 310, the similarity determining module 312, and/or the object identification module 314 may be realized (e.g., compiled together) as one executable software program (e.g., software application or simply referred to as an “app”), which for example may be stored in the memory 304 and executable by the at least one processor 306 to perform the functions/operations as described herein according to various embodiments.
  • one executable software program e.g., software application or simply referred to as an “app”
  • the system 300 corresponds to the method 200 as described hereinbefore with reference to FIG. 2, therefore, various functions/operations configured to be performed by the least one processor 306 may correspond to various steps or operations of the method 200 described hereinbefore according to various embodiments, and thus need not be repeated with respect to the system 300 for clarity and conciseness.
  • various embodiments described herein in context of the methods are analogously valid for the respective systems (e.g., which may also be embodied as devices).
  • the memory 304 may have stored therein the image obtaining module 308, the image processing module 310, the similarity determining module 312, and/or the object identification module 314, which respectively correspond to various steps or operations of the method 200 as described hereinbefore, which are executable by the at least one processor 36 to perform the corresponding functions/operations as described herein.
  • a computing system, a controller, a microcontroller or any other system providing a processing capability may be provided according to various embodiments in the present disclosure.
  • Such a system may be taken to include one or more processors and one or more computer-readable storage mediums.
  • the system 300 described hereinbefore may include a processor (or controller) 306 and a computer-readable storage medium (or memory) 304 which are for example used in various processing carried out therein as described herein.
  • a memory or computer-readable storage medium used in various embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).
  • DRAM Dynamic Random Access Memory
  • PROM Programmable Read Only Memory
  • EPROM Erasable PROM
  • EEPROM Electrical Erasable PROM
  • flash memory e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).
  • a “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof.
  • a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g., a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor).
  • a “circuit” may also be a processor executing software, e.g., any kind of computer program, e.g., a computer program using a virtual machine code, e.g., Java.
  • a “module” may be a portion of a system according to various embodiments in the present invention and may encompass a “circuit” as above, or may be understood to be any kind of a logic-implementing entity therefrom.
  • the present specification also discloses a system (which may also be embodied as a device or an apparatus) for performing the operations/functions of the methods described herein.
  • a system may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer.
  • the algorithms presented herein are not inherently related to any particular computer or other apparatus.
  • Various general-purpose machines may be used with computer programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate.
  • the present specification also at least implicitly discloses a computer program or software/functional module, in that it would be apparent to the person skilled in the art that the individual steps or operations of the methods described herein may be put into effect by computer code.
  • the computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein.
  • the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the scope of the invention.
  • modules described herein may be software module(s) realized by computer program(s) or set(s) of instructions executable by a computer processor to perform the required functions, or may be hardware module(s) being functional hardware unit(s) designed to perform the required functions. It will also be appreciated that a combination of hardware and software modules may be implemented.
  • a computer program/module or method described herein may be performed in parallel rather than sequentially.
  • Such a computer program may be stored on any computer readable medium.
  • the computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general- purpose computer.
  • the computer program when loaded and executed on such a general- purpose computer effectively results in an apparatus that implements the steps or operations of the methods described herein.
  • a computer program product embodied in one or more computer-readable storage mediums (non-transitory computer- readable storage medium), comprising instructions (e.g., the image obtaining module 308, the image processing module 310, the similarity determining module 312, and/or the object identification module 314) executable by one or more computer processors to perform a method 200 of image processing for object identification as described hereinbefore with reference to FIG. 2.
  • instructions e.g., the image obtaining module 308, the image processing module 310, the similarity determining module 312, and/or the object identification module 31
  • various computer programs or modules described herein may be stored in a computer program product receivable by a system (e.g., a computer system or an electronic device) therein, such as the system 300 as shown in FIG. 3, for execution by at least one processor 306 of the system 300 to perform the required or desired functions.
  • the software or functional modules described herein may also be implemented as hardware modules. More particularly, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC). Numerous other possibilities exist. Those skilled in the art will appreciate that the software or functional module(s) described herein can also be implemented as a combination of hardware and software modules.
  • ASIC Application Specific Integrated Circuit
  • the above-mentioned computer system may be realized by any computer system (e.g., portable or desktop computer system), such as a computer system 400 as schematically shown in FIG. 4 as an example only and without limitation.
  • Various methods/operations or functional modules e.g., the image obtaining module 308, the image processing module 310, the similarity determining module 312, and/or the object identification module 314) may be implemented as software, such as a computer program being executed within the computer system 400, and instructing the computer system 400 (in particular, one or more processors therein) to conduct the methods/functions of various embodiments described herein.
  • the computer system 400 may comprise a computer module 402, input modules, such as a keyboard 404 and a mouse 406, and a plurality of output devices such as a display 408, and a printer 410.
  • the computer module 402 may be connected to a computer network 412 via a suitable transceiver device 414, to enable access to e.g. the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN).
  • the computer module 402 in the example may include a processor 418 for executing various instructions, a Random Access Memory (RAM) 420 and a Read Only Memory (ROM) 422.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • the computer module 402 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 424 to the display 408, and I/O interface 426 to the keyboard 404.
  • I/O interface 424 to the display 408
  • I/O interface 426 to the keyboard 404.
  • the components of the computer module 402 typically communicate via an interconnected bus 428 and in a manner known to the person skilled in the relevant art.
  • an image (e.g., query image) of a vehicle to be identified may be processed based on a plurality of predefined object portions related to the vehicle, performing transformation of the image based on an object portions definition data set related to the vehicle and determining a similarity of the processed image with respect to a reference image of a reference vehicle so as to determine whether the object corresponds to the reference vehicle in the reference image.
  • a vehicle e.g., an object
  • a vehicle may be divided into multiple parts (object portions) to define a plurality of predefined object portions related to the vehicle.
  • Each selected part of the vehicle may be an approximately planar part for its corresponding predefined object portion.
  • the planar parts of an image may be identified (and extracted) and the corresponding affine transformation matrix may be estimated as affine transformation can only be applied to planar objects.
  • FIG. 5A illustrates a schematic 500a of a vehicle divided into four planar parts, including a front face 502, a tail face 504, a left side 506, and a right side 508 to define four predefined object portions (a plurality of predefined object portions) of the vehicle.
  • Each planar part of the vehicle may be selected to define the corresponding predefined object portion such that the predefined object portion is unique to the vehicle from other predefined object portions.
  • unique parts may be selected to define the plurality of predefined object portions related to the object so as to avoid subsequent mismatching problem during processing of the image.
  • FIG. 5B illustrates an exemplary schematic 500b of an object portions definition data set which may be defined in relation to the plurality of predefined object parts of the object (e.g., vehicle) in FIG. 5A.
  • the object portions definition data set may be a predefined table comprising a plurality of predefined data subsets (e.g., 512, 514, 516 and 518) defining the plurality of predefined object portions (e.g., 502, 504, 506, and 508), respectively, related to the object (e.g., vehicle) in FIG. 5A. It is understood that the number and name of each vehicle part in FIG. 5A and FIG. 5B is merely for purpose of illustration.
  • each of the plurality of predefined data subsets comprises a set of rectified coordinates (or data points) 520 for defining a shape of a transformed image portion corresponding to the corresponding predefined object portion.
  • each set of rectified coordinates may comprise four or more rectified coordinates (or data points) for defining a shape of a transformed image portion corresponding to the corresponding predefined object portion.
  • Each rectified coordinate may correspond to a key point of a predefined object portion.
  • the rectified coordinates of the set may be connected to uniquely define the shape of the transformed image portion corresponding to the corresponding predefined object portion.
  • the set of rectified coordinates defines a planar polygon of the corresponding predefined object portion.
  • each rectified coordinate or data point may be indicated or defined in the object portions definition data set related to the object.
  • the rectified coordinates or data point in the object portions definition data set related to the object may correspond to upper left corner of left fog lamp of the vehicle, upper right corner of right fog lamp of the vehicle, lower right corner of right headlight of the vehicle, lower left corner of left headlight of the vehicle, upper left corner of left rear fog lamp of the vehicle, etc.
  • Each rectified coordinate or data point may be a unique point in the object according to its definition. Thus, it may be desirable to use general terms rather than special terms that are used only by some specific makers/users to define, for example, the vehicle coordinates or data points in the case the object is a vehicle.
  • FIGS. 5 A and 5B illustrates the rectified coordinates in each set of coordinates corresponding to key points for defining a shape of the corresponding predefined object portion to be corner points.
  • the predefined data subset 512 for the front part 502 of the vehicle comprises a set of rectified coordinates 520 which includes coordinates corresponding to key points (e.g., upper left corner of left fog of lamp, upper right corner of right fog of lamp, lower right corner of right headlight, lower left corner of left headlight) to define a shape of the transformed image portion corresponding to the front part 502 of the vehicle.
  • key points e.g., upper left corner of left fog of lamp, upper right corner of right fog of lamp, lower right corner of right headlight, lower left corner of left headlight
  • the rectified coordinates of each set of rectified coordinates for defining the shape of a transformed image portion corresponding to the corresponding predefined object portion are not limited to corner points.
  • some coordinates or data points inside the vehicle parts may also be used if the coordinates or data points in the vehicle can be uniquely defined.
  • the set of rectified coordinates 520 are the expected coordinates of vehicle points in a transformed image portion (rectified image portion or rectified view or undistorted image portion).
  • a transformed image portion may be a normalized image corresponding to the object portion (vehicle part) which is invariant of viewpoint change.
  • a top-down view may be selected to be the transformed image portion.
  • all vehicle parts may be transformed or rectified to their top- down views before comparison.
  • the set of rectified coordinates 520 are parameters for calculating the transformed image portion. One may roughly choose the value so that the transformed image portion may have good visual features.
  • FIG. 6A illustrates another exemplary schematic 600a of a vehicle divided into planar parts to define a plurality of predefined object portions of the vehicle
  • FIG. 6B illustrates another exemplary schematic 600b of an object portions definition data set in relation to the plurality of predefined object parts related to the object (e.g., vehicle) in FIG. 6A.
  • the object portions definition data set may be a predefined table comprising a plurality of predefined data subsets (e.g., 612, 614, 616, 618, 620, 622 and 624) defining the plurality of predefined object portions, respectively, related to the object (e.g., vehicle) in FIG. 6A.
  • Each of the plurality of predefined data subsets comprises a set of rectified coordinates or data points 520 for defining a shape of the corresponding predefined object portion.
  • “50 pixels” may be used for both length and width of the transformed image portion corresponding to the predefined object portion 624 (e.g., Roof).
  • the value of the set of rectified coordinates 520 depends on the shape of the transformed image portion of the vehicle part which may be experience -based. In other words, these values reflect the expected shape of the vehicle part (in a transformed image portion).
  • FIGS. 5A-5B and FIGS. 6a-6B illustrate eleven vehicle parts being defined which encompasses most visible parts of the vehicle by a camera view.
  • each vehicle part may be further divided into multiple small parts.
  • the “LeftBody” part may be divided into “front door”, “back door”, etc.
  • the predefined object portions may overlap each other.
  • the predefined object portions related to the object may be defined such that the object portions are overlapping.
  • a rectified coordinate may form two or more sets of rectified coordinates corresponding to two or more predefined object portions.
  • an image of an object (e.g., vehicle) to be identified or recognized may be obtained, and based on the plurality of predefined object portions related to the object, image portions of the image corresponding to object portions of the object may be identified or extracted.
  • a transformation may be performed for each image portion identified based on the object portions definition data set to rectify distortions in visual appearance so as to reduce or minimize visual appearance difference in the image portion identified, for example, which may be caused by a camera’s viewpoint change.
  • the transformation may reduce or minimize visual appearance difference of corresponding object portions of a same object in different images (e.g., obtained image and reference image).
  • the transformation may be an affine transformation used to rectify each image portion identified so as to obtain transformed image portions corresponding to object portions.
  • Each transformed image portion may be compared to a corresponding reference image portion of a corresponding object portion of a reference object to determine the similarity.
  • FIGS. 7A and 7B illustrate exemplary frameworks 700a and 700b, respectively, of image processing for object identification according to various example embodiments of the present invention.
  • an image (query image) 702 of an object e.g., vehicle
  • the image 702 of the object may be an arbitrary image of vehicle to be identified.
  • the image of the object 702 may be a real-time frame captured by an outdoor surveillance camera.
  • the query image 702 of the object may be divided into multiple image portions corresponding to multiple object portions (e.g., vehicle parts) 704, respectively, according to the plurality of predefined object portions 706 related to the object (e.g., identifying at least one image portion of the image corresponding to at least one object portion of the object based on a plurality of predefined object portions related to the object).
  • at least one image portion of the image 702 may be identified by detecting, for each image portion corresponding to the corresponding object portion of the object, a set of key points in the image 702 corresponding to a predefined object portion of the plurality of predefined object portions related to the object.
  • the set of key points in the image 702 may be automatically detected based on a machine learning model trained based on the plurality of predefined object portions related to the object.
  • the machine learning model may be a Convolutional Neural Network (CNN) model.
  • CNN Convolutional Neural Network
  • the key points may be detected sequentially or simultaneously.
  • a transformation (or rectification) 708 of each image portion identified may be performed to obtain transformed image portions 710 corresponding to object portions identified in the query image 702.
  • the transformation 708 of each image portion identified may be performed according to an estimated affine transformation matrix.
  • the detected key points and corresponding rectified coordinates predefined in vehicle part definition
  • the affine transformation matrix may be estimated.
  • the affine transformation matrix is a 3x3 homogenous matrix which defines the relationship between two planes. It should be noted that this matrix has eight degrees-of-freedom, which means at least four corresponding coordinate pairs should be used for estimation (e.g., at least four coordinates for each set of coordinates are necessary for each predefined objection portion.
  • the estimated affine transformations may be applied to each image portion identified to rectify the visual appearance difference, for example, caused by one or more cameras’ viewpoint change.
  • Each transformed image portion 710 corresponding to an object portion identified in the query image 702 may be compared to a corresponding reference image portion 720 of a corresponding object portion of a reference object to determine a similarity of the transformed image portion 710 corresponding to the object portion (i.e., comparing image portions of corresponding object portion pairs).
  • the corresponding reference image portion 720 of the corresponding object portion of the reference object may be obtained from a database 725.
  • a reference image of the reference object may be obtained and pre-processed and stored into the database 725.
  • pre-processing of the reference image may similarly include dividing the reference image into multiple reference image portions corresponding to multiple reference object portions, respectively, according to the plurality of predefined object portions 706 related to the object (identifying at least one reference image portion of the reference image corresponding to at least one reference object portion of the reference object based on the plurality of predefined object portions), and performing a transformation of each reference image portion identified to obtain transformed reference image portions corresponding to reference object portions identified in the reference image.
  • the transformed reference image portions may be stored into the database 725 or cached to speed up the image processing, for example, if an image gallery (comprising the reference image) is seldom changed.
  • processing of the reference image may be performed in real-time.
  • a reference image 732 of the reference object may be obtained, for example, from an image gallery.
  • the reference image 732 may be processed in the same or similar manner to the processing of the query image of the object 702 (e.g., dividing the reference image into multiple reference image portions corresponding to multiple reference object portions, respectively, according to the plurality of predefined object portions 706 related to the object, and performing a transformation of each reference image portion identified to obtain transformed reference image portions corresponding to reference object portions identified in the reference image).
  • the reference object in the reference image 732 may be known.
  • the reference object in the reference image may have been registered into a database.
  • an example implementation may be to identify the object in the query image 702 by comparing the query image captured in real-time with the reference images of reference objects that have already been registered (e.g., reference images of reference object in the image gallery).
  • the similarity of this object portion pair may be determined.
  • a determination of whether the object corresponds to the reference object may be made, for example, based on similarities of image portions of corresponding object portion pairs (e.g., image portion corresponding to front face of vehicle from query image compared to reference image portion corresponding to front face of vehicle reference image).
  • FIG. 8 illustrates another exemplary schematic 800 of image processing for object identification according to various example embodiments of the present invention.
  • FIG. 9 shows an exemplary process flow 900 of image processing for object identification according to various example embodiments of the present invention.
  • the process flow 900 illustrates processing from vehicle detection to transformation (e.g., affine rectification) for one image portion of the image corresponding to one object portion of the object.
  • an image region corresponding to the vehicle may be detected.
  • the output of vehicle detection may be a rectangular bounding box including the vehicle (object).
  • the image region corresponding to the vehicle may be cropped using this bounding box.
  • vehicle detection processing may be applied to detect the location of the vehicle in the query image (i.e., pixel coordinates corresponding to a vehicle inside the original query image).
  • Various algorithms may be used for the vehicle detection processing such as Yolov3 which uses a deep learning model to detect location of object in image (bounding box) as described in Redmon, Joseph, and Ali Farhadi.
  • key point detection may be performed on the cropped image. It should be noted that key points may also be detected directly from the original image of the vehicle as captured. For example, step 910 and 920 may be omitted. However, it is noted that key points detection on the cropped image may be easier because the searching area may be reduced to the bounding box area rather than performed on the whole image. Key points corresponding to each predefined object portion (vehicle part) may be detected respectively. These key points may be detected simultaneously or one by one.
  • Such correspondence information may be provided by the key point detection algorithm (machine learning model for the key point detection).
  • a correspondence relationship between the set of key points detected for the image portion and the set of rectified coordinates of the corresponding predefined object portion may be determined.
  • the correspondence relationship may be an affine transformation matrix.
  • Each object portion e.g., vehicle part
  • the corresponding affine matrix may be applied to its corresponding vehicle part.
  • the estimated affine matrices are used for transforming (or rectifying) each image portion identified.
  • a transformation may be performed to the image portion identified. For example, an image portion corresponding to an object portion may be cropped from the image of the vehicle, and a transformation (e.g., affine rectification) may be applied to obtain a transformed image portion. Alternatively, the transformation may be applied to the whole image and then the transformed image portion corresponding to the object portion (vehicle part) may be cropped using the corresponding set of rectified coordinates.
  • FIG. 10 shows an exemplary schematic diagram of a process 1000 of image processing for object identification according to various example embodiments of the present invention.
  • An exemplary transformed image portion 1010 is illustrated.
  • FIG. 11 shows two exemplary images 1110a and 1120a of an object each comprising an image portion of the image 1110a, 1120a corresponding to an object portion of the object identified based on a plurality of predefined object portions related to the object.
  • Transformed image portions 1110b, 1120b corresponding to the object portions may be obtained by applying a transformation (e.g., affine transformation) based on an object portions definition data set related to the object. It can be seen that the transformed image portions 1110b, 1120b have much more similar visual appearance after the transformation.
  • a transformation e.g., affine transformation
  • FIG. 12 illustrates a schematic 1200 in relation to key point detection and affine rectification for two different vehicle parts.
  • an image portion corresponding to an object portion may be transformed to a “skewed” view 1210 in the transformed image portion.
  • the orientation of the transformed image portion may be implicitly defined by the predefined set of rectified coordinates in the object portions definition data set related to the object.
  • the process flow 1300 shows another exemplary process flow 1300 of image processing for object identification according to various example embodiments of the present invention.
  • the process flow 1300 illustrates processing from key points detection to transformation (e.g., affine rectification) for each image portion of the image corresponding to each object portion of the object based on a plurality of predefined object portions related to the object.
  • the process flow 1300 is similar to the process flow 900 described with respect to FIG. 9 and will not be discussed in detail in the interest of brevity.
  • key points corresponding to each predefined object portion (vehicle part) detection may be performed (e.g., to identify image portion corresponding to object portion of the object based on the plurality of predefined object portions related to the object), a correspondence relationship between the set of key points detected for the image portion and the set of coordinates of the corresponding predefined object portion may be determined (e.g., affine transformation matrix estimation), and a transformation may be performed to the image portion identified.
  • the similarity for corresponding object portion pairs from different images may be computed.
  • Various algorithms may be used to compute the similarity between two images.
  • the similarity between the image portion from the query image and the image portion from the reference image may be computed based on pixel intensity correlation, structure similarity or image feature distance.
  • the similarity of the at least one image portion identified may be determined based on pixel intensity correlation such as using the Normalized Crossed Correlation (NCC). Equation (1) as follows shows a formula for calculating the NCC between the rectified vehicle part pair from a query image and a reference image.
  • Q(i,j) is the pixel intensity at location (i,j) of a query image
  • G(i,j) is the pixel intensity of a reference image.
  • Other techniques such as template matching methods may be implemented for calculating similarity of each corresponding object portion pair.
  • Another common approach for comparing two images is to extract features from both images, and compute the feature distance (e.g., the Euclidean distance) as similarity (or lack of similarity thereof), i.e., image feature distance.
  • feature distance e.g., the Euclidean distance
  • image feature distance e.g., image feature which needs to be designed manually by researcher/engineer
  • deep learning/machine learning models such as CNN models may also be used for image feature extraction.
  • the overall similarity for each image portion of the at least one image portion may be determined.
  • the NCC for each image portion identified may be computed and the average of all NCCs may be computed to obtain the overall similarity value.
  • the two vehicles in the query image and the reference image may be recognized as the same vehicle, same class of vehicle, same type of vehicle, etc.
  • FIG. 14 shows an exemplary process flow 1400 for determining a similarity of at least one transformed image portion corresponding to at least one object portion with respect to at least one corresponding reference image portion of at least one corresponding object portion of a reference object (e.g., comparing image portions corresponding to vehicle parts) according to various example embodiments of the present invention.
  • a score from the similarities may be estimated. For example, an average over all similarities may be determined or the maximum/minimum similarity value may be taken as the score. Alternatively, a coefficient for each vehicle part may be defined, and the weighted sum of similarities may be computed as the score. The score may be used for making a final identification decision. For example, two vehicles captured by two different image may be identified to be the same object if the score is higher than a predetermined threshold. [0085] In various example embodiments, the image processing method according to various example embodiments of the present invention may be combined with vehicle number plate recognition, as illustrated in FIG. 15.
  • the recognized “vehicle number information” may or may not contain the complete vehicle number.
  • the confidence of the vehicle number information may be used as coefficient when computing the similarity between two vehicle numbers. This is similar as the “weighted similarity” described above.
  • the vehicle number information for example, may be recognized alphabet and numbers.
  • the matching rate may be computed as the similarity (e.g., the number or amount of characters that are correctly matched).
  • the vehicle number information may be image feature extracted from the number plate area. In this case, the extracted image feature may be viewed as encoded “alphabet and numbers”, and the feature distance may be used for determining similarity.
  • the similarity for the detected number plate area may be computed, and combined with determined similarities from other vehicle parts.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de traitement d'images pour l'identification d'objets. Le procédé consiste à : obtenir une image d'un objet; identifier au moins une partie d'image de l'image correspondant à au moins une partie d'objet de l'objet sur la base d'une pluralité de parties d'objet prédéfinies associées à l'objet, l'au moins une partie d'objet correspondent à au moins une partie d'objet prédéfinie parmi la pluralité de parties d'objet prédéfinies; déterminer une similarité de l'au moins une partie d'image identifiée par rapport à au moins une partie d'image de référence correspondante d'au moins une partie d'objet correspondante d'un objet de référence, respectivement; et déterminer si l'objet correspond à l'objet de référence sur la base de la similarité déterminée de l'au moins une partie d'image identifiée. L'invention concerne également un système correspondant de traitement d'images pour l'identification d'objets.
PCT/SG2020/050416 2020-07-17 2020-07-17 Procédé de traitement d'images pour l'identification d'objets et système associé Ceased WO2022015236A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG2020/050416 WO2022015236A1 (fr) 2020-07-17 2020-07-17 Procédé de traitement d'images pour l'identification d'objets et système associé

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2020/050416 WO2022015236A1 (fr) 2020-07-17 2020-07-17 Procédé de traitement d'images pour l'identification d'objets et système associé

Publications (1)

Publication Number Publication Date
WO2022015236A1 true WO2022015236A1 (fr) 2022-01-20

Family

ID=79555798

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2020/050416 Ceased WO2022015236A1 (fr) 2020-07-17 2020-07-17 Procédé de traitement d'images pour l'identification d'objets et système associé

Country Status (1)

Country Link
WO (1) WO2022015236A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130039569A1 (en) * 2010-04-28 2013-02-14 Olympus Corporation Method and apparatus of compiling image database for three-dimensional object recognition
CN105488479A (zh) * 2015-12-03 2016-04-13 浙江宇视科技有限公司 一种基于加权局部特征比对的车辆套牌识别方法及装置
CN108596277A (zh) * 2018-05-10 2018-09-28 腾讯科技（深圳）有限公司 一种车辆身份识别方法、装置和存储介质
US20180293552A1 (en) * 2017-04-11 2018-10-11 Alibaba Group Holding Limited Image-based vehicle maintenance plan
CN109145759A (zh) * 2018-07-25 2019-01-04 腾讯科技（深圳）有限公司 车辆属性识别方法、装置、服务器及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130039569A1 (en) * 2010-04-28 2013-02-14 Olympus Corporation Method and apparatus of compiling image database for three-dimensional object recognition
CN105488479A (zh) * 2015-12-03 2016-04-13 浙江宇视科技有限公司 一种基于加权局部特征比对的车辆套牌识别方法及装置
US20180293552A1 (en) * 2017-04-11 2018-10-11 Alibaba Group Holding Limited Image-based vehicle maintenance plan
CN108596277A (zh) * 2018-05-10 2018-09-28 腾讯科技（深圳）有限公司 一种车辆身份识别方法、装置和存储介质
CN109145759A (zh) * 2018-07-25 2019-01-04 腾讯科技（深圳）有限公司 车辆属性识别方法、装置、服务器及存储介质

Similar Documents

Publication Publication Date Title
Talmi et al. Template matching with deformable diversity similarity
Alvarez et al. Road scene segmentation from a single image
US9690977B2 (en) Object identification using 3-D curve matching
US8406470B2 (en) Object detection in depth images
Heo et al. Pedestrian detection at night using deep neural networks and saliency maps
WO2016145940A1 (fr) Procédé et dispositif d'authentification faciale
Nebehay Robust object tracking based on tracking-learning-detection
US11380010B2 (en) Image processing device, image processing method, and image processing program
WO2021017283A1 (fr) Procédé et appareil de suivi en ligne basé sur un procédé hors ligne, dispositif informatique et support d'enregistrement
US9679218B2 (en) Method and apparatus for image matching
CN111079571A (zh) 证卡信息识别及其边缘检测模型训练方法、装置
CN109919128B (zh) 控制指令的获取方法、装置及电子设备
Bellavia et al. HarrisZ+: Harris corner selection for next-gen image matching pipelines
CN112686247A (zh) 一种身份证号码检测方法、装置、可读存储介质和终端
CN105095835A (zh) 行人检测方法及系统
Jelača et al. Vehicle matching in smart camera networks using image projection profiles at multiple instances
Zhu et al. Single image pop-up from discriminatively learned parts
Shi et al. License plate localization in complex environments based on improved GrabCut algorithm
Cyganek et al. Vehicle logo recognition with an ensemble of classifiers
US9418305B1 (en) Segmentation free approach to automatic license plate recognition
CN116129414A (zh) 车牌识别方法、装置、电子设备及存储介质
Huang et al. Multi‐class obstacle detection and classification using stereovision and improved active contour models
Abbas Recovering homography from camera captured documents using convolutional neural networks
Silva et al. Vehicle occupancy detection for HOV/HOT lanes enforcement
CN113284253A (zh) 一种改进核相关过滤波（kcf）算法的ar目标跟踪方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20944927

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20944927

Country of ref document: EP

Kind code of ref document: A1