[go: up one dir, main page]

WO2016187681A1 - Procédé de construction d'une base d'images, procédé de reconnaissance d'images, système de reconnaissance d'images et ses utilisations - Google Patents

Procédé de construction d'une base d'images, procédé de reconnaissance d'images, système de reconnaissance d'images et ses utilisations Download PDF

Info

Publication number
WO2016187681A1
WO2016187681A1 PCT/BR2016/000017 BR2016000017W WO2016187681A1 WO 2016187681 A1 WO2016187681 A1 WO 2016187681A1 BR 2016000017 W BR2016000017 W BR 2016000017W WO 2016187681 A1 WO2016187681 A1 WO 2016187681A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
characteristic points
images
database
pairs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/BR2016/000017
Other languages
English (en)
Portuguese (pt)
Inventor
José Mario DE MARTINO
Helio PEDRINI
Renan Ricardo Soares LOBO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Squadra Tecnologia S/a
Universidade Estadual de Campinas UNICAMP
Original Assignee
Squadra Tecnologia S/a
Universidade Estadual de Campinas UNICAMP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from BR102015012437-6A external-priority patent/BR102015012437B1/pt
Application filed by Squadra Tecnologia S/a, Universidade Estadual de Campinas UNICAMP filed Critical Squadra Tecnologia S/a
Publication of WO2016187681A1 publication Critical patent/WO2016187681A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries

Definitions

  • the present invention relates to a method of constructing an image base, an image recognition method, an image recognition system and its uses.
  • tourism is a billion dollar industry and major source of resources in many countries.
  • tourist activity involves the exploration of the visited place, with emphasis on the so-called local tourist spots that, for historical, geographical or leisure reasons, characterize the visited place.
  • tourists Being out of their environment, tourists often have difficulty recognizing a certain point, and do not know detailed information about that location.
  • US document 2011/0176734 Al is aimed at recognizing specific areas of an image, especially areas containing buildings and constructions.
  • the technology in question adopts the strategy of grouping characteristic points to perform identification, which ends up being a filtering process that reduces the ability to recognize image details as well as utilizes pose estimation, which leads to uncertainties and compromise the effectiveness of recognition.
  • the scientific works of Liu (2014) and Hu (2006) also have similar application in the recognition of building facades, but restricted to this type of structure.
  • US 2009/7565139 B2 presents a proposal to provide the description of an image by combining optical character recognition, rigid objects, and face recognition. However, its application is restricted to the description of images (and their components), and is not intended to identify the photographed location.
  • Some technologies use geolocation data from the device in question to match images or photos in a collection of images, such as latitude / longitude coordinates or device orientation. This is the case with patents US 2008/0147730, US 2012/8189964 B2, US 2013/0198176 Al and US 8467810, and scientific work by Chen (2013), Zheng (2009), Abe (2010), Kim (2012) and Byungsoo (2009). [8] Alternatively, location recognition alternatives are dealt with in US patent 8483715 and in the works by Knopp (2010) and Chen (2009), who use images available on the Internet, on large bases such as Panoramio, Google Street Vie, Flickr, among others, associating with the recognition process geolocation or label information produced by the authors of these images.
  • the present invention relates to a method of constructing an image base, an image recognition method, an image recognition system and its uses.
  • the method of constructing an image base comprises the steps of:
  • the image recognition method is responsible for identifying and recognizing an input image by through efficient comparison with records in the database. Its steps are:
  • the image recognition system comprises the application of the above methods together with the image acquisition and processing equipment, namely at least one mobile device with camera and Internet connection, and a server for remote processing.
  • FIG. 3 illustrates the different recording conditions of the images under the different driving conditions, view, time, weather, and camera model.
  • FIG. 5 illustrates the criterion for excluding characteristic point pairs found as a function of their deviation from other pairs.
  • FIG. 8 illustrates the comparison of the two images from another angle with the pairs of characteristic points joined by lines, but with the presence of two lines that are highly divergent from the others.
  • FIG. 9 illustrates the situation that the tourist spot in question is photographed from the opposite angle to previous situations. ; .
  • the present invention relates to a method of constructing an image base, an image recognition method, an image recognition system and its uses.
  • the method of building an image base comprises the steps of:
  • Figure 1 illustrates the steps of the image base construction method from acquiring reference images from a place of interest (101).
  • the construction of the database and definition of the indexing and search mechanisms is performed by pre-processing these images (102), extracting their characteristic points.
  • the image acquisition step 101 follows a well-defined protocol illustrated by Figures 2 and 3.
  • Each site of interest or reference to be employed in the construction The image base should be photographed from different positions, preferably arranged in a circle around the reference, equally spaced ; and at a distance sufficient for your frame.
  • the photographs can also be taken in the opposite direction of the place of interest, associating their surroundings and giving robustness to the image base (figure 2).
  • each reference site may require a specific procedure, the image acquisition protocol being customizable.
  • the more images acquired from a place of interest the more likely it is that a photo of that place will be associated with an image from the image base.
  • Places of interest with access or restricted viewing positions at certain angles do not require a total of 8 positions to be used for image acquisition, requiring only photos from positions without access impediment.
  • Preprocessing these images (102) is to resize each image to ensure that the largest side of all images is the same size. This resizing occurs to maintain the original aspect ratio (width divided by height) of the image. In embodiments of this invention, dimensions between 300 to 1000 pixels have been used.
  • a feature detection and extraction algorithm (103) is applied, which is transformed into a data representation to be recorded in the database.
  • This representation contains all the information necessary for the recognition method to function, and it is no longer necessary to keep the original image stored.
  • the Speed-Up Robust Features (SURF) algorithm was used for the extraction of feature points from the image, and the JSON format for data representation in the database.
  • the SIFT algorithm ⁇ Scale-Invariant Feature Transform) can be used as an alternative to SURF, however, being less performance-efficient and less robust when applied to images with different transformations.
  • Another suitable data representation format may also be used without compromising the essence of the invention.
  • a data structure is created to maintain associations between the place of interest data representations of a photograph and the place of interest name associated with that photograph.
  • the recognition method works by receiving an input image to be identified (401), which must also go through a preprocessing step (402), where it resizes while maintaining the aspect ratio. and extracting its feature points (403), which can also be used SURF algorithm and the representation of JSON data in one: the implementations of this invention. [29] The method then traverses the database and compares the characteristic points of the input image with the characteristic points of the reference images that make up a ; looking for similarities (404). This step seeks to find correspondences or common characteristic points in the two images, then generating a list of the pairs of points that met the proposed similarity requirement. :
  • the threshold value influences the robustness of the solution. A small threshold value results in a less demanding recognition process and can lead to false acknowledgments. Too high a value is restrictive and may lead to non-recognition of the point of interest.
  • This threshold can range from 10 pairs for smaller images (largest side near 300 pixels) to 20 pairs for larger images (over 1000 pixels), and values ; were determined by comparing various images of different resolutions and verifying the results of the comparisons for different threshold values.
  • the 15-pair value has been shown to be empirically adequate to be used as the threshold for any pair of images without significant impairment to method processing and reliability.
  • a threshold value of 15 point pairs was used, and the positive association found was obtained with 100% credibility.
  • the algorithm becomes more restrictive. For smaller and smaller values, the algorithm becomes more relaxed. For values between 10 and 20 pairs as threshold, the best results were observed.
  • process reliability and speed can be further enhanced by using image-associated metadata such as GPS data, photo registration time, device direction, etc., which may eventually be available if used.
  • image-associated metadata such as GPS data, photo registration time, device direction, etc., which may eventually be available if used.
  • a mobile device such as a smartphone or tablet. In these cases, the device used sends to the server the image to be identified and other available metadata (photo time, latitude and longitude, device direction, etc.).
  • This metadata is then used by the server to guide the search for database images that must first be compared to the input image, allowing you to speed up the input image identification process by reducing the number of comparisons required until a match is found. satisfactory.
  • database records that correspond to metadata images that have a greater degree of similarity to the metadata associated with the input image receive higher priority when compared to the input image.
  • the image recognition system comprises the application of the methods mentioned above, was combined with the image acquisition and processing equipment, namely at least one mobile device with camera and internet connection, and also a server for remote processing.
  • the image acquisition and processing equipment namely at least one mobile device with camera and internet connection, and also a server for remote processing.
  • Its general operation is illustrated in Figure 6 and consists of a mobile photography user identifying a place of interest (1), register the location by means of a photograph (2) and send your photo to a server (3), which remotely analyzes the photo and, in the case of a positive identified association, returns with the name, other identifying information, of the location of interest identified and additional information registered in the database (4).
  • FIGs 7, 8 and 9 illustrate the application of the invention in question to the identification of the S ⁇ o Francisco de Assis da Pampulha Church in Belo Horizonte.
  • Figure 7 illustrates the comparison between the image to be identified (7b) and the image from the database (7a), with the characteristic points marked, as well as the lines joining similar pairs of characteristic points at? Two images.
  • Figure 8 whose image to be identified (8b) is taken from another angle, and compared to the database image (8a), also illustrates the pairs of characteristic points joined by lines. In this situation, two straight lines are observed with angles very different from the median of the straight angles of all pairs of characteristic points found. The Point pairs of these two lines are discarded by the straight line acceptance criterion of the algorithm.
  • Figure 9 illustrates the situation that the tourist spot in question is photographed from its other end, highlighting the importance of the capture protocol for setting up a multi-image base. angles to a particular place of interest.
  • the image being identified (9b) is not associated with the database image (9a).
  • the capture angles are similar in both the database image (9c) and the image to be identified (9b), there is a positive identification of the place of interest.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé de construction d'une base d'images, un procédé de reconnaissance d'images, un système de reconnaissance d'images et ses utilisations. Elle relève du domaine des technologies de l'information, notamment dans les domaines de l'infographie, la vision informatisée, le traitement et la reconnaissance d'images, trouvant une application dans l'identification et dans la reconnaissance automatique de lieux d'intérêt à partir de sa représentation visuelle.
PCT/BR2016/000017 2015-05-28 2016-02-22 Procédé de construction d'une base d'images, procédé de reconnaissance d'images, système de reconnaissance d'images et ses utilisations Ceased WO2016187681A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
BR102015012437-6A BR102015012437B1 (pt) 2015-05-28 Método de reconhecimento de imagens, sistema de reconhecimento de imagens e seus usos
BRBR1020150124376 2015-05-28

Publications (1)

Publication Number Publication Date
WO2016187681A1 true WO2016187681A1 (fr) 2016-12-01

Family

ID=57392274

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/BR2016/000017 Ceased WO2016187681A1 (fr) 2015-05-28 2016-02-22 Procédé de construction d'une base d'images, procédé de reconnaissance d'images, système de reconnaissance d'images et ses utilisations

Country Status (1)

Country Link
WO (1) WO2016187681A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7565139B2 (en) * 2004-02-20 2009-07-21 Google Inc. Image-based search engine for mobile phones with camera
US20100260426A1 (en) * 2009-04-14 2010-10-14 Huang Joseph Jyh-Huei Systems and methods for image recognition using mobile devices
US7872669B2 (en) * 2004-01-22 2011-01-18 Massachusetts Institute Of Technology Photo-based mobile deixis system and related techniques
US20110123120A1 (en) * 2008-06-03 2011-05-26 Eth Zurich Method and system for generating a pictorial reference database using geographical information

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7872669B2 (en) * 2004-01-22 2011-01-18 Massachusetts Institute Of Technology Photo-based mobile deixis system and related techniques
US7565139B2 (en) * 2004-02-20 2009-07-21 Google Inc. Image-based search engine for mobile phones with camera
US20110123120A1 (en) * 2008-06-03 2011-05-26 Eth Zurich Method and system for generating a pictorial reference database using geographical information
US20100260426A1 (en) * 2009-04-14 2010-10-14 Huang Joseph Jyh-Huei Systems and methods for image recognition using mobile devices

Also Published As

Publication number Publication date
BR102015012437A8 (pt) 2022-09-20
BR102015012437A2 (pt) 2017-04-25

Similar Documents

Publication Publication Date Title
KR101617681B1 (ko) 히스토그램들을 갖는 다중 층 연결 요소들을 사용하는 텍스트 검출
Chen et al. Crowd map: Accurate reconstruction of indoor floor plans from crowdsourced sensor-rich videos
Ardeshir et al. GIS-assisted object detection and geospatial localization
WO2019218824A1 (fr) Procédé d'acquisition de piste de mouvement et dispositif associé, support de stockage et terminal
WO2019042426A1 (fr) Procédé et appareil de traitement de scène en réalité augmentée, et support de stockage informatique
US9390330B2 (en) Apparatus and method for extracting correspondences between aerial images
WO2018014828A1 (fr) Procédé et système de reconnaissance d'informations de localisation dans un code bidimensionnel
CN111310728B (zh) 基于监控相机和无线定位的行人重识别系统
WO2022237048A1 (fr) Procédé et appareil d'acquisition de pose, et dispositif électronique, support de stockage et programme
US11651288B2 (en) Learning data generation apparatus, change area detection method and computer program
CN111767839B (zh) 一种车辆行驶轨迹确定方法、装置、设备及介质
CN103488966A (zh) 一种能够识别实名制车票信息的智能手机
Tarrit et al. Vanishing point detection for visual surveillance systems in railway platform environments
KR102249380B1 (ko) 기준 영상 정보를 이용한 cctv 장치의 공간 정보 생성 시스템
Haider et al. Zebra crosswalk region detection and localization based on deep convolutional neural network
CN111242139B (zh) 一种基于点线特征的最大直方图异源图像匹配方法
WO2016187681A1 (fr) Procédé de construction d'une base d'images, procédé de reconnaissance d'images, système de reconnaissance d'images et ses utilisations
Tu et al. Automatic recognition of civil infrastructure objects in mobile mapping imagery using a markov random field model
CN103903269B (zh) 球机监控视频的结构化描述方法和系统
BR102015012437B1 (pt) Método de reconhecimento de imagens, sistema de reconhecimento de imagens e seus usos
KR102555668B1 (ko) 맵 생성 방법 및 이를 이용한 이미지 기반 측위 방법
Nowicki et al. Experimental evaluation of visual place recognition algorithms for personal indoor localization
Nüchter et al. Skyline-based registration of 3D laser scans
CN105930813B (zh) 一种在任意自然场景下检测行文本的方法
Moun et al. Localization and building identification in outdoor environment for smartphone using integrated GPS and camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16798968

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16798968

Country of ref document: EP

Kind code of ref document: A1