TW201808042A - Method and system for indoor positioning and device for creating indoor maps thereof - Google Patents

Abstract

An indoor positioning method, indoor positioning system and indoor maps creating device thereof are provided. The method includes converting a panorama image corresponding to an indoor environment into a sequence of perspective images, and capturing a plurality of reference feature points and descriptors of the reference feature points from the perspective images; taking a shooting position of the panorama image as an origin, and recoding a plurality of 3D reference coordinate values corresponding to a central position of each perspective images; calculating 3D coordinate values of the reference feature points relatively relating to the origin, and storing the 3D coordinate values and the descriptors of the reference feature points as an indoor map corresponding to the indoor environment. The method also includes positioning a 3D target coordinate value of a portable electronic device relatively relating to the origin in the indoor environment according to the indoor maps.

Description

Indoor positioning method and system and indoor map establishing device

The present invention relates to an indoor positioning method, and more particularly to an indoor positioning method, an indoor positioning system, and an indoor map establishing device for locating a portable electronic device in an indoor environment.

With the increasing popularity of wireless mobile devices, more and more users have completed various activities in their lives through mobile devices, and various wireless Internet environments have also increased. As the wireless environment matures, each device manufacturer also focuses on value-added applications in the wireless environment. One of them is location-aware services that provide location information. Through location-aware services, users can obtain various information and services in their vicinity. To quickly find your own goals. On the other hand, for merchants or service providers, it is desirable to utilize location awareness to push messages (e.g., advertisements) to users within their business premises (e.g., stores, restaurants, parking lots, department stores, etc.). Based on the above requirements, there must be an indoor positioning navigation system that can quickly locate and navigate indoors, so as to exchange information between merchants or service providers and users.

Current commercial positioning systems, such as the Global Positioning System (GPS), are basically used for outdoor positioning, which can achieve higher positioning accuracy and faster positioning speed in a well-signaled environment. Since the satellite signals are greatly attenuated indoors, these outdoor positioning systems are more difficult to apply in indoor positioning situations. The three-dimensional object positioning or three-dimensional object tracking technology currently applied indoors is also limited by the inability to receive GPS signals, and the user device having wireless communication functions (for example, Bluetooth, WIFI, etc.) transmits signals according to the signal strength index values. To determine the distance from the surrounding objects, or to configure the wireless communication function tag on the indoor object to use the short-range wireless communication technology to read and write related data through radio signals and identify specific targets. However, since the three-dimensional object localization technology is also realized based on the signal reception and the strength of the signal, it is also easily limited by the influence of the environment, resulting in unstable signals, thereby affecting the correctness of the three-dimensional object positioning. In addition, the RFID function can only identify a specific target through the tag, and cannot determine the distance between the user device and the surrounding object, and the number of tag configurations is also limited by the cost budget.

Furthermore, the use of a base station for positioning is also a common method. For example, a positioning device (such as a user's mobile device) can calculate the current position by calculating the time at which the signal arrives at the base station. However, since the error of this method may be as high as several hundred meters, it is also not suitable for use in indoor positioning situations.

Based on the above, the positioning method based on the wireless network is more suitable for a barrier-free outdoor space such as a spacious outdoor or large stadium, and the positioning effects in buildings such as indoors and corridors are not good. Moreover, the above various techniques are more suitable for planar positioning, and cannot achieve accurate three-dimensional positioning. Obviously, the above various positioning technologies cannot meet the needs of indoor positioning, so there is still no indoor positioning system that can be widely used by the public.

The invention provides an indoor positioning method, an indoor positioning system and an indoor map establishing device thereof, wherein an indoor map is quickly built by a computer and uploaded to a server end in an offline state, and is portable at a user end in a connected state. By downloading the indoor map, the electronic device can locate the portable electronic device through the instant image and the indoor map captured by the portable electronic device, thereby improving the quality and efficiency of the user's instant access service. More efficient delivery of user-specific information, thereby enhancing the user's more convenient operating experience.

An exemplary embodiment of the present invention provides an indoor positioning method for locating a portable electronic device. The indoor positioning method includes: converting a ring image corresponding to an indoor environment into a plurality of fluoroscopic images, and capturing the same a plurality of reference feature points in the fluoroscopic image and a description matrix of the reference feature points; taking a shooting position of the shooting ring image as an origin, recording a plurality of three-dimensional reference coordinate values corresponding to a central position of each fluoroscopic image; The three-dimensional reference coordinate values calculate a three-dimensional coordinate value of the reference feature point relative to the origin, and store a three-dimensional coordinate value of the reference feature point and a description matrix of the reference feature point as an indoor environment corresponding to the indoor environment a map; and locating a three-dimensional target coordinate value of the portable electronic device relative to the origin in the indoor environment according to the indoor map.

In an exemplary embodiment of the present invention, the step of recording a three-dimensional reference coordinate value corresponding to a center position of each fluoroscopic image by taking a shooting position of the shooting ring image as an origin includes: obtaining a center position of each fluoroscopic image A plurality of reference pixels in the direction of the horizontal axis and the direction of the horizontal axis, and the three-dimensional coordinate values of the reference pixels are recorded as three-dimensional reference coordinate values.

In an exemplary embodiment of the present invention, each of the reference feature points corresponds to a feature pixel in the fluoroscopic image, and the three-dimensional coordinate value of the feature point relative to the origin is calculated according to the three-dimensional reference coordinate value. The step of: obtaining a first reference pixel corresponding to the feature point pixel in the vertical axis direction and a second reference pixel corresponding to the feature point pixel in the horizontal axis direction; and according to the first reference The three-dimensional reference coordinate value of the pixel and the three-dimensional reference coordinate value of the second reference pixel calculate a three-dimensional coordinate value of the feature point pixel.

In an exemplary embodiment of the present invention, the operation of establishing an indoor map is performed in an offline mode, and the offline mode is a state that is not connected to the Internet.

In an exemplary embodiment of the present invention, the indoor map is stored in a positioning database, and the step of locating the three-dimensional target coordinate value of the portable electronic device relative to the origin in the indoor environment according to the indoor map includes: Obtaining an instant image through the portable electronic device, and capturing a plurality of target feature points in the instant image and a description matrix of the target feature points; comparing the description matrix of the reference feature point with the description of the target feature point a matrix, and when a difference value between the description matrix of the reference feature point and the description matrix of the target feature point conforms to a preset tolerance value, is taken from a positioning database and a description matrix of the target feature point The difference value conforms to the three-dimensional coordinate value of the reference feature point of the preset tolerance value; and calculates the three-dimensional target coordinate value and the rotation angle of the portable electronic device by using the three-dimensional coordinate value of the reference feature point taken out from the positioning database .

In an exemplary embodiment of the present invention, the operation of positioning the portable electronic device in the indoor environment relative to the three-dimensional target coordinate value of the origin is performed in the connection mode, and the connection mode is the Internet. The state of the connection. The step of comparing the description matrix of the reference feature point with the description matrix of the target feature point further comprises: determining an indoor environment in which the portable electronic device is located through the Internet, and downloading an indoor corresponding to the indoor environment A map to the portable electronic device.

An exemplary embodiment of the present invention provides an indoor positioning system, including: a photographing device, a portable electronic device, and an indoor map establishing device. The photographing device is configured to capture a ring image corresponding to an indoor environment, and the indoor map establishing device is connected to the photographing device. The indoor map establishing device includes: a storage device and a processor. The storage device stores a positioning database, and the processor is coupled to the storage device, and is configured to convert the ring image into a plurality of fluoroscopic images, and capture a plurality of reference feature points in the fluoroscopic images and the Refer to the description matrix of the feature points. The processor is further configured to take a shooting position of the ring image as an origin, and record a plurality of three-dimensional reference coordinate values corresponding to the center position of each of the fluoroscopic images. In addition, the processor is further configured to calculate a three-dimensional coordinate value of the reference feature point relative to the origin according to the three-dimensional reference coordinate values, and the three-dimensional coordinate value of the reference feature point and the description matrix of the reference feature point are The indoor map of the indoor environment is stored in the location database. The portable electronic device is configured to locate a three-dimensional target coordinate value of the portable electronic device relative to the origin in the indoor environment according to the indoor map.

In an exemplary embodiment of the present invention, the processor is further configured to obtain a plurality of reference pixels in a direction of a longitudinal axis and a direction of a horizontal axis of a center position of each fluoroscopic image, and to use the reference pixels The three-dimensional coordinate value is recorded as the three-dimensional reference coordinate value.

In an exemplary embodiment of the present invention, each of the reference feature points corresponds to a feature pixel in the fluoroscopic image, and the processor is further configured to obtain a corresponding pixel of the feature point in the vertical axis direction. A reference pixel and a second reference pixel corresponding to the feature point pixel in the horizontal axis direction. Moreover, the processor is further configured to calculate a three-dimensional coordinate value of the feature point pixel according to the three-dimensional reference coordinate value of the first reference pixel and the three-dimensional reference coordinate value of the second reference pixel.

In an exemplary embodiment of the present invention, the portable electronic device is further configured to obtain an instant image, and capture a plurality of target feature points and a description matrix of the target feature points in the instant image. The portable electronic device is further configured to determine an indoor environment in which the portable electronic device is located through the Internet, and download an indoor map corresponding to the indoor environment to the portable electronic device. The portable electronic device is further configured to compare a description matrix of the reference feature point with a description matrix of the target feature point, and a description matrix of the reference feature point and a description matrix of the target feature point When the difference value meets a preset tolerance value, the three-dimensional coordinate value of the reference feature point that matches the description matrix of the target feature points from the positioning database is in accordance with the preset tolerance value. Moreover, the portable electronic device is further configured to calculate the three-dimensional target coordinate value and the rotation angle of the portable electronic device by using the three-dimensional coordinate value of the reference feature point taken out from the positioning database.

In an exemplary embodiment of the present invention, the processor operates in an offline mode and the portable electronic device operates in a connection mode, wherein the offline mode is a state that is not connected to the Internet, and the connection mode is The state of being connected to the Internet.

An exemplary embodiment of the present invention provides an indoor map establishing device including a storage device and a processor. The storage device stores a positioning database and a plurality of modules. The processor is coupled to the storage device, loads and executes a plurality of modules stored in the storage device, and the processor operates in an offline mode. The module includes: an input module, an image processing module, a feature capture module, and an operation module. The input module is configured to receive a ring image corresponding to the indoor environment; the image processing module is configured to convert the ring image into a plurality of perspective images; and the feature capturing module is configured to capture a plurality of reference features in the perspective image And a description matrix of the reference feature points; and the operation module is configured to record a plurality of three-dimensional reference coordinate values corresponding to the center position of each of the fluoroscopic images with the shooting position of the image of the ring image as an origin. In addition, the computing module is further configured to calculate a three-dimensional coordinate value of the reference feature point relative to an origin according to the three-dimensional reference coordinate value, and describe a three-dimensional coordinate value of the reference feature point and the reference feature point. The indoor map of the corresponding indoor environment composed of the matrix is stored in the positioning database.

In an exemplary embodiment of the present invention, the computing module is further configured to obtain a plurality of reference pixels in a direction of a longitudinal axis and a direction of a horizontal axis of a center position of each fluoroscopic image, and use the reference pixels The three-dimensional coordinate value is recorded as the three-dimensional reference coordinate value.

In an exemplary embodiment of the present invention, each of the reference feature points corresponds to a feature pixel in the fluoroscopic image, and the operation module is further configured to obtain a pixel corresponding to the feature point in the vertical axis direction. The first reference pixel and the second reference pixel corresponding to the feature point pixel in the horizontal axis direction. And the computing module is further configured to calculate a three-dimensional coordinate value of the feature point pixel according to the three-dimensional reference coordinate value of the first reference pixel and the three-dimensional reference coordinate value of the second reference pixel.

In an exemplary embodiment of the present invention, the offline mode is a state that is not connected to the Internet.

Based on the above, the above exemplary embodiment is a three-dimensional reference coordinate value of a plurality of reference pixels in the vertical axis direction and the horizontal axis direction by the center position of the fluoroscopic image developed by recording only the ring image in the offline mode. The technical solution achieves an effective saving of the memory space of the storage device, and improves the performance of obtaining the three-dimensional coordinates of each feature point in the fluoroscopic image, thereby rapidly establishing the indoor map. Accordingly, the portable electronic device can quickly locate the portable electronic device of the user in the connection mode by comparing the instant image and the indoor map captured by itself.

The above described features and advantages of the invention will be apparent from the following description.

The present invention provides a user with the ability to quickly and easily find their own destination or object in an indoor environment, and to provide a function for the operator to exchange data with the user through the positioning and navigation functions in an indoor environment. Offline analysis of the 360-degree circular image (also known as panoramic image) to build an indoor map, and compare the images and indoor maps captured by the user's portable electronic device through image recognition technology, thereby positioning and using Portable electronic device. Therefore, the positioning of the portable electronic device is not limited to the situation that the signal is unstable and the positioning is incorrect due to the wireless network and the space barrier. Based on this, the quality and efficiency of the user's instant access to the location service are effectively improved.

FIG. 1 is a schematic diagram of an indoor positioning system according to an embodiment of the invention. Referring to FIG. 1 , the indoor positioning system 100 includes an indoor map establishing device 200 , a photographing device 110 , and a portable electronic device 120 . The photographing device 110 is, for example, a catadioptric camera composed of a convex lens and a monocular camera. However, the present invention is not limited thereto. For example, the photographing device 110 may be another panorama that can capture a 360-degree surround image. Omni-directional camera. The portable electronic device 120 can be a mobile device, a personal digital assistant (PDA), a tablet computer, or the like, or can be connected to the Internet 102 using a wireless communication network, and communicate with the indoor map establishing device 200. Electronic devices such as data transmission. In addition, the Internet 102 is, for example, a Wireless Fidelity (WiFi) network or a Global System for Mobile (GSM) network. However, it must be understood that the Internet 102 can also be other suitable network communication protocols, and the invention is not limited.

For example, the indoor map establishing device 200 is operated in an offline mode, and the indoor map establishing device 200 receives the ring image from the photographing device 110 to construct an indoor map according to the ring image; and the portable electronic device 120 will be operated in a connection mode to download an indoor map from the indoor map establishing device 200 via the Internet 102 to perform a positioning operation on the portable electronic device 120, wherein the offline mode is not with the Internet 102 The state of the connection, and the connection mode is a state of being connected to the Internet 102. It should be understood that the present example is described by taking a portable electronic device as an example, but the present invention is not limited thereto. For example, the indoor positioning system 100 can include more portable electronic devices, and each portable electronic device can perform positioning calculation to obtain a three-dimensional coordinate value in the indoor environment.

2 is a block diagram of an indoor map establishing apparatus according to an embodiment of the invention. Referring to FIG. 2, the indoor map establishing device 200 includes a storage device 202 and a processor 204. In the present embodiment, the indoor map establishing device 200 may be a server or a computer system having an arithmetic function.

The storage device 202 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, Solid State Drive (SSD) or similar components or a combination of the above. In this embodiment, the storage device 202 is configured to store software programs such as the input module 210, the image processing module 220, the feature capturing module 230, the computing module 240, and the positioning database 250.

The processor 204 is coupled to the storage device 202. The processor 204 can be a central processing unit (CPU) with a single core or multiple cores, or other programmable general purpose or special purpose microprocessor (Microprocessor), digital signal processor (Digital Signal) Processor, DSP), programmable controller, Application Specific Integrated Circuit (ASIC) or other similar components or a combination of the above. In the exemplary embodiment, the processor 204 is configured to access and execute the input module 210, the image processing module 220, the feature capturing module 230, the computing module 240, and the positioning database recorded in the storage device 202. 250, by which the indoor positioning method of the embodiment of the present invention is implemented. In order to more clearly describe the operation of the indoor positioning system 100 of the present invention and its indoor map establishing device 200 and portable electronic device 120, an example will be described below with reference to FIGS. 1, 2 and 3.

FIG. 3 is a flow chart of an indoor positioning method according to an embodiment of the invention. Referring to FIG. 1 to FIG. 3 simultaneously, the input module 210 of the indoor map establishing device 200 receives a ring image from the photographing device 110, and in step S301, the image processing module 220 receives the image received from the input module 210. The ring image is converted into a plurality of fluoroscopic images, and the feature capturing module 230 captures a plurality of feature points (also referred to as reference feature points) in the fluoroscopic images and descriptions of the reference feature points. Descriptors.

Specifically, the ring image can be divided into a spherical panoramic image, a cylindrical panoramic image, and a tetragonal panoramic image. In the present exemplary embodiment, the circular field image captured by the imaging device 110 belongs to a cylindrical panoramic image. Since the cylindrical panoramic image can be expanded into a fluoroscopic image with only curvature changes on the left and right borders, the cylindrical panoramic image can retain the original state of most of the feature points of the picture.

In the exemplary embodiment of the present invention, the target feature points of the real-time image captured by the portable electronic device 120 of the user and the reference feature points of the ring image converted into the fluoroscopic image are used, and then the comparison is performed according to the comparison. The three-dimensional coordinates of the same feature points are used to locate the portable electronic device 120. Since the live image captured by the portable electronic device 120 is a Cartesian Coordinate System, and the cylindrical panoramic image is a Cylindrical Coordinate System, when the two are performed on the two When the comparison is made, the number of the calculated feature points will not be completely consistent with the position due to the difference in the coordinate system, and the inaccurate comparison result will be obtained. In addition, because a ring image cannot be directly converted into an entire image belonging to the Cartesian coordinate system. Accordingly, in the above step S301, the image processing module 220 converts the received surround image into a plurality of fluoroscopic images.

4A-4D are schematic diagrams of converting a ring image into a plurality of fluoroscopic images according to an embodiment of the invention.

Referring to FIGS. 4A-4D, the center point of the ring image 400 captured by the photographing device 110 is the photographing position where the photographing device 110 is located. In other words, as shown in FIG. 4A, the photographing position of the photographing device 110 is the loop image. The origin 410 of 400. In the exemplary embodiment, as shown in FIG. 4B, the image processing module 220 cuts the ring image 400 into eight strip-shaped fluoroscopic images 400a-400h of equal width, and each strip-shaped fluoroscopic image will cover A 45-degree horizontal view (Field-of-View) seen by the origin 410 of the cylindrical toroidal image 400. It is worth mentioning that, in the exemplary embodiment, the viewing angle based on a commercially available mobile device (for example, a smart phone) is about 45 degrees, so a 45-degree horizontal viewing angle is used as a cutting standard, but the present invention does not intend to The size of the horizontal viewing angle as the cutting standard is limited. For example, in another exemplary embodiment, the cutting standard may be set to a horizontal viewing angle greater than or less than 45 degrees depending on actual needs.

As described above, since the live image captured by the portable electronic device 120 is a Cartesian coordinate system, and the cylindrical panoramic image is a cylindrical coordinate system, the process of expanding the cylindrical annular image 400 into a fluoroscopic image is performed. In the middle, it will involve the conversion of two coordinate systems. As shown in FIG. 4C, taking the fluoroscopic image 400a as an example, the top and bottom of the fluoroscopic image 400a converted to the Cartesian coordinate system exhibit a concave region, which also causes the left and right boundaries of the fluoroscopic image 400a to have a curvature variation. For example, in the fluoroscopic image 400a, there will be a strip line (not shown) composed of pixels that have not been filled with gray color, which is called an Aliasing effect. In view of this, in an exemplary embodiment of the present invention, the image processing module 220 performs image processing on the fluoroscopic image 400a to assign a strip line composed of the pixels of the gray-filled color to the left and right adjacent pixels. average value.

In particular, in an exemplary embodiment of the present invention, the image processing module 220 generates two sets of the expanded perspective image 420 and the expanded perspective image 420 of the corresponding toroidal image 400 in the operation of converting the ring image 400 into a plurality of perspective images. '(As shown in FIG. 4D), wherein the unfolded fluoroscopic image 420 is obtained by unfolding the fluoroscopic images 400a-400h cut according to the cutting angle shown in FIG. 4B, and the unfolded fluoroscopic image 420' is compared with FIG. 4B. The oscillating images 400i~400p obtained by cutting the cutting angles with a cutting angle of 22.5 degrees are obtained. By performing the fluoroscopic images (ie, unfolding the fluoroscopic image 420 and expanding the fluoroscopic image 420 ′) by the two groups, the alignment inaccuracy caused by the curvature variation of the left and right boundaries of each fluoroscopic image can be improved, and the subsequent fluoroscopic images are added. The accuracy of the operation of the feature points. For example, taking the fluoroscopic image 400b as an example, the problem of inaccurate alignment caused by the curvature modification of the left and right boundaries can be compensated by using the reference feature points generated by the fluoroscopic image 400i and the fluoroscopic image 400j.

In addition, in the foregoing step S301, the operation of the plurality of reference feature points in the fluoroscopic images 400a-400h and the description matrix of the reference feature points performed by the feature capture module 230 is performed, for example, by using a BRIEF (Binary Robust) Independent Elementary Features), ORB (Oriented FAST and Rotated BRIEF) or BRISK (Binary Robust Invariant Scalable Keypoints) perform a feature extraction operation using a binary string as an algorithm for describing a matrix vector. Specifically, the BRIEF algorithm selects a plurality of pixel pairs in the vicinity of the feature points, and compares the gray values of the pixel pairs to combine the comparison results into a two-dimensional string for describing Feature points. Also, the Hamming Distance is used to calculate whether the feature description matrix matches. However, the ORB algorithm improves the feature points of theRIEF algorithm without rotation invariance, scale invariance and noise-sensitive defects. It performs Gaussian blur on the image and generates scale space to make the feature points have scale invariance. Then, the Moment vector operation is performed on the feature points, so that the feature points have direction invariance. Finally, the feature points are described by the BRIEF feature, so that the obtained feature points have rotation invariance, scale-scale invariance, and are not susceptible to Noise interference and other advantages. Similarly, the BRISK algorithm is also based on the improved algorithm of the feature points with scale invariance and rotation invariance.

Next, in step S303, the computing module 240 records the plurality of three-dimensional coordinate values corresponding to the center position of each of the fluoroscopic images 400a-400h (also referred to as a three-dimensional reference) with the shooting position of the shooting ring image 400 as the origin 410. Coordinate value). In other words, the three-dimensional reference coordinate value recorded by the computing module 240 is a relative coordinate position relative to the three-dimensional coordinate value of the origin 410. Here, the plurality of three-dimensional reference coordinate values corresponding to the center position of each of the fluoroscopic images 400a to 400h refer to a plurality of vertical axis directions and horizontal axis directions of the center position of each of the fluoroscopic images 400a to 400h. The coordinate value corresponding to the pixel (also known as multiple reference pixels).

FIG. 5 is a schematic diagram of recording a plurality of three-dimensional reference coordinate values corresponding to a center position of a fluoroscopic image according to an embodiment of the invention.

Referring to FIG. 5 , in the exemplary embodiment, the fluoroscopic image 400 d developed by the ring image 400 is taken as an example, and the computing module 240 obtains the vertical axis direction 502 and the horizontal axis direction 504 of the center position 500 of the fluoroscopic image 400 d . Multiple reference pixels on the top, and only record the three-dimensional reference coordinate values of these reference pixels. Similarly, the computing module 240 will also record only a plurality of reference pixels on the longitudinal axis direction 502 and the horizontal axis direction 504 of the center position 500 of the other perspective images (ie, the fluoroscopic images 400a-400c, 400e-400h). 3D reference coordinate value. Since the computing module 240 does not need to record the three-dimensional coordinate values of all the pixels in each fluoroscopic image, the memory space of the storage device 202 can be effectively saved.

In particular, in the process of expanding the cylindrical toroidal image 400 into a plurality of fluoroscopic images 400a-400h, the reference pixel on the vertical axis direction 502 does not actually change in the coordinate system conversion operation. The reference pixel on the horizontal axis direction 504 has only a slight variation in the reference pixel near the edge in the coordinate system conversion operation. In other words, the reference pixel on the vertical axis direction 502 and the horizontal axis direction 504 of the center position 500 of each fluoroscopic image can be regarded as a pixel that does not have a curvature change in the fluoroscopic image. Therefore, the operation module 240 can further The three-dimensional coordinate values of the plurality of reference feature points in the fluoroscopic image captured by the feature capturing module 230 are estimated by linear interpolation by using the three-dimensional coordinate values of the pixels without the curvature variation. For example, in step S305, the operation module 240 calculates a three-dimensional coordinate value of the reference feature point relative to the origin 410 according to the three-dimensional reference coordinate value, and describes the three-dimensional coordinate value of the reference feature point and the reference feature point. An indoor map of the corresponding indoor environment composed of the matrix is stored in the location database 250. Here, the indoor map described in the exemplary embodiment of the present invention is composed of feature points having three-dimensional coordinate values and a description matrix thereof, and the three-dimensional coordinate values of the feature points are three-dimensional coordinate values with respect to the origin 410. Relative coordinate position.

FIG. 6 is a schematic diagram of calculating a three-dimensional coordinate value of a feature point according to a three-dimensional reference coordinate value of a corresponding center position in a fluoroscopic image according to an embodiment of the invention.

In detail, each of the reference feature points captured by the feature capture module 230 from the fluoroscopic image belongs to a pixel in the fluoroscopic image, that is, each reference feature point corresponds to one of the fluoroscopic images. Prime (also known as feature dot pixels). Here, the perspective image 400d developed by the ring image 400 is taken as an example to illustrate how to obtain the three-dimensional coordinate value of the feature point. Referring to FIG. 6 , it is assumed that the feature pixel 500a in the fluoroscopic image 400d is a feature point captured by the feature capture module 230, and the operation module 240 obtains the feature point corresponding to the vertical axis direction 502. a reference pixel 500b (also referred to as a first reference pixel 500b) of the pixel 500a and a reference pixel 500c (also referred to as a second reference pixel 500c) corresponding to the feature pixel 500a in the horizontal axis direction 504, and The three-dimensional reference coordinate value of the first reference pixel 500b and the three-dimensional reference coordinate value of the second reference pixel 500c have been recorded by the arithmetic module 240 in step S303. Accordingly, the operation module 240 can quickly calculate the three-dimensional coordinate value of the feature point pixel 500a according to the three-dimensional reference coordinate value of the first reference pixel 500b and the three-dimensional reference coordinate value of the second reference pixel 500c.

FIG. 7A is a schematic diagram showing the correspondence between a cylindrical coordinate system and a Cartesian coordinate system according to an embodiment of the invention. 7B is a top plan view of a ring image as illustrated in accordance with an embodiment of the invention. 7C is a side view of a ring image as depicted in accordance with an embodiment of the invention. 7D is a schematic diagram showing the symmetry of a cylindrical coordinate system according to an embodiment of the invention.

Referring to FIG. 7A to FIG. 7C , FIG. 7A illustrates the correspondence between the cylindrical coordinate system of the ring image 400 and the Cartesian coordinate system in the above exemplary embodiment, and FIG. 7B and FIG. 7C respectively show the top view of the ring image 400. And a side view, here, the perspective image 400a after the ring image 400 is expanded is taken as an example, wherein the cylindrical coordinates ( r , θ, h ) on the ring image 400 correspond to the fluoroscopic image of the ring image 400 . The three-dimensional coordinates ( x , y , z ) of the Cartesian coordinate system on the 400a. In step S303 of the above exemplary embodiment of the present invention, the plurality of three-dimensional reference coordinate values in each of the fluoroscopic images 400a-400h recorded by the computing module 240 are obtained by the following equations (1) to (3). of.

...equation (1)

... equation (2)

... equation (3)

Hereinafter, how the present invention derives equations (1) to (3) will be described in detail with reference to Figs. 7A to 7D.

Referring to FIG. 7A to FIG. 7D, since the circumference of the cylindrical ring image 400 is the width of the ring image 400, the equation of the circumference length is Where c represents the perimeter of the cylindrical toroidal image 400. Therefore, the radiation distance r of the cylindrical coordinates ( r , θ , h) can be expressed as the following equation (4).

... equation (4)

Next, in an exemplary embodiment of the present invention, it is assumed that the arc length of the azimuth angle θ of the cylindrical coordinate system is l , and the equation of the arc length is . Therefore, the azimuth angle θ of the cylindrical coordinate system can be expressed as the following equation (5).

... equation (5)

In particular, based on the symmetry of the cylinder, therefore, as shown in Fig. 7D, the value of θ will be between -22.5 degrees and 22.5 degrees, that is, -22.5 ̊ < θ < 22.5 ̊. As described above, the exemplary embodiment of the present invention uses a 45 degree horizontal viewing angle as a cutting standard to cut the ring image 400 into eight equal-width strip-shaped fluoroscopic images 400a-400h, according to which the angle covered by 22.5 ̊ Half of the image for each strip of perspective.

Then, according to equations (4), (5), and by the standard trigonometric calculation and the linear proportional relationship, the above equations (1) to (3) can be further calculated to obtain the cylindrical coordinates of the ring image 400 ( The Cartesian coordinates (x, y, z) corresponding to r, θ , h). Specifically, in the exemplary embodiment, the image processing module 220 sequentially scans the pixels on the ring image 400 corresponding to the fluoroscopic image 400a during the process of converting the ring image 400 into the fluoroscopic image 400a. The pixels are sequentially converted into coordinates, wherein the scanning order of the image processing module 220 is from left to right and from top to bottom. Since the scanning order is from left to right and from top to bottom, the Cartesian coordinates ( x , y , z ) are calculated in the order of x , y , z .

Referring to FIG. 7B again, L in FIG. 7B is the distance from the coordinate origin to the fluoroscopic image 400a when the image processing module 220 scans the fluoroscopic image 400a to the angle β. ,therefore, . Next, the following equation (6) of x can be obtained from the trigonometric function.

... equation (6).

In addition, in the present exemplary embodiment, the angle of the center line of each fluoroscopic image 400a~400h is 45 degrees, that is, α=45 ̊, therefore, equation (6) can be rewritten as Thus, the above equation (1) is obtained. It should be noted that n in the equation (1) is represented as the nth fluoroscopic image. For example, in this example, the image processing module 220 is to cut the circular field image 400 into eight equal-width strip-shaped fluoroscopic images 400a. ~400h, that is, the 0th perspective image is the perspective image 400a, the first perspective image 400b, and so on, the 2nd to 7th perspective images are the perspective images 400c~400h, in other words, in this example In the embodiment, n =0, 1, 2, 3, 4, 5, 6, 7.

Similarly, according to the trigonometric function, the above equation of y can be obtained. That is, equation (2).

Finally, again referring to Figure 7C, the linear relationship between the ratio of z to h can be determined in Equation (3) of z. due to ,therefore, That is, equation (3).

It is to be noted that, in the above steps S301 to S305, the indoor map establishing device 200 performs an operation of building an indoor map in an offline mode, and the indoor map is stored in the positioning database 250. Thereby, the positioning calculation of the portable electronic device 120 can be completed by the portable electronic device 120. Specifically, referring to FIG. 3, in step S307, the portable electronic device 120 locates the portable electronic device 120 relative to the origin in the indoor environment according to the indoor map established in steps S301-S305. The three-dimensional coordinate value of 410 (also known as the three-dimensional target coordinate value). In other words, the posture of the portable electronic device 120 (ie, the three-dimensional coordinate value and the rotation angle of the portable electronic device 120) need not be calculated by the indoor map establishing device 200 at the server end. In addition, the indoor map establishing apparatus 200 in the exemplary embodiment of the present invention constructs an indoor map by using a 360-degree circular field image obtained by off-line analysis, and accordingly, the technology of the present invention rebounds and points through a laser compared with the conventional one. Cloud technology, signal strength indicator values, GPS and other technologies that need to be paired with the Internet to build indoor and outdoor maps not only effectively save computing resources for building indoor maps, but also improve the accuracy of indoor maps.

In the exemplary embodiment, the portable electronic device 120 is operated in a connection mode, so that the portable electronic device 120 can continuously obtain the current real-time image and capture multiple features in the instant image. A description matrix of points (also known as target feature points) and such target feature points. It should be noted that, in the exemplary embodiment of the present invention, for convenience of description, the feature points captured by the indoor map establishing device 200 from the fluoroscopic images 400a-400h deployed by the ring image 400 are referred to as reference feature points, and The feature points captured by the portable electronic device 120 from the live image are referred to as target feature points.

Specifically, when the user holds his portable electronic device 120 and wants to enter a specific indoor environment (for example, a convenience store), the portable electronic device 120 determines the portable electronic device through the Internet 102. The indoor environment in which the device 120 is located is downloaded from the indoor map establishing device 200 to the indoor map corresponding to the indoor environment to the portable electronic device 120. For example, the portable electronic device 120 knows which convenience store the user currently wants to enter through the GPS, and downloads the indoor map belonging to the convenience store from the location database 250 of the indoor map establishing device 200 on the server side to the user. The portable electronic device 120 at the end.

Next, the portable electronic device 120 compares the description matrix of the reference feature point with the description matrix of the target feature point, and the difference value between the description matrix of the reference feature point and the description matrix of the target feature point. When a preset tolerance value is obtained, the three-dimensional coordinate value of the reference feature point whose difference value from the description matrix of the target feature points is in accordance with the preset tolerance value is taken out from the positioning database 250. For example, the BRISK algorithm uses a circular ring structure to sample around the feature points, and can further calculate the description matrix of the feature points by the difference in grayscale values between the two sample points. The comparison value of the sample points of the innermost circle has a decisive influence on the similarity of the two feature description matrices, that is, the recognition characteristics corresponding to the comparison values of the sample points from the innermost circle to the outermost circle are from high to low. And in the description matrix, the bit values corresponding to the sample point comparison values of the innermost circle to the outermost circle are sequentially recorded.

The operation of comparing the description matrix of the reference feature point with the description matrix of the target feature point is to utilize the Hamming distance to determine the similarity between the two description matrices, in particular, in the present exemplary embodiment. The portable electronic device 120 calculates the Hamming distance between the description matrix of the reference feature point and the description matrix of the target feature point by the inner circle to the outer circle. For example, when the portable electronic device 120 determines two If the Hamming distance of the byte in the current circle of the description matrix is less than the preset tolerance value, the similarity of the two description matrices of the next circle (ie, the next outer circle of the current circle) is further calculated, for example When the difference value between each bit value of the description matrix of the reference feature point and each circle bit value of the description matrix of the target feature point (ie, the Hamming distance) is consistent with the preset tolerance value, The portable electronic device 120 removes the three-dimensional coordinate value of the reference feature point that is different from the description matrix of the target feature points from the positioning database 250 according to the preset tolerance value; otherwise, the portable electronic device 120 Determine two descriptions The operations in which the matrices are not similar and stop determining the similarity of the two description matrices.

Then, the portable electronic device 120 can calculate the three-dimensional target coordinate value and the rotation angle of the portable electronic device 120 by using the three-dimensional coordinate values of the reference feature points taken out from the positioning database 250. For example, in the exemplary embodiment of the present invention, the portable electronic device 120 firstly carries the three-dimensional coordinate value of the reference feature point that meets the preset tolerance value between the description matrix of the target feature points into the EPnP. The Ransac algorithm is used to filter out the erroneous three-dimensional coordinate values to obtain the three-dimensional target coordinate value and the rotation angle of the portable electronic device 120.

As can be seen from the above, the exemplary embodiments of the present invention can be divided into an operation of establishing an indoor map (also referred to as a first stage) performed by the indoor map establishing device 200 in an offline mode, and connected by the portable electronic device 120. The positioning operation performed by the mode (also known as the second phase). It is worth mentioning that, by the technical solution of recording the three-dimensional reference coordinate values of the plurality of reference pixels in the longitudinal direction and the horizontal axis direction of the central position of the fluoroscopic image in the first stage, the storage can be effectively saved. The memory space of the device can further achieve the three-dimensional coordinates of each feature point in the fluoroscopic image by using the reference coordinate values to record the indoor map formed by the three-dimensional coordinates of the feature points in the positioning database. . Moreover, by storing the indoor map corresponding to different indoor environments in the first stage, the portable electronic device can download only the indoor map corresponding to the current indoor environment in the second stage to compare with the captured instant image. In particular, in the second stage, the portable electronic device can quickly filter out the non-conforming description matrix by comparing the description matrix of the indoor map and the instant image according to the characteristics of the description matrix of the feature points. The three-dimensional coordinate value and the rotation angle of the posture of the portable electronic device (ie, the portable electronic device 120) are efficiently and accurately located.

In summary, the indoor positioning method, the indoor positioning system, and the indoor map establishing device thereof according to the embodiments of the present invention can analyze the obtained 360-degree circular field image in an offline mode, and quickly construct an indoor map; In addition, the portable electronic device can locate the user's portable electronic device in the connection mode by comparing the instant image and the indoor map captured by itself. On the other hand, the portable electronic device of the present invention can quickly filter out a description matrix that does not match between the indoor map and the live image to quickly locate the posture of the portable electronic device, thereby providing an efficient manner. Give users the corresponding information, which will enhance the user's more convenient operation experience.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

100‧‧‧ indoor positioning system

102‧‧‧Internet

110‧‧‧Photographing device

120‧‧‧Portable electronic devices

200‧‧‧ indoor map building device

202‧‧‧Storage device

204‧‧‧ Processor

210‧‧‧Input module

220‧‧‧Image Processing Module

230‧‧‧Feature capture module

240‧‧‧ Computing Module

250‧‧‧Location database

S301, S303, S305, S307‧‧‧ steps of indoor positioning method

400‧‧‧ ring image

410‧‧‧ origin

400a~400h, 400i~400p‧‧‧ perspective image

420, 420’ ‧ ‧ Vision image

500‧‧‧ central location

502‧‧‧ vertical axis direction

504‧‧‧ horizontal axis direction

500a‧‧‧Characteristics

500b‧‧‧ first reference pixel

500c‧‧‧second reference pixel

FIG. 1 is a schematic diagram of an indoor positioning system according to an embodiment of the invention. 2 is a block diagram of an indoor map establishing apparatus according to an embodiment of the invention. FIG. 3 is a flow chart of an indoor positioning method according to an embodiment of the invention. 4A-4D are schematic diagrams of converting a ring image into a plurality of fluoroscopic images according to an embodiment of the invention. FIG. 5 is a schematic diagram of recording a plurality of three-dimensional reference coordinate values corresponding to a center position of a fluoroscopic image according to an embodiment of the invention. FIG. 6 is a schematic diagram of calculating a three-dimensional coordinate value of a feature point according to a three-dimensional reference coordinate value of a corresponding center position in a fluoroscopic image according to an embodiment of the invention. FIG. 7A is a schematic diagram showing the correspondence between a cylindrical coordinate system and a Cartesian coordinate system according to an embodiment of the invention. 7B is a top plan view of a ring image as illustrated in accordance with an embodiment of the invention. 7C is a side view of a ring image as depicted in accordance with an embodiment of the invention. 7D is a schematic diagram showing the symmetry of a cylindrical coordinate system according to an embodiment of the invention.

Claims (15)

An indoor positioning method for locating a portable electronic device, the indoor positioning method includes: converting a ring image corresponding to an indoor environment into a plurality of fluoroscopic images, and capturing a plurality of reference features in the fluoroscopic images Pointing a description matrix of the reference feature points; taking a shooting position of the ring image as an origin, and recording a plurality of three-dimensional reference coordinate values corresponding to a center position of each of the fluoroscopic images; calculating according to the three-dimensional reference coordinate values The reference feature points are relative to the three-dimensional coordinate values of the origin, and the three-dimensional coordinate values of the reference feature points and the description matrix of the reference feature points are stored as an indoor map corresponding to the indoor environment; and according to the indoor The map locates a three-dimensional target coordinate value of the portable electronic device relative to the origin in the indoor environment. The indoor positioning method according to claim 1, wherein the step of capturing the shooting position of the circular field image is the origin, and the step of recording the three-dimensional reference coordinate values corresponding to the center position of each of the fluoroscopic images comprises: obtaining A plurality of reference pixels in a direction of a horizontal axis and a direction of a horizontal axis of each of the fluoroscopic images are recorded, and the three-dimensional coordinate values of the reference pixels are recorded as the three-dimensional reference coordinate values. The indoor positioning method of claim 2, wherein each of the reference feature points corresponds to a feature point pixel in the fluoroscopic image, and the reference feature points are calculated according to the three-dimensional reference coordinate values. The step of the three-dimensional coordinate value of the origin includes: obtaining a first reference pixel corresponding to the feature point pixel in the vertical axis direction and a second reference pixel corresponding to the feature point pixel in the horizontal axis direction; Calculating a three-dimensional coordinate value of the feature point pixel according to the three-dimensional reference coordinate value of the first reference pixel and the three-dimensional reference coordinate value of the second reference pixel. The indoor positioning method according to claim 1, wherein the operation of establishing the indoor map is performed in an offline mode, and the offline mode is in a state of not being connected to an internet. The indoor positioning method of claim 1, wherein the indoor map is stored in a positioning database, and the three-dimensional positioning of the portable electronic device relative to the origin in the indoor environment is determined according to the indoor map. The step of the target coordinate value includes: obtaining an instant image through the portable electronic device, and capturing a plurality of target feature points in the instant image and a description matrix of the target feature points; Describe a matrix and a description matrix of the target feature points, and when the difference value between the description matrix of the reference feature points and the description matrix of the target feature points meets a predetermined tolerance value, from the positioning database Extracting a three-dimensional coordinate value of a reference feature point that matches a description matrix of the target feature points according to the preset tolerance value; and using a three-dimensional coordinate value of the reference feature points taken from the positioning database Calculating the three-dimensional target coordinate value and a rotation angle of the portable electronic device. The indoor positioning method of claim 5, wherein the operation of positioning the portable electronic device in the indoor environment relative to the three-dimensional target coordinate value of the origin is performed in a wired mode, and The connection mode is a state connected to an Internet, wherein the steps before the description matrix of the reference feature points and the description matrix of the target feature points further comprise: determining, by using the Internet, the The indoor environment in which the portable electronic device is located, and downloading the indoor map corresponding to the indoor environment to the portable electronic device. An indoor positioning system, comprising: a photographing device for photographing a ring image corresponding to an indoor environment; a portable electronic device; and an indoor map establishing device connected to the photographing device, wherein the indoor map establishing device comprises a storage device for storing a location database; and a processor coupled to the storage device, wherein the processor is configured to convert the ring image into a plurality of perspective images and capture a plurality of the plurality of perspective images Referring to a feature matrix and a description matrix of the reference feature points, wherein the processor is further configured to record a plurality of three-dimensional reference coordinate values corresponding to a center position of each fluoroscopic image by taking a shooting position of the ring image as an origin The processor is further configured to calculate a three-dimensional coordinate value of the reference feature points relative to the origin according to the three-dimensional reference coordinate values, and describe the three-dimensional coordinate values of the reference feature points and the reference feature points. An indoor map of the matrix corresponding to the indoor environment is stored in the location database, wherein the portable electronic device is used According to the positioning of the interior of the portable electronic device map to the indoor environment, a three-dimensional object relative to the coordinate value of the origin. The indoor positioning system of claim 7, wherein the processor is further configured to obtain a plurality of reference pixels in a longitudinal direction and a horizontal axis direction of a central position of each of the fluoroscopic images, and The three-dimensional coordinate values of the reference pixels are recorded as the three-dimensional reference coordinate values. The indoor positioning system of claim 8, wherein each of the reference feature points corresponds to a feature pixel in the fluoroscopic image, and the processor is further configured to obtain the corresponding feature point in the vertical axis direction. a first reference pixel of the pixel and a second reference pixel corresponding to the feature point pixel in the horizontal axis direction, wherein the processor is further configured to use the three-dimensional reference coordinate value of the first reference pixel and the The three-dimensional reference coordinate value of the second reference pixel calculates a three-dimensional coordinate value of the feature point pixel. The indoor positioning system of claim 7, wherein the portable electronic device is further configured to obtain an instant image, and capture a plurality of target feature points and descriptions of the target feature points in the instant image. a matrix, wherein the portable electronic device is further configured to determine an indoor environment in which the portable electronic device is located through an internet, and download the indoor map corresponding to the indoor environment to the portable electronic device, where The portable electronic device is further configured to compare the description matrix of the reference feature points with the description matrix of the target feature points, and the difference between the description matrix of the reference feature points and the description matrix of the target feature points. When the value meets a preset tolerance value, the three-dimensional coordinate value of the reference feature point corresponding to the preset tolerance value is obtained from the positioning database and the difference between the description matrix of the target feature points, wherein the value is The electronic device is further configured to calculate the three-dimensional target coordinate value and a rotation of the portable electronic device by using three-dimensional coordinate values of the reference feature points taken from the positioning database. angle. The indoor positioning system of claim 7, wherein the processor is operated in an offline mode and the portable electronic device operates in a connection mode, wherein the offline mode is not connected to an internet network. State, and the connection mode is a state of being connected to the Internet. An indoor map establishing device includes: a storage device storing a positioning database and a plurality of modules; and a processor coupled to the storage device to load and execute the modules stored in the storage device, The processor is operated in an offline mode, and the modules include: an input module for receiving a ring image corresponding to an indoor environment; and an image processing module for converting the ring image into a plurality of fluoroscopic images; a feature capture module for capturing a plurality of reference feature points in the fluoroscopic images and a description matrix of the reference feature points; and an operation module for capturing the ring field The image capturing position is an origin, and a plurality of three-dimensional reference coordinate values corresponding to a central position of each of the fluoroscopic images are recorded, wherein the computing module is further configured to calculate the reference feature points according to the three-dimensional reference coordinate values. The three-dimensional coordinate value of the origin, and an indoor map corresponding to the indoor environment composed of the three-dimensional coordinate values of the reference feature points and the description matrix of the reference feature points is stored to The location database. The indoor map establishing device according to claim 12, wherein the computing module is further configured to obtain a plurality of reference pixels in a vertical axis direction and a horizontal axis direction of a center position of each fluoroscopic image, The three-dimensional coordinate values of the reference pixels are recorded as the three-dimensional reference coordinate values. The indoor map establishing device according to claim 13, wherein each of the reference feature points corresponds to a feature pixel in the fluoroscopic image, and the operation module is further configured to obtain the vertical axis direction correspondingly a first reference pixel of the feature pixel and a second reference pixel corresponding to the feature pixel in the horizontal axis direction, wherein the operation module is further configured to use the three-dimensional reference coordinate of the first reference pixel The value and the three-dimensional reference coordinate value of the second reference pixel calculate a three-dimensional coordinate value of the feature point pixel. The indoor map establishing device according to claim 12, wherein the offline mode is a state that is not connected to an internet.