WO2025071926A1

WO2025071926A1 - Energy-efficient radio frequency and vision fusion for electronic shelf label systems

Info

Publication number: WO2025071926A1
Application number: PCT/US2024/046144
Authority: WO
Inventors: Marko ANGJELICHINOSKI; Sony Akkarakaran; Varun Amar REDDY; Alexandros MANOLAKOS
Original assignee: Qualcomm Inc
Current assignee: Qualcomm Inc
Priority date: 2023-09-14
Filing date: 2024-09-11
Publication date: 2025-04-03
Anticipated expiration: 2026-03-27

Abstract

This disclosure provides systems, methods, and devices for Electronic Shelf Label (ESL) systems that support indoor positioning of user devices. In a first aspect, a method of communicating in an ESL system includes determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions. Other aspects and features are also claimed and described.

Description

ENERGY-EFFICIENT RADIO FREQUENCY AND VISION FUSION FOR ELECTRONIC SHELF LABEL SYSTEMS

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of Greek Patent Application No. 20230100766, entitled, “ENERGY-EFFICIENT RADIO FREQUENCY AND VISION FUSION FOR ELECTRONIC SHELF LABEL SYSTEMS,” filed on September 27, 2023, and also the benefit of Greek Patent Application No. 20230100737, entitled, “ENERGYEFFICIENT RADIO FREQUENCY AND VISION FUSION FOR ELECTRONIC SHELF LABEL SYSTEMS,” filed on September 14, 2023, both of which are expressly incorporated by reference herein in their entirety.

TECHNICAL FIELD

[0002] Aspects of the present disclosure relate generally to electronic shelf label (ESL) systems, and more particularly, to methods and systems for indoor positioning of user devices in ESL systems.

INTRODUCTION

[0003] In general, retail stores use paper labels to display information about products displayed on shelves, such as prices, discount rates, unit costs, origins, or the like. Using such paper labels for the price display has limitations. For example, when there are changes in product information or locations on shelves, the retailer must generate new paper labels and discard old ones. This raises costs for maintenance in both supplies and employee labor. Further, in environmental terms, replacing the labels wastes raw materials such as paper, which adversely affects the protection of the environment. Still further, humans are prone to make mistakes, such as mislabeling a shelf or product or forgetting to take down temporary price changes on certain shelving, which results in shopper frustration.

[0004] Electronic shelf label (ESL) devices are electronic devices for displaying price or other relevant information for items on retail store shelves, which may be used in place of paper labels. ESL devices may be attached to a front edge of retail shelving and display a variety of information using display devices, such as Liquid Crystal Displays (LCD). Whenever the information about a product or the location of a product is changed, the ESL device may be programmed with new product information. Thus, the same electronic shelf label can be repeatedly used. The ESL devices may be equipped with Bluetooth Low Energy (BLE) radios that can be used for indoor positioning applications, such as tracking the location of BLE-enabled devices. Despite offering very accurate performance for locating nearby BLE devices, however, ESL systems introduce unique positioning-related challenges for BLE devices within a retail environment due in part to their large-scale, dense deployment of ESL-BLE radios with overlapping signals across multiple racks and shelves within the environment.

BRIEF SUMMARY OF SOME EXAMPLES

[0005] The following summarizes some aspects of the present disclosure to provide a basic understanding of the discussed technology. This summary is not an extensive overview of all contemplated features of the disclosure and is intended neither to identify key or critical elements of all aspects of the disclosure nor to delineate the scope of any or all aspects of the disclosure. Its sole purpose is to present some concepts of one or more aspects of the disclosure in summary form as a prelude to the more detailed description that is presented later.

[0006] Electronic shelf label (ESL) devices may be used in a wireless network to provide information and services to shoppers and retailers (as users of an ESL system). For example, ESL devices operating on a wireless network as part of ESL system may support indoor positioning services to identify the position of ESL devices within an environment (e.g., a floor space of a retail store). As another example, the ESL system may support position location services to identify a position of a user within the environment by interacting with the user’s mobile device. As still another example, the ESL system may use position ambiguity resolution or reduction (PARR) to resolve or reduce any ambiguities or inconsistencies detected with respect to the position of the user device within the environment. The disclosed PARR techniques may combine radio frequency (RF) measurements with visual channel state information (CSI) obtained from cameras deployed with the ESL system to overcome the positioning-related challenges associated with large-scale, dense deployments of ESL devices within a retail environment.

[0007] In one aspect of the disclosure, a method includes: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

[0008] In an additional aspect of the disclosure, an apparatus includes at least one processor and a memory coupled to the at least one processor. The at least one processor is configured to perform operations including: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

[0009] In an additional aspect of the disclosure, a non-transitory computer-readable medium stores instructions that, when executed by a processor, cause the processor to perform operations. The operations include: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

[0010] In an additional aspect of the disclosure, an electronic shelf label (ESL) system comprises: a server comprising a first memory and a first at least one processor coupled to the first memory; and an electronic shelf label (ESL) device comprising a second memory and a second at least one processor coupled to the second memory. The first at least one processor of the server is configured to perform operations comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions. The second at least one processor of the ESL device is configured to perform operations comprising: transmitting beacon signals for reception by the user device at a first periodicity; receiving instructions from the server to adjust the first periodicity of the beacon signals; and transmitting the beacon signals for reception by the user device at a second periodicity in accordance with the instructions from the server.

[0011] The foregoing has outlined rather broadly the features and technical advantages of examples according to the disclosure in order that the detailed description that follows may be better understood. Additional features and advantages will be described hereinafter. The conception and specific examples disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Such equivalent constructions do not depart from the scope of the appended claims. Characteristics of the concepts disclosed herein, both their organization and method of operation, together with associated advantages will be better understood from the following description when considered in connection with the accompanying figures. Each of the figures is provided for the purposes of illustration and description, and not as a definition of the limits of the claims.

[0012] Devices, networks, and systems may be configured to communicate via one or more portions of the electromagnetic spectrum. The present disclosure describes certain aspects with reference to certain communications technologies, such as Bluetooth or Wi-Fi. However, the description is not intended to be limited to a specific technology or application, and one or more aspects described with reference to one technology may be understood to be applicable to another technology. Moreover, it should be understood that, in operation, wireless communication networks adapted according to the concepts herein may operate with any combination of licensed or unlicensed spectrum depending on loading and availability. Accordingly, it will be apparent to a person having ordinary skill in the art that the systems, apparatus, and methods described herein may be applied to other communications systems and applications than the particular examples provided.

[0013] For example, the described implementations may be implemented in any device, system, or network that is capable of transmitting and receiving RF signals according to any of the wireless communication standards, including any of the IEEE 802.11 standards, the IEEE 802.15.1 Bluetooth® standards, Bluetooth low energy (BLE), code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), Global System for Mobile communications (GSM), GSM/General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband-CDMA (W-CDMA), Evolution Data Optimized (EV-DO), 1 *EV-DO, EV-DO Rev A, EV-DO Rev B, High Speed Packet Access (HSPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Evolved High Speed Packet Access (HSPA+), Long Term Evolution (LTE), AMPS, 5GNew Radio (5GNR), 6G, or other known signals that are used to communicate within a wireless, cellular, or internet of things (IOT) network, such as a system utilizing 3G, 4G 5G, or 6G technology, or further implementations thereof.

[0014] In various implementations, the techniques and apparatus may be used in wireless communication networks such as code division multiple access (CDMA) networks, time division multiple access (TDMA) networks, frequency division multiple access (FDMA) networks, orthogonal FDMA (OFDMA) networks, single-carrier FDMA (SC-FDMA) networks, LTE networks, GSM networks, 5^th Generation (5G) or new radio (NR) networks (sometimes referred to as “5G NR” networks, systems, or devices), as well as other communications networks. As described herein, the terms “networks” and “systems” may be used interchangeably and may refer to a collection of devices capable of communicating with each other through one or more communications techniques.

[0015] While aspects and implementations are described in this application by illustration to some examples, those skilled in the art will understand that additional implementations and use cases may come about in many different arrangements and scenarios. Innovations described herein may be implemented across many differing platform types, devices, systems, shapes, sizes, or packaging arrangements. For example, implementations or uses may come about via integrated chip implementations or other non-module-component based devices (e.g., end-user devices, vehicles, communication devices, computing devices, industrial equipment, retail devices or purchasing devices, medical devices, AI- enabled devices, etc.).

[0016] Implementations may range from chip-level or modular components to non-modular, non-chip-level implementations and further to aggregated, distributed, or original equipment manufacturer (OEM) devices or systems incorporating one or more described aspects. In some settings, devices incorporating described aspects and features may also include additional components and features for implementation and practice of claimed and described aspects. It is intended that innovations described herein may be practiced in a wide variety of implementations, including both large devices or small devices, chiplevel components, multi-component systems (e.g., radio frequency (RF)-chain, communication interface, processor), distributed arrangements, end-user devices, etc. of varying sizes, shapes, or constitutions.

[0017] In the following description, numerous specific details are set forth, such as examples of specific components, circuits, and processes to provide a thorough understanding of the present disclosure. The term “coupled” as used herein means connected directly to or connected through one or more intervening components or circuits. Also, in the following description and for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that these specific details may not be required to practice the teachings disclosed herein. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring teachings of the present disclosure.

[0018] Some portions of the detailed descriptions which follow are presented in terms of procedures, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. In the present disclosure, a procedure, logic block, process, or the like, is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.

[0019] In the figures, a single block may be described as performing a function or functions. The function or functions performed by that block may be performed in a single component or across multiple components, and/or may be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are described below generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. Also, the example devices may include components other than those shown, including well-known components such as a processor, memory, and the like.

[0020] Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present application, discussions utilizing the terms such as “accessing,” “receiving,” “sending,” “using,” “selecting,” “determining,” “normalizing,” “multiplying,” “averaging,” “monitoring,” “comparing,” “applying,” “updating,” “measuring,” “deriving,” “settling,” “generating” or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system’s registers and memories into other data similarly represented as physical quantities within the computer system’s registers, memories, or other such information storage, transmission, or display devices.

[0021] The terms “device” and “apparatus” are not limited to one or a specific number of physical objects (such as one smartphone, one camera controller, one processing system, and so on). As used herein, a device may be any electronic device with one or more parts that may implement at least some portions of the disclosure. While the below description and examples use the term “device” to describe various aspects of the disclosure, the term “device” is not limited to a specific configuration, type, or number of objects. As used herein, an apparatus may include a device or a portion of the device for performing the described operations.

[0022] As used herein, including in the claims, the term “or,” when used in a list of two or more items, means that any one of the listed items may be employed by itself, or any combination of two or more of the listed items may be employed. For example, if a device is described as containing components A, B, or C, the device may contain A alone; B alone; C alone; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination.

[0023] Also, as used herein, including in the claims, “or” as used in a list of items prefaced by “at least one of’ indicates a disjunctive list such that, for example, a list of “at least one of A, B, or C” means A or B or C or AB or AC or BC or ABC (that is A and B and C) or any of these in any combination thereof.

[0024] Also, as used herein, the term “substantially” is defined as largely but not necessarily wholly what is specified (and includes what is specified; for example, substantially 90 degrees includes 90 degrees and substantially parallel includes parallel), as understood by a person of ordinary skill in the art. In any disclosed implementations, the term “substantially” may be substituted with “within [a percentage] of’ what is specified, where the percentage includes .1, 1, 5, or 10 percent.

[0025] Also, as used herein, relative terms, unless otherwise specified, may be understood to be relative to a reference by a certain amount. For example, terms such as “higher” or “lower” or “more” or “less” may be understood as higher, lower, more, or less than a reference value by a threshold amount.

BRIEF DESCRIPTION OF THE DRAWINGS

[0026] A further understanding of the nature and advantages of the present disclosure may be realized by reference to the following drawings. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If just the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0027] Figure 1A is a block diagram illustrating a perspective view of an example Electronic Shelf Label (ESL) system according to some embodiments of this disclosure.

[0028] Figure IB is a block diagram of an overhead view of the ESL system of Figure 1 A according to some embodiments of the disclosure.

[0029] Figure 1C is a diagram illustrating an example ESL device in the ESL system of Figures 1 A and IB according to some embodiments of the disclosure.

[0030] Figure 2A is a perspective view of a gondola with ESL devices according to some embodiments of the disclosure.

[0031] Figure 2B is a top-down view of a retail environment with ESL devices accessible to a user according to some embodiments of the disclosure.

[0032] Figure 3 is a timing diagram illustrating time division multiplexing for communicating with multiple ESL devices according to some embodiments of the disclosure.

[0033] Figure 4 is a block diagram illustrating an example of an ESL device according to some embodiments of this disclosure.

[0034] Figure 5 is a block diagram of an example server for indoor positioning of Bluetooth Low

Energy (BLE) enabled devices and other objects within a retail environment using position ambiguity reduction or resolution (PARR) according to some embodiments of this disclosure.

[0035] Figure 6 is an overhead view of an example retail environment illustrating multiple candidate positions of a BLE device according to some embodiments of this disclosure.

[0036] Figure 7 illustrates different iterations of a PARR-based process for finding a match between visual location information and candidate positions estimated for a BLE device within a retail environment according to some embodiments of this disclosure.

[0037] Figure 8A is an overhead view of an example retail environment illustrating locating a passive object having no radio frequency (RF) signaling capability within a retail environment according to some embodiments of this disclosure.

[0038] Figure 8B is an overhead view of an example retail environment illustrating locating a passive object having no radio frequency (RF) signaling capability within a retail environment according to some embodiments of this disclosure.

[0039] Figure 9 is a flow chart of an example method for indoor positioning of a user device using position ambiguity reduction or resolution according to some embodiments of this disclosure.

[0040] Figure 10A is a flow diagram of an example process for detecting a trigger condition for activating a PARR service for locating a BLE device within a retail environment according to some embodiments of this disclosure.

[0041] Figure 10B is a flow diagram of an example process for locating the BLE device of Figure 10A after the PARR service has been activated according to some embodiments of this disclosure.

[0042] Figure 11A is a flow diagram of an example process for locating a passive or non-RF object within a retail environment in response to an urgent request according to some embodiments of this disclosure.

[0043] Figure 1 IB is a flow diagram of an example process for locating the non-RF object within the retail environment in response to a non-urgent request according to some embodiments of this disclosure.

[0044] Like reference numbers and designations in the various drawings indicate like elements.

DETAILED DESCRIPTION

[0045] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to limit the scope of the disclosure. Rather, the detailed description includes specific details for the purpose of providing a thorough understanding of the inventive subject matter. It will be apparent to those skilled in the art that these specific details are not required in every case and that, in some instances, well-known structures and components are shown in block diagram form for clarity of presentation.

[0046] The present disclosure provides systems, apparatus, methods, and computer-readable media that support indoor positioning of user devices or objects within a retail environment, such as a retail store or warehouse. In some embodiments, an Electronic Shelf Label (ESL) system may be deployed within the environment for purposes of monitoring or tracking assets (e.g., product locations) and providing indoor positioning and navigation services for users (e.g., customers or store employees) within the environment.

[0047] Particular implementations of the subject matter described in this disclosure may be implemented to realize one or more of the following potential advantages or benefits. In some aspects, the present disclosure provides techniques for wireless communication systems that may be particularly beneficial in Electronic Shelf Label (ESL) applications within retail environments. To address the positioning-related challenges associated with dense deployments of ESL devices within a retail environment, the ESL system may provide indoor positioning services for locating Bluetooth Low Energy (BLE) devices and other objects within the environment using position ambiguity resolution or reduction (PARR). Such objects may include people or passive objects having no radio frequency (RF) signaling capabilities. As will be described in further detail below, the disclosed PARR techniques may utilize RF-based location information in conjunction with visual channel state information (CSI) obtained from cameras deployed within the environment to resolve or reduce ambiguities or inconsistencies with respect to the positioning of BLE devices and non-RF objects in a cost-effective, energy efficient, and timely manner.

[0048] Figure 1 A is a block diagram illustrating a perspective view of an example ESL system 100 according to some embodiments of this disclosure. As shown in Figure 1A, the ESL system 100 may be deployed across multiple aisles of gondolas 112A-112H within a retail environment 110. The retail environment 110 may include, for example, a retail floor space of a retail store. As will be described in further detail below, each of the gondolas 112A-112H may include one or more shelves to which a plurality of ESL devices are attached. The ESL devices may be used to display pricing and/or other information related to products located on the shelves. Each of the ESL devices may include one or more wireless radios for transmitting and receiving information over a wireless network associated with the retail store.

[0049] In some implementations, cameras may be installed on or near the shelves at the ends of the gondolas 112A-112H to capture different views of the aisles and/or other pathways in the areas surrounding the gondolas 112A-112H. Additionally, over-the-top (OTT) cameras 102A-102D (or “OTT cameras 102” collectively) may be positioned at different overhead locations throughout the retail store to capture larger regions of the retail environment 110. In some cases, the OTT cameras 102 may be part of a separate OTT camera system of the retail store used for security and monitoring purposes.

[0050] In some embodiments, images and/or video captured by the OTT cameras 102 and/or those captured by the shelf cameras on the gondolas 112A-112H may be utilized as visual channel state information (CSI) to reduce, if not resolve, any errors or ambiguities detected in the position of a BLE device 124 within the retail environment 110. The BLE device 124 may be, for example, a mobile device of a user (e.g., a customer or employee of the retail store). The BLE device 124 may be equipped with a BLE radio for receiving low power beacons transmitted by the ESL devices attached to the gondolas 112A-112H. As will be described in further detail below with respect to Figure IB, radio frequency (RF) measurements of the ESL beacons captured by the BLE device 124 along with the visual CSI from the OTT cameras 102 and/or the shelf cameras may be provided to a management server of the ESL system 100 for monitoring or tracking the position of the BLE device 124 as it (and the user) traverses different areas of the retail environment 110.

[0051] Figure IB is a block diagram of an overhead view of the ESL system 100 of Figure 1A according to some embodiments of the disclosure. As shown in Figure IB, the ESL system 100 may further include a management server 122 that is integrated with or coupled to a gateway node 120. The management server 122 may include at least one processor coupled to a memory, in which the at least one processor is configured to execute computer program code stored on a computer-readable medium to cause the management server 122 to perform operations related to managing operation of ESL devices 108A-108D, access points (APs) 106A and 106B, the gateway node 120, and/or other components within the ESL system 100. For example, the management server 122 may perform operations relating to an indoor positioning service with position ambiguity resolution/reduction (PARR) for resolving or reducing ambiguities in the positioning of the BLE device 124 within the retail environment 110, as will be described in further detail below with reference to Figure 5 and/or Figure 6. [0052] The gateway node 120 may communicate with access point (AP) 106 A and access point (AP) 106B. Although only two APs are shown in the example system, fewer or more APs may be included in the ESL system 100. The APs 106A and 106B may communicate through a first communication network, either wired or wireless, with the gateway node 120. The APs 106A and 106B may also communicate through a second communication network with the ESL devices 108A-108D. In some embodiments, the first and second communication networks may be different networks. For example, the first communication network (e.g., between the AP 106A and the gateway node 120) may be a Wi-Fi network, and the second communication network (e.g., between the AP 106A and the ESL device 108 A) may be a Bluetooth network.

[0053] In some embodiments, each of the APs 106A and 106B may communicate with a subset of the ESL devices 108A-108D located in a geographic area assigned to that AP, e.g., based on a location of the AP relative to each ESL device and/or a signal coverage area of the AP. For example, the retail environment 110 may be divided into two equally-sized geographic areas that each cover approximately half of the total retail environment 110. The AP 106 A may be assigned to a geographic area 110A, and the AP 106B may be assigned to a geographic area HOB. Accordingly, the AP 106A may communicate with ESL device 108 A and ESL device 108B located in the geographic area 110A, and the AP 106B may communicate with ESL device 108C and ESL device 108D located in the geographic area HOB.

[0054] Bluetooth technology provides a secure way to connect and exchange information between electronic devices, such as smartphones, other cellular phones, headphones, earbuds, smartwatches, laptops, wearables, and/or shelf labels. Bluetooth communications may include establishing wireless personal area networks (PANs) (also referred to as “ad hoc” or “peer-to-peer” networks). These ad hoc networks are commonly called “piconets.” Each device may belong to multiple piconets. Multiple interconnected piconets may be called scatternets. A scatternet may be formed when a member of a first piconet elects to participate in a second piconet. In the example of Figure IB, the ESL devices 108 A and 108B may be in a piconet with the AP 106 A.

[0055] Because many of the services offered over Bluetooth can expose private data or allow the connecting party to control the connected device, Bluetooth networks may have devices first establish a “trust relationship” before they are allowed to communicate private data to one another. This trust relationship may be established using a process referred to as “pairing,” in which a bond is formed between two devices. This bond enables the devices to communicate with each other in the future without further authentication. Thus, the ESL device 108 A in Figure IB may be bonded in such a manner to the AP 106A. The pairing process may be triggered automatically each time the device is powered on or moved within a certain distance of another Bluetooth device. Pairing information relating to current and previously established pairings may be stored in a paired device list (PDL) in the memory of the Bluetooth device, such as the ESL device 108 A and/or the AP 106A. This pairing information may include a name field, an address field, a link key field, and other similar fields (such as “profile” type) useful for authenticating the device or establishing a Bluetooth communication link. The pairing information may allow the ESL device 108 A to reconnect to the AP 106 A automatically when, for example, power loss causes the ESL system 100 to reset.

[0056] A Bluetooth “profile” describes general behaviors through which Bluetooth-enabled devices communicate with other Bluetooth devices. For example, the hands free profile (HFP) describes how a Bluetooth device (such as a smartphone) may place and receive calls for another Bluetooth device, and the Advanced Audio Distribution Profile (A2DP) describes how stereo-quality audio may be streamed from a first Bluetooth device (such as a smartphone) to another Bluetooth device (such as an earbud). The ESL devices 108A- 108D may be configured with an ESL Profile compliant with the Electronic Shelf Label Profile Specification vl.O dated March 28, 2023, which is incorporated by reference herein. The ESL Profile may specify how the AP 106 A may use one or more ESL Services exposed by the ESL device 108 A.

[0057] The management server 122 may be implemented as a database (DB) server that stores and manages product information regarding products displayed in a retail store or distribution warehouse. The management server 122 may store a variety of information used during the operation of a store in addition to the product information. Furthermore, the management server 122 may generate and transmit command messages to the ESL devices 108A-108D via the first communication network described above. The command messages may be used by the management server 122 to carry out various functions in the ESL system 100, such as the synchronization, updating, and alteration of product information displayed on the ESL devices 108A-108D. The management server 122 may obtain the product information from a product database provided for the ESL devices 108A-108D. That is, the management server 122 may be provided with a database that stores identification information relating to the ESL devices 108A-108D in connection with product information to be displayed on a corresponding one of the ESL devices 108A-108D.

[0058] A command message created by the management server 122 (e.g., a product-information change message or a management-information acquisition message) can be transferred to the gateway node 120 using a communication scheme suitable for the gateway node 120. The command message may be converted into a packet configured according to the communication scheme and the configured packet may be transferred to the gateway node 120. Furthermore, the management server 122 may receive a reception acknowledgement message sent from the gateway node 120 using the communication scheme. In some cases, the acknowledgement message may be first converted into a message format suitable for reception by the management server 122 before being transferred by the gateway node 120.

[0059] In some embodiments, the command messages transmitted by the management server 122 via the gateway node 120 may include various commands directed to different devices in the ESL system 100 as part of the position ambiguity reduction or resolution (PARR) functionality provided by the management server 122 for resolving any ambiguities in the positioning of the BLE device 124 within the retail environment 110. Such ambiguities may be due to various factors that can affect positioning accuracy within an indoor environment having a dense deployment of BLE-enabled devices. For example, channel fading and shadowing may cause the BLE device 124 to receive ESL beacons from neighboring shelfs/aisles. This might skew RF measurements, such as reference signal received power (RSRP), captured by the BLE device 124 and therefore, produce position estimates that are not consistent with the aisle/section in which the BLE device 124 is actually located. Even though the positioning error in this example may remain relatively small, misclassifying the aisle in which a BLE device is located may have significant impacts on system performance and functionality, particularly with respect to indoor navigation. Such a positioning error may cause, for example, a user (e.g., a retail store customer) of the BLE device 124 to be directed to an incorrect shelf, aisle, section of the retail store. Additional examples of positioning errors or ambiguities will be described in further detail below with reference to Figures 6 and 7.

[0060] Examples of commands that may be sent by the management server 122 to resolve or reduce such ambiguities or errors include, but are not limited to, commands directed to the ESL devices 108A-108D for checking and/or changing (e.g., by increasing) the duty cycle or periodicity of beacon transmissions sent by the ESL devices 108A-108D. The management server 122 may also send commands directing the BLE device 124 to capture new RF measurements based on the adjusted duty cycle, which the management server 122 may use to update the position estimates. Additionally, the management server 122 may send commands directed to one or more of shelf cameras 104A-104D and/or one or more of the OTT cameras 102A-102D for providing visual CSI (e.g., images and/or video sequences) to the management server 122 to aid in the position ambiguity reduction or resolution, as will be described in further detail below.

[0061] Although only one gateway node 120 is shown in the ESL system 100, there may be several such gateway nodes communicating with the management server 122. Each gateway node 120 analyzes data received from the management server 122, confirms the presence or absence of a message or data to be sent to one or more of the ESL devices 108A-108D, and then sends the confirmed message or data to the appropriate ESL device. For example, the gateway node 120 may configure a message, which is to be sent to the ESL device 108 A, into a packet according to a communication scheme and send the configured packet to the ESL device 108 A through commanding the AP 106 A to transmit the packet. Furthermore, the gateway node 120 may transfer a reception acknowledgement message received from the ESL device 108 A through the AP 106A to the management server 122.

[0062] Each of the ESL devices 108A-108D may include a display, such as a Liquid Crystal Display (LCD), for displaying data concerning products located on a shelf to which the corresponding ESL device is attached. The information displayed by each of the ESL devices 108A-108D may be received from the gateway node 120. In some implementations, each of the ESL device 108A-108D may be configured with a memory and microprocessor for performing various operations. An example of such a configuration for the ESL device 108 A will be described in further detail below with reference to Figure 4.

[0063] In some embodiments, a video monitoring system may be used to augment the capabilities of the ESL system 100. In some implementations, the video monitoring system may be included as part of the ESL system 100. Additionally or alternatively, the video monitoring system may be part of a separate security monitoring system associated with the retail environment 110. For example, the shelf cameras 104A-104D may be part of the ESL system 100 and used to provide visual location information or CSI to the management server 122. The shelf cameras 104A-104D may be positioned with a field of view that captures one or more shelves of one or more of the gondolas 112A-112H. The shelf cameras 104A-104D may be used to assist in tracking stock levels and/or identifying items picked by users while in the environment. As another example, over-the-top (OTT) cameras 102A-102D may be part of a separate security system used in conjunction with the ESL system 100. As described above, the OTT cameras 102A-102D may be positioned with different fields of view for capturing large regions of the retail environment 110 for processing by the management server 122 of the ESL system 100. Object recognition models or systems may be applied to the image frames or other visual information received from the cameras 102A-102D and/or 104A-104D to determine a presence of, or a count of, objects and/or humans in the field of view of a respective camera.

[0064] The OTT cameras 102A-102D may be used to support the determination of the positions of the ESL device 108A-108D, the BLE device 124, and/or other devices within the retail environment 110. As described above, the BLE device 124 may be a mobile device that a user carries while moving through the retail environment 110. As the user traverses with the BLE device 124, the BLE device 124 may communicate with the ESL devices 108A- 108D, for example, by receiving identification information from the ESL devices 108A- 108D. The location of the ESL devices 108A-108D may be determined by identifying a location of the BLE device 124 from the camera image frames at the time the BLE device 124 receives signals, and/or the strength of the signals, received from the ESL devices 108A-108D.

[0065] Figure 1C is a diagram illustrating an example ESL device in the ESL system of Figures 1A and IB according to some embodiments of the disclosure. The ESL devices 108A- 108D may change price information or be activated or inactivated while communicating with the gateway node 120. A store manager may send the management server 122 a command concerning the synchronization between a product and a specified ESL device and/or a command for the correction of information regarding a product assigned to the particular ESL device. An example of the ESL device 108C is shown Figure 1C with a display device for displaying product information, such as a product description, a product image, a product price, a product barcode, a product rating, a product Stock Keeping Unit (SKU), and/or a product link (e.g., a URL or QR code).

[0066] As described above, the retail environment associated with the ESL system may include ESL devices organized on gondolas and shelves. One example illustration of such an arrangement is shown in Figure 2A. Figure 2A is a perspective view of a gondola with Electronic Shelf Label (ESL) devices according to some embodiments of the disclosure. The gondola 112A may include multiple shelves 202A-202D at different vertical levels from a floor. ESL devices may be attached to the shelves 202A-202D. For example, ESL device 108 A may be attached to shelf 202A to display information regarding products stocked on shelf 202 A in the vicinity of the ESL Device 108 A.

[0067] The ESL devices may provide information to a shopper or store employee operating in the environment, such as to provide information regarding products and/or assist with location determination of products or the user. Figure 2B is a top-down view of a retail environment with Electronic Shelf Label (ESL) devices accessible to a user according to some embodiments of the disclosure. A user pushing a shopping cart 212 through an aisle may use ESL devices to determine the location of a particular product. For example, a mobile device associated with the shopping cart 212 may guide a user to a location 210 corresponding to gondola 112B where stock for a desired product is located.

[0068] Communication within the ESL system 100 between an access point (AP) and Electronic Shelf Label (ESL) devices may be performed according to a Time Division Multiple Access (TDMA) scheme, such as one illustrated in Figure 3. Figure 3 is a timing diagram illustrating time division multiplexing for communicating with multiple Electronic Shelf Label (ESL) devices according to some embodiments of the disclosure. An AP, such as AP 106A, may broadcast information that is received by all ESL devices, including ESL device 108A, during a first time period 302. The ESL devices may communicate with the AP during subsequent time periods. For example, a first ESL device, such as ESL device 108 A, may transmit in time period 304 A, with other ESL devices transmitting in time periods 304B-304E. In ESL systems with significant numbers of ESL devices, the ESL devices may be configured to communicate in different groups. For example, ESL devices 1-5 may be configured to transmit to the AP during a first time cycle and ESL devices 6- 10 may be configured to transmit to the AP during a second time cycle. The first and second time cycles may alternate during operation of the wireless network.

[0069] An ESL device may include components configured together to provide some or all of the functionality described in this disclosure and/or provide additional functionality. Figure 4 is a block diagram illustrating an example Electronic Shelf Label (ESL) device according to some embodiments of this disclosure. The ESL device 108 A may include a low-power microcontroller 410. Although functionality for the ESL device may be configured by the microcontroller 410 in embodiments of this disclosure, any single or combination of processors (e.g., at least one processor) may be used to perform the functions described according to embodiments of this disclosure. [0070] The microcontroller 410 may include a memory 416. The memory 416 may store computer program code that causes a microcontroller 414 to execute operations that carry out some or all of the functionality described in embodiments of this disclosure. Although shown as part of the microcontroller 410, the memory 416 may be located internal to or external to the microcontroller 410. The microcontroller 410 may also include one or more wireless radio(s) 412. The wireless radios 412 may include, for example, a Bluetooth wireless radio including a front end that couples to antenna 408 for transmitting and receiving radio frequency (RF) signals at one or more frequencies in one or more frequency bands. In some embodiments, the microcontroller 410 is a System on Chip (SoC) in which two or more components of the wireless radio(s) 412, the microprocessor 414, and/or the memory 416 are included in a single semiconductor package. In some embodiments, the two or more components may be included on a single semiconductor die.

[0071] The ESL device 108 A may include I/O devices, such as a notification LED 402 and/or an electronic display 404. The notification LED 402 may include one or more light emitting diodes (LEDs), or other light sources configured to flashlight of one or more colors. The notification LED may be triggered to blink at a specific time and/or with a specific color based on a command received from the gateway node 120. For example, a notification LED 402 may blink to attract a user’s attention to a particular location on a shelf. The electronic display 404 may be, for example, an electronic-ink (e-Ink) display configured to output the product information.

[0072] The ESL device 108 A may couple to a battery 406 or other power source to power operations performed by the ESL device 108 A, such as to operate the wireless radio(s) 412, the notification LED 402, the electronic display 404, the memory 416, and/or the microprocessor 414. The battery 406 may allow placement of the ESL device 108A in a place where constant power supply is difficult. Thus, in order that a single battery charge provides a long period of use (e.g., lasting longer than several years), the ESL device 108 may be configured to reduce power consumption during times when frequent commands are not expected. For example, the ESL device 108 A may operate using a wakeup communication scheme. That is, the ESL device 108 A wakes up according to predetermined time intervals to determine whether data is waiting to be received. When no data is waiting, power to the ESL device 108 A is turned off until the next wakeup period to reduce power consumption. When there is data to be received, the ESL device 108 A wakes up to perform communication operations. [0073] Figure 5 is a block diagram of an example server 500 for indoor positioning of Bluetooth Low Energy (BLE) enabled devices and other objects (e.g., non-RF devices or objects) within a retail environment (e.g., retail environment 110 of Figures 1A and IB, as described above) using position ambiguity reduction or resolution (PARR) according to some embodiments of this disclosure. The server 500 may be used to implement, for example, the management server 122 of Figure IB, as described above. As shown in Figure 5, the server 500 includes a visual CSI (or “vCSI”) module 510, an RF-based module 520, and a positioning module 530.

[0074] In some embodiments, the server 500 may use the vCSI module 510 to process visual information or vCSI (e.g., images and/or video sequences) received from one or more cameras (e.g., shelf cameras 104A-104D or OTT cameras 102A-102D of Figure IB, as described above) located within the retail environment. As described above, shelf cameras may be installed on each side of each gondola within the retail environment. These cameras may be inactive most of the time to save power and activated only when pinged or instructed to do so by the server 500 for positioning purposes. For example, the server 500 (or vCSI module 510 thereof) may send commands or instructions to one or more of the shelf cameras to send either raw image snapshots or pre-processed vCSI data (e.g., including metadata used to define bounding boxes around objects of interest within an image). In contrast to the shelf cameras, the OTT cameras may be part of a legacy surveillance system deployed within the retail store or warehouse for security or monitoring purposes. Accordingly, the surveillance system and OTT cameras thereof may remain active either most or all the time. These cameras may also maintain connection with the server 500, which may be used to send either raw images or reports with processed vCSI data upon being requested by the server 500.

[0075] In some embodiments, the server 500 may use the RF-based module 520 to process RF measurements received from a BLE-enabled device (e.g., the BLE device 124 of Figures 1A and IB, as described above) and ESL devices (e.g., the ESL devices 108A-108D of Figure 1A and IB). The RF measurements may include, for example, RSRP measurements of the beacons transmitted by the ESL devices. As described above, the BLE device may be an RF device that also supports BLE technology and receives beacons from the ESL devices attached to the shelves on gondolas located within the environment, as described above. The BLE device may be, for example, a mobile device of a customer in the retail store. Alternatively, the BLE device may be a proprietary user device operated by an employee (e.g., a warehouse worker). Such a user device may be considered a store asset in the form of a BLE-enabled RF device. The BLE device in this example may maintain a connection with the server 500 via a wireless communication network while the user and the device remain within the store. The BLE device may use this connection to transmit RF measurements (e.g., RSRP) of the ESL beacons received from nearby ESL devices located in different areas of the retail environment. Such measurements may be transmitted or reported by the BLE device to the server 500 on a periodic basis or on demand, e.g., in response to a command or request sent by the server 500. In some embodiments, the occurrence of certain events may trigger the server 500 to send a command or request for RF measurements to the BLE device.

[0076] In some embodiments, the visual information and RF measurements processed by the vCSI module 510 and the RF-based module 520, respectively, may be used by the positioning module 530 to determine a position of the BLE device or a non-RF object within the retail environment. As shown in Figure 5, the positioning module 530 may include a PARR module 532 for locating BLE devices within the retail environment and a non-RF object location module 534 for locating non-RF objects within the environment. [0077] A non-RF object may be either a device with RF capability that is no longer transmitting RF signals (e.g., due to a malfunction, battery depletion, etc.) or a passive object or person (e.g., a store worker, a customer, a child, an intruder, etc.) with no RF equipment or wireless transmission capabilities of any kind and thus, no prior RF history. A non-RF object that falls into the first category may be, for example, a BLE device that was once active and whose position within the environment was once known based on RF measurements received by the server 500 from the device. The server 500 may determine that the device is no longer active (and consider it a non-RF object) if the server 500 detects that no RF measurements or reports have been received from the device for an extended period of time (e.g., a period lasting longer than a delay threshold). The determination may be made based on, for example, a RF history associated with the device, which may be stored by the server 500 in a local data store or remote database (not shown) coupled to the server 500.

[0078] In some embodiments, the PARR module 532 may use the visual information processed by the vCSI module 510 to resolve ambiguities or errors detected in the RF-based positioning information used to determine a position of the BLE device within the retail environment. As described above, an ESL system with a dense deployment of ESL devices across different shelves and gondolas within the environment may present various positioning-related issues, where the BLE device of a user may be mistakenly positioned in an incorrect aisle or section (or even impossible places). Therefore, one of the aims of the PARR module 532 may be to help reduce and resolve such ambiguities with the aid of the vCSI module 510.

[0079] The aims of the non-RF object location module 534 may include providing support for locating BLE devices that have (temporarily) lost their RF ability. The non-RF object location module 534 may utilize historical RF-based position information previously obtained for the device in conjunction with the vCSI to locate the device. The non-RF object location module 534 may also use one or more vCSI collection strategies to handle urgent requests to locate such devices or other non-RF objects in critical situations where time is of the essence, such as locating a lost child or a thief within the retail environment. Such urgent requests may be initiated by, for example, a control center 540 of the retail store or warehouse in this example. The control center 540 may include, for example, a computing device (e.g., a server, workstation, or desktop computer) located within the retail store or at a remote location (e.g., in a cloud computing environment). The control center 540 may be communicatively coupled to the server 500 via a wired or wireless network. In some embodiments, the control center 540 may be used to send requests for locating RF devices or non-RF objects within the environment. Such requests may have different levels of urgency and may be sent automatically (e.g., upon detecting that a certain BLE device has been inactive for a specified period of time) or manually (e.g., based on commands input by an operator). Additional details regarding techniques for locating non-RF objects will be described below with reference to Figures 8A and 8B.

[0080] The benefits of employing a server-based positioning system as in the example of Figure 5 may include providing energy-efficient strategies for leveraging visual CSI for indoor positioning of BLE-enabled and non-RF objects in dense, large-scale ESL-BLE deployments. The ESL system devices in such a deployment often lack access to power grid resources. By contrast, a server-based system (e.g., the server 500) not only has access to power but also has access to information regarding the layout and distribution of ESL devices and shelves throughout the environment of the ESL system. This information may be used by the positioning system to estimate or determine a position of a user and/or user device within the environment in an energy efficient manner.

[0081] In some embodiments, the position ambiguity reduction or resolution (PARR) functionality of the server 500 for locating the position of a BLE device may be triggered based on the detection of certain conditions or events. Such trigger conditions or events may include, for example, the detection of unreliable RF measurements and/or RF-based positioning information or the detection of a mismatch between visual information (e.g., as obtained by the vCSI module 510 from shelf cameras and/or OTT cameras) and RF- based positioning information (e.g., as obtained by the RF-based module 520 from the BLE device as described above).

[0082] In some embodiments, the reliability of the RF measurements and/or a position estimate derived therefrom may be evaluated based on a reliability metric. Such a metric may be, for example, a weighted value or confidence score that expresses a likelihood that a set of RF measurements and/or a position estimate is accurate or reliable. The reliability metric may be designed to reflect the common errors/ambiguities that can occur in BLE scenarios with large-scale, dense deployments of ESL devices. In other words, the metric should indicate if the RF measurements (ESL RSRPs) are likely to produce a BLE device position estimate consistent with the actual section and aisle in which the BLE device is located. The reliability metric may be computed by the server 500 based on RF measurement reports sent by the BLE device. Alternatively, the BLE device may send a request to the server 500 to initiate indoor positioning of the BLE device using PARR, as will be described in further detail below.

[0083] In some cases, the reliability metric may be compared against a minimum reliability threshold. If the threshold is crossed, the server 500 may initiate a procedure for locating the device using PARR. A reliability metric less than the minimum threshold may indicate, for example, that the RF-based position information is likely inconsistent with the actual location of the BLE device within the environment. Such inconsistencies may be due to various reasons, such as the presence of numerous outliers in the ESL beacon measurements as a result of channel uncertainty and the inclusion of RF measurements in for beacons received by the BLE device from ESL devices located in other aisles different from a current aisle in which the BLE device is located. The other aisles may include, for example, neighboring aisles located in the same or a nearby section of the retail store. Such inconsistencies may also be due to the relatively low duty cycle or periodicity of transmissions produced by the ESL devices. For example, the BLE radios of the ESL devices may be calibrated to transmit low power beacons at a very low base duty cycle or periodicity to conserve energy.

[0084] In addition to unreliable RF measurements, a mismatch or inconsistency between the RF measurements and visual location information (vCSI) captured for the BLE device may qualify as a trigger condition or event for activating the PARR functionality of the server 500. The visual information may include, for example, images captured by one or more cameras (e.g., shelf cameras and/or OTT cameras), as described above. The server 500 or vCSI module 510 thereof may utilize an object detection model to analyze an image obtained from the camera(s) and visually detect the BLE device as well as one or more reference objects (e.g., one or more gondolas at fixed locations within the retail environment). The server 500 may be able to identify the reference object(s) and their respective locations, which can then be used to estimate the location of the BLE device within the environment. In some cases, the server 500 may detect an inconsistency between the estimated location of the BLE device in the image and the RF-based position of the device. The reliability metric in this case may be based on an association cost between the visual detection of the BLE device (e.g., within a bounding box) and the corresponding RF-based measurements and position information. For example, the Euclidean distance between the visual location and RF-based position of the BLE device may be used as the reliability metric, in which case PARR may be triggered if the association cost or calculated distance is larger than a specified number of meters. The inconsistency associated with the triggering condition in this example may be helpful in detecting cases where the RF-based reports and the related positioning information from the BLE device may have been compromised or tampered.

[0085] Upon detecting one of the above trigger conditions or events and before activating PARR, the server 500 in some cases may check or scan the relevant ESL-BLE radio devices to determine their current activity level and duty cycle or periodicity. The server 500 may also instruct the ESL devices to increase their duty cycle and transmit new beacons for the BLE device to measure and report back to the server 500. A duty cycle increase may be particularly warranted in cases where the RF measurements have been breached or compromised by an adversary or unauthorized third-party or cases where the number of reported ESL beacons is very low or less than a minimum threshold (e.g., as a result of certain ESL devices becoming inoperative and transitioning to inactive state for a prolonged period). After instructing the ESL devices to increase their duty cycle and receiving new RF measurements as a result, the server 500 may reassess the trigger condition based on the new RF measurement data to determine whether PARR should still be executed.

[0086] In cases where the server 500 detects a period of inactivity in which the BLE device has not reported any ESL beacon measurements, the server 500 may use the visual information from the camera(s) to locate the BLE device and identify nearby ESL devices. The server 500 may then instruct the identified ESL devices to transmit beacons and instruct the BLE device to report the corresponding RF measurements.

[0087] Once all the relevant RF and visual information is collected after detecting a trigger condition or event, the server 500 may initiate indoor positioning using PARR by activating a PARR procedure or process including a sequence of steps for resolving any inconsistencies detected with respect to the location of the BLE device within the environment. The initial step of such a PARR process may include estimating a plurality of candidate positions of the BLE device within the environment, as shown in Figure 6.

[0088] Figure 6 is an overhead view of an example retail environment 600 illustrating multiple candidate positions 610A-610E of a BLE device 610 according to some embodiments of this disclosure. The retail environment 600 may include an ESL system similar to ESL system 100 of Figures 1A and IB described above. For purposes of this example, it is assumed that the location of the BLE device 610 within the environment 600 as shown in Figure 6 is the actual or true position of the BLE device within the environment 600. As will be described in further detail below, the candidate positions 610A-610E of the BLE device 610 may be estimated using RF -based positioning information. In some embodiments, one of the candidate positions 610A-610E may be selected using a matching process that fuses or combines RF-based information with visual information for each candidate position. The visual information may include, for example, images and/or video sequences obtained from an OTT camera 602 and/or an OTT camera 604 for different locations or areas of the environment associated with each candidate position. Although not shown in Figure 6, it should be appreciated that shelf cameras may also be installed at the ends of the gondolas within the retail environment 600 and that the visual location information obtained for each candidate position may also include images and/or video obtained from the shelf cameras.

[0089] In some embodiments, each of the candidate positions 610A-610E may be estimated (e.g., by the server 500 of Figure 5, as described above) based on a portion of the positioning information received from the BLE device 610. The positioning information may include a set of RF measurements and the portion used for each candidate position may be a subset of the original set. In some implementations, the subsets may be generated by subsampling the original set of RF measurements. A preliminary assessment of the candidate positions may then be performed to identify and filter out any candidate position that may be implausible. For example, candidate position 610E may be removed from consideration as it corresponds to an implausible location on top of a gondola. In some implementations, the remaining plausible candidate positions may be ranked in order of reliability, e.g., based on a reliability metric value calculated for each candidate position based on the corresponding subset of RF measurements, as described above. The remaining candidate positions (or “candidates”) 610A-610D may then be evaluated using an iterative process that attempts to find a match between a current candidate and relevant visual location information (vCSI) obtained from the OTT cameras 602 and 604 (and/or one or more shelf cameras), as will be described below with reference to Figure 7.

[0090] Figure 7 illustrates different iterations of a PARR-based matching process 700 for resolving positioning-related ambiguities or errors for the BLE device 610 of Figure 6. For example, process 700 may be initiated after performing the preliminary assessment described above to remove any implausible candidate positions, such as the candidate position 610E in the example of Figure 6. Although only candidate 610E was removed in the above example, only three of the remaining candidate positions (610A-610C) will be considered in the example of Figure 7, where a different candidate position is considered or evaluated at each of the three iterations of process 700. While only three iterations are shown for process 700 in Figure 7, it should be appreciated that process 700 is not intended to be limited thereto and that process 700 may include any number of iterations for evaluating any number of candidate positions. The candidate positions may be evaluated over the plurality of iterations of process 700 in any order as desired for a particular implementation. For example, a candidate position may be randomly selected at each iteration of process 700. In the example of Figure 7, candidate positions 610A, 610B, and 610C are evaluated over first, second, and third iterations of process 700, respectively.

[0091] As in the example of Figure 6, it is assumed that the location of the BLE device 610 in Figure 7 corresponds to the actual or true position of the BLE device within the environment. At each iteration of process 700, a different candidate position of the BLE device 610 within the environment may be evaluated by comparing the corresponding RF -based information with visual information obtained for the candidate position. The visual information (or vCSI) may include images and/or video sequences obtained from one or more cameras located near the candidate position within the environment. For example, a server (e.g., management server 122 of Figure IB or server 500 of Figure 5, as described above) in the ESL system may identify such cameras based on the RF -based position information associated with the candidate position and instruct the identified cameras to collect vCSI for the candidate position. [0092] In some implementations, the server may prioritize OTT cameras located near the candidate position over shelf cameras because the OTT cameras remain active, e.g., as part of a security monitoring system, unlike the shelf cameras that are activated only upon request by the server to conserve power. However, if the current candidate is located in an area with no OTT cameras nearby, the server will activate the nearest shelf cameras and instruct the cameras to collect and disseminate vCSI (e.g., images of the environment within each camera’s field of view). Before selecting a shelf camera to be activated, the server may take into account the remaining life of the camera and select a different camera with more battery life if available. During each iteration of process 700, the images and other vCSI received from a camera may be processed using an object detection model to visually detect objects (e.g., the BLE device 610 or a portion thereof) within the camera’s field of view, and the processed vCSI (including the visual detections of objects) may be fused with RF-based information associated with the current candidate. For example, the RF -based information for the candidate position being evaluated may be compared with the processed vCSI to determine a likelihood of a match between the candidate position and the actual position of the BLE device 610 within the environment. The match likelihood may depend on whether the objects detected in the vCSI captured by a camera located near the candidate position include the BLE device 610.

[0093] The number of iterations and candidate positions considered using process 700 may vary depending on the termination option selected for a particular implementation. For example, several options may be available for deciding when to stop iterating through a set of candidate positions estimated for the BLE device. One option may be to stop iterating through the set of candidate positions upon finding the first candidate position with a match likelihood that meets or exceeds a minimum threshold. Another option may be to run through all viable candidate positions, compute the match likelihood for each, and select the candidate position with the highest match likelihood if more than one candidate exceeds the minimum likelihood threshold.

[0094] In the example of Figure 7, candidate position 610B, which is closest to the actual position of the BLE device 610, has the highest match likelihood as the BLE device 610 is more likely to be detected in the images captured by one or more of the shelf cameras located near this position relative to the images captured by cameras located near the other candidate positions. Therefore, under the first termination option discussed above, process 700 may stop after the second iteration and select candidate position 610B as the estimated position of the BLE device 610 within the environment. However, it should be appreciated that performing all three iterations of process 700 under the second termination option would produce the same result because candidate position 61 OB would still have the highest match likelihood among the three candidate positions 610A-610C.

[0095] It is assumed that the BLE device 610 in the example of Figure 7 periodically communicates with the server of the ESL system, e.g., to report RF measurements of beacons transmitted by ESL devices within the environment. However, there may be a scenario in which the BLE device 610 becomes inactive (e.g., due to a malfunction or loss of power) and stops communicating with the server. In some embodiments, the server may detect such inactivity when no RF measurement reports are received from the BLE device 610 for an extended period of time (e.g., a period lasting longer than a specified delay threshold). The BLE device 610 in this scenario may be considered a non-RF object. Upon detecting that the BLE device 610 is in an inactive state, the server may automatically initiate a search to locate the inactive BLE device 610 (or non-RF object) within the environment. The server may perform the search based on prior RF -based position information that may be available (e.g., from a database used to store RF history) for the BLE device 610 in conjunction with vCSI that may be collected from relevant OTT cameras and/or shelf cameras within the environment.

[0096] In some embodiments, the server may also rely on vCSI obtained from relevant cameras within the environment to locate other types of non-RF objects, such as an object that has no RF capability and for which no prior RF information is available. Such an non-RF object in a retail store environment may be, for example, a non-RF asset (e.g., a product or merchandise item placed on a shelf) or a person of interest (e.g., a store worker, a customer, a lost child, an intruder, or a thief). A search to locate such a non-RF object may be initiated by the server in response to receiving an alert or notification from a control center (e.g., control center 540 of Figure 5, as described above) coupled to the server (e.g., via a wired or wireless communication network). The alert or notification may be triggered either automatically, e.g., upon the activation of a security alarm, or manually, e.g., by an operator via a user interface of the control center.

[0097] To locate the inactive BLE device 610 (or non-RF object) within the environment, the server may rely on prior RF-based position information that may be available (e.g., from a database used to store RF history) for the BLE device 610 in conjunction with relevant vCSI that may be collected from selected cameras within the environment. In some embodiments, the server may determine which cameras to select (and activate if necessary) based on possible trajectories of the BLE device 610 estimated from a last known position of the device, as will be described in further detail below with reference to Figures 8 A and 8B.

[0098] Figure 8A is an overhead view of a retail environment 800A similar to the retail environment 600 of Figure 6 described above. Upon detecting that the BLE device 610 is inactive, the RF history for the BLE device 610 may be used to determine a last known position 802 A of the BLE device 610 within the environment 800 A. Upon detecting that the BLE device 610 has stopped transmitting RF measurements, the server of the ESL system in this example may assume that the device is still in the same aisle as the last known position 802A if the elapsed time since the last RF measurement report received from the device is within a given margin. However, the BLE device 610 may have moved to a different aisle within that time, as shown by an actual trajectory 804 A of the BLE device 610. Accordingly, the vCSI obtained from the shelf cameras located near the last known position 802A will reveal to the server that the BLE device 610 is no longer within the same aisle. Upon detecting that the BLE device 610 has moved to a different aisle, the server may begin activating cameras along estimated trajectories of the BLE device 610, as shown in Figure 8B.

[0099] Figure 8B is an overhead view of a retail environment 800B in which the BLE device 610 has moved from a last known position 802B along a trajectory 804B to its current position. Upon failing to detect the BLE device 610 within the vCSI obtained from shelf cameras located within the aisle corresponding to the last known position 802B, the server may begin activating shelf cameras in an increasing area starting from the last position 802B. For example, the server may activate the shelf cameras along possible trajectories 804B, 806, and 808, which may be estimated based on the prior RF history and prior trajectory information associated with the BLE device 610. The server in this example may determine that trajectory 804B is the actual trajectory upon detecting the BLE device 610 within the vCSI obtained from the shelf cameras located within the aisle corresponding to the current position of the BLE device 610 within the retail environment 800B.

[0100] In some embodiments, the number and/or location of cameras activated for the search of a non-RF object may vary based on a level of urgency associated with the search and/or type of object subject to the search. For example, a search for certain non-RF objects, such as a displaced asset (e.g., a misplaced product) or an inactive BLE device whose RF history is outdated (e.g., over one day old), may be considered non-urgent and not time critical. The server in such non-urgent cases may decide to defer the searching/locating of the non-RF object to save on ESL resources (until more information becomes available in the system). Alternatively, the server may decide to activate random shelf cameras (and/or OTT cameras) to collect vCSI and try to match a camera location with the obj ect’ s current position based on visual traces.

[0101] Conversely, the search may be considered urgent in cases where the need to find the non- RF object is time critical. For example, the non-RF object may be a lost child, an intruder or burglar involved in a security breach, or any object that poses a potential safety hazard. The server in such urgent cases may find and activate all viable cameras within the environment 800B to locate the non-RF object. In some embodiments, the level of urgency of each search may be indicated in a search request transmitted to the server by a control center, e.g., control center 540 of Figure 5, as described above, or an operator of the control center.

[0102] Figure 9 is a flow chart of an example method 900 for indoor positioning of a user device using position ambiguity reduction or resolution according to some embodiments of this disclosure. The user device in this example may be a BLE-enabled device, e.g., BLE device 124 of Figures 1A and IB or BLE device 610 of Figures 6-8B, as described above. The steps of method 900 may be performed by a server (e.g., management server 122 of Figure IB or server 500 of Figure 5, as described above) of an ESL system (e.g., ESL system 100 of Figures 1A and IB, as described above) deployed within a retail environment.

[0103] Method 900 begins at block 902, which includes estimating a plurality of candidate positions of a user device within an indoor environment (e.g., a retail environment as described above with reference to Figures 1-8B) based on positioning information received for the user device. The positioning information may include radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices (e.g., ESL devices 108A-108D of Figure IB).

[0104] At block 904, visual location information for the user device is obtained from one or more cameras located near each candidate position of the plurality of candidate positions.

[0105] At block 906, a match between the visual location information and at least one candidate position of the plurality of candidate positions may be identified based on a comparison between the visual location information and a portion of the positioning information corresponding to each candidate position.

[0106] At block 908, a position of the user device within the indoor environment may be determined based on the at least one candidate position. [0107] Figures 10A-11B are additional flowcharts illustrating example processes for triggering and performing position ambiguity resolution or reduction (PARR) to resolve positioning-related ambiguities, errors, and/or inconsistencies for either an active BLE device or a non-RF object (such as a non-active BLE device) within a retail environment. Figures 10A and 10B pertain to the positioning of BLE devices while Figures 11A and 1 IB pertain to the positioning of non-RF objects. The examples of Figures 10A-1 IB will be described with reference to various components of the ESL system 100 of Figures 1A and IB and the control center 540 of Figure 5 for purposes of discussion and explanation. However, the processes described in these figures are not intended to be limited thereto.

[0108] Figure 10A is a flowchart of an example process 1000A in which the PARR functionality of the server 122 may be triggered based on the detection of trigger conditions or events. The trigger conditions in this example may include a first trigger condition (“Tl”) corresponding to unreliable RF measurements from the BLE device 124 and a second trigger condition (“T2”) corresponding to a mismatch between visual information obtained from the OTT cameras 102 and the RF -based positioning information obtained from the BLE device 124.

[0109] At block 1002 of process 1000 A, the server 122 may calculate reliability metrics for a position of the BLE device 124 within the environment based on RF measurements of the ESL radio beacons obtained from the BLE device 124 and vCSI obtained from the OTT cameras 102. At block 1004, the server 122 may determine whether the metrics calculated at block 1002 indicate that the RF measurements are unreliable (whether condition Tl is triggered) or that the RF measurements are inconsistent with the vCSI (whether condition T2 is triggered). If either condition is triggered, the server 122 may decide to increase the duty cycle of the ESL radio beacons at block 1006 and send appropriate instructions to the corresponding ESL devices at block 1008. At block 1010, the server 122 may calculate additional reliability metrics based on new RF measurements received from the BLE device 124 after increasing the duty cycle of the ESL devices. At block 1012, the server 122 uses the reliability metrics calculated at block 1010 to again determine whether any of the trigger conditions (Tl or T2) have been detected. Upon detecting either condition, the PARR functionality for the BLE device 124 may be activated at block 1014 and performed using process 1000B of Figure 10B.

[0110] Figure 10B is a flow diagram of an example process 1000B for locating the BLE device of Figure 10A after the PARR functionality has been activated by the server 122 according to some embodiments of this disclosure. At block 1016 of process 1000B, the server 122 may generate subsets of the RF positioning information received from the BLE device 124 (e.g., at block 1010 of Figure 10A) and use each subset to estimate a different candidate position in a list of candidate positions for the BLE device 124 within the environment. At block 1018, the server 122 may set a current position of the BLE device 124 to one of the estimated candidate positions selected at random from the list of candidates.

[OHl] At block 1020, the server 122 identifies relevant OTT cameras 102 and/or shelf cameras 104 (e.g., one or more of shelf cameras 104A-104D of Figure IB) located near the current candidate position of the BLE device 124 and requests vCSI from the identified cameras. At block 1022, the server attempts to find a candidate position of the BLE device that matches a visual detection of the BLE device within the vCSI obtained from the corresponding OTT cameras 102 and/or the shelf cameras 104.

[0112] At block 1024, the server 122 may iterate through remaining candidate position on the list until reaching a termination condition. As described above, one termination option may be to stop iterating through the list of candidate positions upon finding the first candidate position with a match likelihood that crosses or exceeds a minimum threshold. Another option may be to run through all viable candidate positions, compute the match likelihood for each, and select the candidate position with the highest match likelihood if more than one candidate exceeds the minimum likelihood threshold.

[0113] Figure 11 A is a flow diagram of an example process 1100A for locating a non-RF object within a retail environment in response to an urgent request according to some embodiments of this disclosure. For example, process 1100A may be initiated by the server 122 upon receiving a control message with the urgent request from control center 540. Figure 1 IB is a flow diagram of an example process 1100B for locating a non-RF object within the retail environment in response to receiving a similar control message with a non-urgent request according to some embodiments of this disclosure.

[0114] In both urgent and non-urgent cases, the server 122 may attempt to find a possible match between the non-RF object and the objects detected within the vCSI obtained from relevant OTT cameras 102 and/or shelf cameras 104 located within the environment. As described above, the server 122 may start by activating cameras within a search area surrounding a last known position of the object. The size of the search area and number of cameras activated may be gradually increased until the non-RF object is found.

[0115] In some cases (e.g., where the non-RF object is an inactive BLE device), the last known position may be determined from prior RF measurements obtained from a RF history associated with the object. In such cases, the cameras that are activated may be located along one or more possible trajectories of the non-RF object that may be estimated from the RF history.

[0116] It is noted that one or more blocks (or operations) described with reference to Figures 1- 4 may be combined with one or more blocks (or operations) described with reference to another of the figures. For example, one or more blocks (or operations) of Figure 5 may be combined with one or more blocks (or operations) of Figures 1-4. As another example, one or more blocks associated with Figure 9 may be combined with one or more blocks associated with Figures 1-4.

[0117] In one or more aspects, techniques for supporting ESL systems may include additional aspects, such as any single aspect or any combination of aspects described below or in connection with one or more other processes or devices described elsewhere herein. In a first aspect, a method comprises: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

[0118] In a second aspect, in combination with the first aspect, determining the plurality of candidate positions comprises: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

[0119] In a third aspect, in combination with one or more of the first aspect or the second aspect, the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device. [0120] In a fourth aspect, in combination with one or more of the first aspect through the third aspect, the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme.

[0121] In a fifth aspect, in combination with one or more of the first aspect through the fourth aspect, prior to determining the plurality of candidate positions, the method further comprises: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

[0122] In a sixth aspect, in combination with one or more of the first aspect through the fifth aspect, determining the ambiguity comprises: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

[0123] In a seventh aspect, in combination with one or more of the first aspect through the sixth aspect, the environment is a retail floor space, and wherein the plurality of ESL devices are attached to shelves on gondolas located within the retail floor space.

[0124] In an eighth aspect, in combination with one or more of the first aspect through the seventh aspect, each of the gondolas include one or more shelf cameras, and wherein the visual information includes images captured by one or more shelf cameras attached to at least one of the gondolas located near the updated position of the user device.

[0125] In a ninth aspect, in combination with one or more of the first aspect through the eighth aspect, the visual information further includes additional images captured by one or more over-the-top cameras associated with a security monitoring system of the retail floor space.

[0126] In a tenth aspect, in combination with one or more of the first aspect through the ninth aspect, the one or more over-the-top cameras are associated with a legacy surveillance system deployed within the indoor environment.

[0127] In an eleventh aspect, an apparatus comprises a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including: determining the plurality of candidate positions comprises: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

[0128] In a twelfth aspect, in combination with the eleventh aspect, the operations further include: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

[0129] In a thirteenth aspect, in combination with one or more of the eleventh aspect or the twelfth aspect, the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device.

[0130] In a fourteenth aspect, in combination with one or more of the eleventh aspect through the thirteenth aspect, the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme.

[0131] In a fifteenth aspect, in combination with one or more of the eleventh aspect through the fourteenth aspect, the operations further include: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

[0132] In a sixteenth aspect, in combination with one or more of the eleventh aspect through the fifteenth aspect, the operations further include: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

[0133] In a seventeenth aspect, in combination with one or more of the eleventh aspect through the sixteenth aspect, the environment is a retail floor space, and wherein the plurality of ESL devices are attached to shelves on gondolas located within the retail floor space.

[0134] In an eighteenth aspect, in combination with one or more of the eleventh aspect through the seventeenth aspect, each of the gondolas include one or more shelf cameras, and wherein the visual information includes images captured by one or more shelf cameras attached to at least one of the gondolas located near the updated position of the user device. [0135] In a nineteenth aspect, in combination with one or more of the eleventh aspect through the eighteenth aspect, the visual information further includes additional images captured by one or more over-the-top cameras associated with a security monitoring system of the retail floor space.

[0136] In a twentieth aspect, in combination with one or more of the eleventh aspect through the nineteenth aspect, the one or more over-the-top cameras are associated with a legacy surveillance system deployed within the indoor environment.

[0137] In a twenty-first aspect, a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

[0138] In a twenty-second aspect, in combination with the twenty-first aspect, the operations further comprise: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

[0139] In a twenty -third aspect, in combination with one or more of the twenty-first aspect or the twenty-second aspect, the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device.

[0140] In a twenty-fourth aspect, in combination with one or more of the twenty-first aspect through the twenty -third aspect, the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme. [0141] In a twenty-fifth aspect, in combination with one or more of the twenty-first aspect through the twenty-fourth aspect, the operations further comprise: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

[0142] In a twenty-sixth aspect, in combination with one or more of the twenty-first aspect through the twenty-fifth aspect, the operations further comprise: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

[0143] In a twenty-seventh aspect, an electronic shelf label (ESL) system comprises: a server comprising a first memory; and a first at least one processor coupled to the first memory and configured to perform operations comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions; and an electronic shelf label (ESL) device comprising a second memory; and a second at least one processor coupled to the second memory and configured to perform operations comprising: transmitting beacon signals for reception by the user device at a first periodicity; receiving instructions from the server to adjust the first periodicity of the beacon signals; and transmitting the beacon signals for reception by the user device at a second periodicity in accordance with the instructions from the server.

[0144] In a twenty-eighth aspect, in combination with the twenty-seventh aspect, the operations performed by the first at least one processor further comprise: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of the beacon signals received by the user device from the ESL device; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information. [0145] In a twenty-ninth aspect, in combination with one or more of the twenty-seventh aspect or the twenty-eighth aspect, the operations performed by the first at least one processor further comprise: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

[0146] In a thirtieth aspect, in combination with one or more of the twenty-seventh aspect through the twenty-ninth aspect, the user device is a Bluetooth Low Energy (BLE)- enabled mobile device, and wherein the ESL device includes BLE radios that transmit the beacon signals measured by the user device.

[0147] Components, the functional blocks, and the modules described herein with respect to the figures described above include processors, electronics devices, hardware devices, electronics components, logical circuits, memories, software codes, firmware codes, among other examples, or any combination thereof. Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, application, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, and/or functions, among other examples, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise. In addition, features discussed herein may be implemented via specialized processor circuitry, via executable instructions, or combinations thereof.

[0148] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. Skilled artisans will also readily recognize that the order or combination of components, methods, or interactions that are described herein are merely examples and that the components, methods, or interactions of the various aspects of the present disclosure may be combined or performed in ways other than those illustrated and described herein.

[0149] The various illustrative logics, logical blocks, modules, circuits and algorithm processes described in connection with the implementations disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. The interchangeability of hardware and software has been described generally, in terms of functionality, and illustrated in the various illustrative components, blocks, modules, circuits and processes described above. Whether such functionality is implemented in hardware or software depends upon the particular application and design constraints imposed on the overall system.

[0150] The hardware and data processing apparatus used to implement the various illustrative logics, logical blocks, modules and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose single- or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or any conventional processor, controller, microcontroller, or state machine. In some implementations, a processor may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some implementations, particular processes and methods may be performed by circuitry that is specific to a given function.

[0151] In one or more aspects, the functions described may be implemented in hardware, digital electronic circuitry, computer software, firmware, including the structures disclosed in this specification and their structural equivalents thereof, or in any combination thereof. Implementations of the subject matter described in this specification also may be implemented as one or more computer programs, that is one or more modules of computer program instructions, encoded on a computer storage media for execution by, or to control the operation of, data processing apparatus.

[0152] If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The processes of a method or algorithm disclosed herein may be implemented in a processor-executable software module which may reside on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that may be enabled to transfer a computer program from one place to another. A storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such computer-readable media may include random-access memory (RAM), read-only memory (ROM), electrically erasable programmable readonly memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection may be properly termed a computer-readable medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and instructions on a machine readable medium and computer-readable medium, which may be incorporated into a computer program product.

[0153] Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to some other implementations without departing from the spirit or scope of this disclosure. Thus, the claims are not intended to be limited to the implementations shown herein but are to be accorded the widest scope consistent with this disclosure, the principles and the novel features disclosed herein.

[0154] Certain features that are described in this specification in the context of separate implementations also may be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also may be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0155] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Further, the drawings may schematically depict one more example processes in the form of a flow diagram. However, other operations that are not depicted may be incorporated in the example processes that are schematically illustrated. For example, one or more additional operations may be performed before, after, simultaneously, or between any of the illustrated operations. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products. Additionally, some other implementations are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results.

[0156] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMS WHAT IS CLAIMED IS:

1. A method comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

2. The method of claim 1, wherein determining the plurality of candidate positions comprises: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

3. The method of claim 2, wherein the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device.

4. The method of claim 3, wherein the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme.

5. The method of claim 4, wherein, prior to determining the plurality of candidate positions, the method further comprises: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

6. The method of claim 2, wherein determining the ambiguity comprises: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

7. The method of claim 2, wherein the environment is a retail floor space, and wherein the plurality of ESL devices are attached to shelves on gondolas located within the retail floor space.

8. The method of claim 7, wherein each of the gondolas include one or more shelf cameras, and wherein the visual information includes images captured by one or more shelf cameras attached to at least one of the gondolas located near the position of the user device.

9. The method of claim 8, wherein the visual information further includes additional images captured by one or more over-the-top cameras associated with a security monitoring system of the retail floor space.

10. The method of claim 9, wherein the one or more over-the-top cameras are associated with a legacy surveillance system deployed within the indoor environment.

11. An apparatus, comprising: a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

12. The apparatus of claim 11, wherein the operations further include: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

13. The apparatus of claim 12, wherein the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device.

14. The apparatus of claim 13, wherein the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme.

15. The apparatus of claim 14, wherein the operations further include: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

16. The apparatus of claim 12, wherein the operations further include: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

17. The apparatus of claim 12, wherein the environment is a retail floor space, and wherein the plurality of ESL devices are attached to shelves on gondolas located within the retail floor space.

18. The apparatus of claim 17, wherein each of the gondolas include one or more shelf cameras, and wherein the visual information includes images captured by one or more shelf cameras attached to at least one of the gondolas located near the position of the user device.

19. The apparatus of claim 18, wherein the visual information further includes additional images captured by one or more over-the-top cameras associated with a security monitoring system of the retail floor space.

20. The apparatus of claim 19, wherein the one or more over-the-top cameras are associated with a legacy surveillance system deployed within the indoor environment.

21. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions.

22. The non-transitory, computer-readable medium of claim 21, wherein the operations further comprise: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of beacon signals received by the user device from a plurality of electronic shelf label (ESL) devices; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

23. The non-transitory, computer-readable medium of claim 22, wherein the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the plurality of ESL devices include BLE radios that transmit the beacon signals measured by the user device.

24. The non-transitory, computer-readable medium of claim 23, wherein the plurality of ESL devices transmit the beacon signals according to a Time Division Multiple Access (TDMA) scheme.

25. The non-transitory, computer-readable medium of claim 24, wherein the operations further comprise: transmitting, via a network to the plurality of ESL devices, instructions for the plurality of ESL devices to increase a periodicity of the beacon signals transmitted over time.

26. The non-transitory, computer-readable medium of claim 22, wherein the operations further comprise: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

27. An electronic shelf label (ESL) system, comprising: a server comprising a first memory; and a first at least one processor coupled to the first memory and configured to perform operations comprising: determining a plurality of candidate positions of a user device within an indoor environment based on positioning information received for the user device; receiving visual information for the user device from one or more cameras, each of the one or more cameras having a field of view including at least one candidate position of the plurality of candidate positions; determining a subset of the plurality of candidate positions that satisfy a criteria based on a comparison of positioning information for each of the plurality of candidate positions with the visual information; and determining a position of the user device within the indoor environment based on the subset of the plurality of candidate positions; and an electronic shelf label (ESL) device comprising a second memory; and a second at least one processor coupled to the second memory and configured to perform operations comprising: transmitting beacon signals for reception by the user device at a first periodicity; receiving instructions from the server to adjust the first periodicity of the beacon signals; and transmitting the beacon signals for reception by the user device at a second periodicity in accordance with the instructions from the server.

28. The ESL system of claim 27, wherein the operations performed by the first at least one processor further comprise: receiving the positioning information from the user device, the positioning information including radio frequency (RF) measurements of the beacon signals received by the user device from the ESL device; determining an ambiguity in the position of the user device based on the received positioning information; and responsive to the ambiguity, determining each candidate position of the plurality of candidate positions based on a corresponding subset of the positioning information.

29. The ESL system of claim 28, wherein the operations performed by the first at least one processor further comprise: determining a reliability metric for the positioning information; determining whether the reliability metric meets a minimum reliability threshold; and determining the ambiguity upon determining that the reliability metric does not meet the minimum reliability threshold.

30. The ESL system of claim 27, wherein the user device is a Bluetooth Low Energy (BLE)-enabled mobile device, and wherein the ESL device includes BLE radios that transmit the beacon signals measured by the user device.