WO2021064747A1

WO2021064747A1 - System and method for location relevant augmented reality based communication

Info

Publication number: WO2021064747A1
Application number: PCT/IN2020/050838
Authority: WO
Inventors: Sunil Maheshwari; Lekh Omprakash Joshi
Original assignee: Triumb Technologies Pvt Ltd
Current assignee: Triumb Technologies Pvt Ltd
Priority date: 2019-10-01
Filing date: 2020-09-29
Publication date: 2021-04-08
Anticipated expiration: 2022-04-01

Abstract

Embodiments provide methods and systems for facilitating location relevant augmented reality based communication. The method performed by server includes receiving a request for accessing a communication thread associated with a location. The request includes at least a location data of the location from a user device of a user. The method includes determining the communication thread associated with the location based, at least in part, on an association data stored in a communication thread database. The communication thread database stores one or more association data including mappings of a set of locations with corresponding communication threads. The method further includes transmitting one or more instruction sets to the user device to generate an augmented reality scene of the location including the communication thread in the user device.

Description

SYSTEM AND METHOD FOR LOCATION RELEVANT AUGMENTED REALITY BASED COMMUNICATION

Cross-reference to related applications

This Application is International patent application stage filing of an Indian provisional patent application number IN/201941039629, filed on October 01, 2019. The disclosure of Indian provisional patent application number IN/201941039629 is incorporated herein by reference in its entirety.

The present disclosure generally relates to communication systems and in particular relates to a location relevant communication thread by placing the communication thread, virtually as an augmented reality scene at a relevant location.

Background

There has been an evolution in communication systems from paper based to voice based to electronic device based systems. There have been many communication systems in use implemented in electronic systems mostly in cellular medium or the internet or both. For example, there are electronic mail systems (email) which let users communicate with each other over a variety of network of electronic communication systems. There are various social media based applications operating over a variety of networks, by which the users can communicate in various objects of communication like text, image, multimedia or combinations thereof.

As the medium of electronic communication has increased, the volume of communication that users receive and/or send has also increased. However, creation and/or access of communication thread which is relevant or contextual or both to a location continue to be a problem. For example, if a person or a user is at a particular location, a solution to facilitate an access of communication threads which is relevant or contextual or both, to the location of access does not exist.

Hence, there has been a long felt need to overcome the limitation of access of communication which is contextual or relevant or both to a specific location.

Various embodiments of the present disclosure provide methods and systems for facilitating location relevant communication in augmented reality environment.

In one embodiment, a method for facilitating location relevant communications is disclosed. The method performed by a server includes receiving a request for accessing a communication thread associated with a location. The request includes at least a location data of the location from a user device of a user. The method includes determining the communication thread associated with the location based, at least in part, on an association data stored in a communication thread database. The communication thread database stores one or more association data including mappings of a set of locations with corresponding communication threads. The method further includes transmitting one or more instruction sets to the user device to generate an augmented reality scene of the location including the communication thread in the user device.

In another embodiment, a server for facilitating location relevant communications is disclosed. The server includes a communication interface, a communication thread database, a memory including executable instructions, and a processor communicably coupled to the communication interface and the memory. The processor is configured to execute the executable instructions to cause the server to receive a request for accessing a communication thread associated with a location. The request includes at least a location data of the location from a user device of a user. The server is further caused to determine the communication thread associated with the location based, at least in part, on an association data stored in a communication thread database. The communication thread database stores one or more association data including mappings of a set of locations with corresponding communication threads. The server is further caused to transmit one or more instruction sets to the user device to generate an augmented reality scene of the location including the communication thread in the user device.

In yet another embodiment, a method is disclosed. The method performed by the server includes receiving an augmented reality scene of a location and location data of the location from the user device. The method includes receiving a communication thread from the user device. The method includes placing the communication thread within the augmented reality scene to form an association data including a mapping of the communication thread and the location. The method further includes storing the association data in a communication thread database for providing access to the association data to one or more authorized users of the communication thread upon receipt of the request for accessing the communication thread associated with the location.

Other aspects and example embodiments are provided in the drawings and the detailed description that follows.

Object of the Invention

The object of this invention is to facilitate location relevant communication among users.

Another object of this invention is to enable users to provide access to a plurality of communication threads at a particular location.

Yet another object of this invention is to provide users a communication system, wherein unless and until a user is in the proximity of a location, communication thread is not accessible.

Yet another object of this invention is to provide a location relevant broadcast method to the users.

For a more complete understanding of example embodiments of the present technology, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:

Fig.1

illustrates an example representation of an environment related to at least some example embodiments of the present disclosure.

Fig.2

illustrates a block diagram of the user device, in accordance with an embodiment of the present disclosure.

Fig.3

illustrates a block diagram of location relevant messaging system, in accordance with an embodiment of the present disclosure.

Fig.4A

is an example representation of a communication thread created at a particular location in an augmented reality scene accessible to one or more users, in accordance with an embodiment of the present disclosure.

Fig.4B

is an exemplary user interface depicting a map view of communication threads accessible to a user at multiple locations, in accordance with an embodiment of the present disclosure.

Fig.5

is an example representation of a communication thread accessible to one or more users, in accordance with an embodiment of the present disclosure.

Fig.6

illustrates a block diagram of plurality of the communication threads in the form of virtually placed augmented reality scenes placed in the proximity of about the same location and where multiple users can access said communication threads, in accordance with an embodiment of the present disclosure.

Fig.7

illustrates a flowchart of an operation of a user device with location identifying circuit, in accordance with an embodiment of the present disclosure.

Fig.8

illustrates a flowchart of an operation of a user device without location identifying circuit, in accordance with an embodiment of the present disclosure.

Fig.9

illustrates a flowchart of an operation of the server having communication threads stored in its memory, in accordance with an embodiment of the present disclosure.

Fig.10

is a simplified block diagram of a server, in accordance with an embodiment of the present disclosure.

Fig.11

is a simplified block diagram of a user device, without location identifying circuit, capable of implementing at least some embodiments of the present disclosure.

The drawings referred to in this description are not to be understood as being drawn to scale except if specifically noted, and such drawings are only exemplary in nature.

Detailed Description

The following description is presented to enable any person skilled in the art to make and use the present disclosure, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily understood to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but is to be consistent with the widest scope consistent with the claims.

Reference in this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrase "in an embodiment" in various places in the specification is not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.

Moreover, although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to said details are within the scope of the present disclosure. Similarly, although many of the features of the present disclosure are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the present disclosure is set forth without any loss of generality to, and without imposing limitations upon, the present disclosure.

The term "communication" used herein is intended to denote imparting or exchanging information by speaking, writing, any other human sensory stimulus, human sensory stimuli, machine readable stimulus or combinations thereof.

The term "user device" used herein is intended to denote at least one of hardware, software and combinations of hardware and software including of at least one processor alternatively termed as a microprocessor, memory, an input device and an output device.

The term "virtual" used herein is intended to denote not a physically existing but made by a user device to appear to do so. The term "reality" used herein is intended to denote existential state. The reality includes tangibles or intangibles in the real world. Such renderings or reproductions can include by way of example, an image, a (screenshot) shot, photo, video, live stream of a physical scene and/or its visible component or recordings or (live) stream of an audible component, e.g., sound of an airplane, traffic noise, Niagara falls, birds chirping.

The terms “extended reality” or “augmented reality” refer to a technology displaying one or more two-dimensional or three-dimensional objects on a device such as a mobile device, a tablet computer or a head mount display device so that the one or more two-dimensional or three-dimensional objects exist in an environment of the user device. The environment of the user can be captured by a camera of the device and displayed together with the one or more two-dimensional or three-dimensional objects. Alternatively, the user can see the environment of the customer through a transparent display that shows one or more two-dimensional or three-dimensional objects. Although the terms "augmented reality" or "AR" are used in this disclosure, the term "mixed reality", "MR", hybrid reality, meta-reality or any other terminologies can be applicable in a similar manner. In other words, the augmented reality technology superimposes an image generated by electronic generation system on a user's view of the real world, thus providing a composite view.

The term "message" used herein denotes recorded communication sent to or left for at least one of the recipients.

The term "communication thread" used herein denotes a group of linked messages, and messages may include a text message, an audio message, a video message, or any other graphical message.

OVERVIEW: The technical problem solved by the present disclosure is disassociation between user locations and accessibility of communication threads to the user at the user locations, leading to a bad user experience. In accordance with the present disclosure, and based on the observation, the system and method embodying the principle of associating the location data with the communication thread is described.

The present invention provides techniques for creating communication threads (i.e. group of linked messages), and associating the communication threads to particular locations, and storing the communication threads associated with corresponding locations in a database, and providing access to the communication threads to authorized users. The present invention provides techniques for associating the communication threads to the corresponding locations, and provides pre-defined manners in which the communication threads can be provided to the intended users in form of augmented reality scenes on their user devices. For instance, the present invention provides an application, using which a user A can create a communication thread T1 for a location L1, and using the embodiments of the present invention, the thread T1 can be placed in an augmented reality scene associated with the location L1, and associated data (the thread T1, augmented reality scene and the location L1) can be stored in the database. Further, when another user B (or even the same user A) is at the location L1, he/she can access the augmented reality scene of the location L1 through the application, and the communication thread T1 (previously stored as part of the association data) can be superimposed in the augmented reality scene displayed to the user B (or the same user A).

The present disclosure provides another dimension to digital communication through integration with the real-world environment and real-world contexts to enhance relevancy, and/or vanity value through optimized and contextual location relevant communication setting. In general, the communication threads depicted via the disclosed system can be contextually (e.g., location specific, or user-specific, etc.) relevant and/or contextually aware. Specifically, the communication threads can have attributes that are associated with relevant real-world places, real-world events, real-world entities, real-world things, real-world objects, and/or times of the physical world, and thus their deployment as an augmentation of a digital experience provides additional real life utility.

Various example embodiments of the present disclosure are described hereinafter with reference to FIGS. 1 to 11.

FIG. 1 illustrates an exemplary representation of an environment 100 related to at least some example embodiments of the present disclosure. Although the environment 100 is presented in one arrangement, other embodiments may include the parts of the environment 100 (or other parts) arranged otherwise depending on, for example, enabling location relevant communication in augmented reality, etc. The environment 100 generally includes a plurality of entities, for example, a server 102, a plurality of

users

108a, 108b, 108c, and 108d each coupled to, and in communication with (and/or with access to) a network 106. Each user is associated with a user device. For example, the user 108a is associated with a user device 110a.

In an embodiment, the user device 110a is equipped with a functionality that enables the user 108a to virtually place augmented reality scene at a particular location. The network 106 may include, without limitation, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, a virtual network, and/or another suitable public and/or private network capable of supporting communication among the entities illustrated in FIG. 1, or any combination thereof. Various entities in the environment 100 may connect to the network 106 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), 2nd Generation (2G), 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G) communication protocols, Long Term Evolution (LTE) communication protocols, or any combination thereof.

The server 102 is configured to perform one or more of the operations described herein. In general, the server 102 is configured to store one or more association data in a communication thread database 104 that includes mappings of a set of locations with corresponding communication threads. In an embodiment, the server 102 stores and manages a location-relevant communication enabled application 116, which can be made available on the user devices of the

users

108a, 108b, 108c, and 108d. Hence, the server 102 is configured to localize a communication thread virtually as an augmented reality scene that associates the communication thread with its localization.

Examples of the

user devices

110a and 110b may include a mobile phone, a smart telephone, a computer, a laptop, a PDA (Personal Digital Assistant), a Mobile Internet Device (MID), a tablet computer, an Ultra-Mobile personal computer (UMPC), a phablet computer, a handheld personal computer and the like. In some embodiments, the location-relevant communication enabled application 116 (e.g., a phone application, a messaging application, etc.), interchangeably referred as “mobile application” throughout the description, may be installed on the user device (such as,

user device

110a, 110b). The

user devices

110a and 110b are capable of real-world imaging and providing users with the real-world image augmented with communication threads associated with particular location points. In general, the communication threads are generally digitally rendered or synthesized by the user devices to be presented in the AR environment. The detailed explanation for the user device is explained with reference to FIG. 2.

In one embodiment, the user 108a (i.e., a thread creating user) may place or create a communication thread in an augmented reality environment by pointing his/her user device 110a at a particular location (see, 112). The particular location can be any suitable location such as a specific set of longitude and latitude coordinates, a set of GPS coordinates, a store front identified by a photo of the store front, a park bench, an office department, a restaurant, etc. Thereafter, the server 102 is configured to receive the location data from the user device 110a of the thread creating user 108a and map the location data and the communication thread to form an association data stored in the communication thread database 104.

In one embodiment, the user 108a drafts, writes or composes a communication message having augmented reality (“AR”) content. The AR content can include one or more digitally rendered real-world objects. The communication message can include 2D or 3D graphical rendering, which can include one or more of: text, images, audio, video, or computer graphics animation. The communication thread can appear and be accessed (preview, view, shared, edited, modified), acted on and/or interacted with via an imaging device such as a smartphone camera, wearable device such as an augmented reality (AR) or virtual reality (VR) headset, any wearable technology, AR glasses, wearable smart watch, wearable computer, heads up display, etc. In one example, if the user 108a virtually places the communication thread visually at the particular location, the communication thread can be saved to that particular location or near that particular location or within a range of the particular location. The user 108a can also place and save the communication thread for example, at any angle e.g., 10 degrees to the right of their front position at the particular location.

In one example, when a user (such as, user 108a) wants to place a communication thread on a physical object (i.e., door of HR office department) positioned at a particular location (e.g., see 112), at first, the user 108a needs to identify the location of point on the physical object where the communication thread will be created in an augmented reality scene. The user device 110a associated with the user 108a then captures the particular location 112 and sends a communication message along with location data of the particular location 112 for initiating a communication thread to the server 102. In one embodiment, the server 102 is configured to map or associate the location data of the particular location to the communication thread to form an association data and store the association data in the communication thread database 104.

The thread creating user i.e., the user 108a may also set access permissions for another user (for example, user 108b) or one or more users so as to superimpose the communication thread in the form of an augmented reality scene at a location selected in the user's view. It is understood that the communication thread set by the user 108a may only be available to those users who have the permission to view it. In one embodiment, the plurality of users may have different augmented reality scenes based on the access permissions set by the thread creating users for corresponding communication threads. For example, a first user may view a first augmented reality scene superimposed with a first communication thread, while a second user may view a totally different augmented reality scene with a second communication thread.

In one embodiment, the thread creating user 108a may set location access requirement for accessing the communication thread. In particular, the user 108b may require being present at the particular location for accessing the communication thread. In another example, the user 108b may not require being present at the particular location for accessing the communication thread. In other words, when the location access requirement is set by the thread creating user 108a, then any user who wishes to access the communication thread, needs to be physically present at the location and point camera of his user device to intended views of the location using the location-relevant communication enabled application 116.

In one embodiment, the communication thread associated with a particular location may be displayed on a particular physical object in the AR environment. In another embodiment, the communication thread may be displayed and not attached to any object (for example, in front of person’s field of view).

Whenever another user 108b (or even the thread creating user 108a) is at or in proximity to the particular location 112 (e.g., HR office department) where the communication thread was placed as an augmented reality object and points the user device at the particular location 112, the server 102 is configured to transmit one or more instruction sets to the user device to generate an augmented reality scene of the location in the user device. The augmented reality scene includes the communication thread set by the thread creating user 108a and real-world image associated with the particular location. The another user 108b can also add a communication message in the communication thread. In response to addition of the communication message in the communication thread, the server 102 updates the communication thread corresponding to the particular location stored in the communication thread database 104.

In particular, when another user 108b, with the access permission for the communication thread, goes to the particular location or in proximity to the particular location 112 (e.g., HR office department), the server 102 determines the communication thread associated with the particular location based on one or more association data stored at the communication thread database 104 and sends the recent communication thread associated with the particular location to the user device. In one embodiment, when the another user 108b accesses the augmented reality scene of the particular location through the location-relevant communication enabled application 116 available on the user device of the another user 108b by pointing the camera of his/her user device to the particular location, the server 102 is configured to display the communication thread associated with the particular location to the user device. More illustratively, the data from the server 102 would be loaded back to the user device and would be available for communication over at least one of display on the user device’s camera so as to get the communication thread superimposed on the user's view on display or sound on speaker or human sensory stimulus.

In one embodiment, the server 102 is configured to transmit a notification to one or more authorized users and the thread creating user 108a associated with the communication thread when a change in the association data of the communication thread occurs.

In similar manner, the user 108c may also place a communication thread in an augmented reality environment at a location 114 (e.g., Book Shelf) in a home. The user 108c (or another user 108d) is able to access the communication thread later using his user device in the augmented reality scene by pointing his user device towards the location 114.

While creating the communication thread by the user 108a or the user 108c (i.e., a thread creating user), the server 102 is configured to receive location data along with the communication thread from the user device associated with the user 108a or the user 108c. The server 102 is configured to map or associate the location data and the communication thread to form an association data. The communication thread is a group of linked messages. In one embodiment, the server 102 is configured to configure the association data to virtually place the augmented reality scene at the particular location upon receipt of the request for accessing the communication thread associated with the location by an authorized user of the one or more authorized users. Further, the server 102 is configured to transmit one or more instruction sets including the configured association data to the authorized user. The communication thread is specific to a specific user and the location where it needs to be relevant.

The number and arrangement of systems, devices, and/or networks shown in FIG. 1 are provided as an example. There may be additional systems, devices, and/or networks; fewer systems, devices, and/or networks; different systems, devices, and/or networks; and/or differently arranged systems, devices, and/or networks than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally, or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of the environment 100.

Referring now to FIG. 2, a simplified block diagram of a user device 200 is shown, in accordance with an embodiment of the present disclosure. The user device 200 is similar to the

user devices

110a and 110b. The user device 200 includes a processor 202, a memory 204, at least one input 206, and at least one output 208. The user device 200 can also be a machine capable of accessing messages or communication thread without human intervention. The user device 200 can connect to the plurality of other user devices. Alternatively, the user device 200 may be mobile phone, VR headset with joystick or any other device known to the art including at least one micro-processor, input, output and storage memory.

In one embodiment, the user device 200 includes a location identifying circuit 210 for determining location data associated with the user device 200. The location identifying circuit 210 may implement positioning technology such as, global positioning system (GPS), a global navigation satellite system (GLONASS), a compass navigation system (COMPASS) or BeiDou-2 or BDS, a NAVIC or Navigation with Indian Constellation, a Galileo positioning system, a quasi-zenith satellite system (QZSS), a wireless fidelity (WiFi) positioning technology, marker based Localisation Techniques, localisation using photos captured and assisted by user, methods using an anchor set by user, localisation techniques using the underlying platforms on smartphone device, user assisted localisation techniques, methods of localisation using beacons like Bluetooth beacons, triangulation method of localisation, localisation using people recognition, localisation using compass, localisation using celestial object positioning or the like, or any other positioning technology known to the art or any combination thereof.

In another embodiment, when the user device 200 does not have the location identifying circuit 210, the user 108a inputs the location along with a communication message to the communication thread. The communication message is appended to the referred communication thread placed at the selected location. In one embodiment, the communication message is appended to the referred communication thread placed at a default location or a location with pre-configuration location set. In one embodiment, the user device 200 may place the communication thread at pre-configured location sets.

In one embodiment, the user device 200 places the communication thread at a location by localising techniques such as EKF SLAM, FastSLAM 1.0, FastSLAM 2.0, L-SLAM, GraphSLAM, Occupancy Grid SLAM, DP-SLAM, Parallel Tracking and mapping(PTAM), LSD-SLAM, S-PTAM, ORB-SLAM, ORB-SLAM2, OrthoSLAM, MonoSLAM, CoSLAM, SeqSlam, iSAM (Incremental Smoothing and Mapping), CT-SLAM (Continuous Time), RGB-D SLAM or any other technique known to the art or combinations thereof.

In one embodiment, a thread creating user 108a initiates a communication via a text, image or multimedia using a mobile application 116 installed on his/her user device and places a virtual communication trail (i.e., a communication message log) as an augmented reality object at a particular location identified by the location identifying circuit 210. In particular, the thread creating user 108a superimposes the communication thread within an augmented reality scene at the particular location. The thread creating user 108a also provides access permissions to one or more users for viewing the communication thread in the same augmented reality scene at the particular location.

In one embodiment, a user different from the thread creating user sends a request for accessing communication threads associated with the particular location to the server 102. The request includes, but is not limited to, location data of the particular location and user identifier. The server 102 verifies access permissions associated with one or more communication threads associated with the particular location. Based on a successful verification of the user for a specific communication thread, the server 102 may transmit a signal to the user device 200 for generating the augmented reality scene at the particular location including the specific communication thread in the user device.

Thus, one or more users having access to a communication thread are able to access the augmented reality based communication thread while pointing to the same location or in its proximate range where the communication thread was created. The user can also append communication messages to the virtually placed augmented reality based communication thread.

In one embodiment, the thread creating user 108a and one or more authorized users with the access permission for the communication thread can update, delete, edit, or merge the communication thread. Further, the thread creating user 108a and the one or more users can modify the location associated with the communication thread or replicate the same communication thread at different locations. The thread creating user 108a and the one or more authorized users can also modify an access permission of the communication thread.

With this technical solution, the onus of remembering or bookkeeping the context of communication thus gets eliminated. For example, at home, a user can create an augmented reality based note for personal details relevant to his home placed at home location. While at the office or workplace, he will be able to create or communicate with an already created augmented reality based communication thread relevant to office. In an alternate way, a user can also change the location of the communication thread to be replicated, or changed to another location or both.

FIG. 3 illustrates a block diagram of a location relevant messaging system 300, in accordance with an embodiment of the present disclosure. As mentioned previously, the user device 118a accesses the server 102 through the network 106 and receives a communication thread that is virtually placed at a particular location. The network 106 can be one of: cellular network, a wifi network, a private network or a combination thereof.

As shown in the FIG. 3, the location relevant messaging system 300 includes, but is not limited to,

servers

306, 310 and 316, and users "user 1" 302, “user2"308....."user N" 314. A first user 302 is connected to a first network 304 by which the first user 302 initiates a request for accessing or creating a communication thread for a particular location. The access goes through the first network 304 to a first server 306, which is again connected to the first network 304. The data from the first server comes to the first user 302 through the first network 304. A second user 308 accesses the first server 306 through the first network 304 and accesses a second server 310 through the second network 312.

For example, when the first user 302 wants to access and update a communication thread at a location that is also accessible to the second user 308, then the first user 302 would point his/her user device that is capable of creating and/or accessing a communication thread virtually placed, at that location as an augmented reality scene. The user device also transmits a communication messaged added into the communication thread and location data over the first network 304 to the second server 310. The second server 310 stores the communication thread along with the location data. In this example considering the first network 304 is an office wifi. The second user 308 accesses the same communication thread through the second network 312 which is a cellular network.

FIG. 4A represents an example representation 400 of a communication thread created at a particular location in an augmented reality scene, in accordance with an embodiment of the present disclosure. The communication thread is any recorded message left by one user to at least one user that is placed within an augmented reality scene at the particular location. In particular, the communication thread is a group of linked communication messages. The communication thread can be in form of text, image, video, audio, a computer-readable content, or a plurality of computer readable instructions by an augmented reality (AR) device to provide human sensory output.

In this example, initially, a user 402 (“Sunil”) creates an augmented reality based communication message 410 corresponding to a particular location 414 (for example, in a client meeting room) using his user device 404. The user 402 (“Sunil”) also sets access permissions for the communication message to a user 406 (“Lekh”). The server 102 stores the communication message 410 along with location data of the particular location 414 in form of an association data in the communication thread database. When the user 406 (“Lekh”) is at or in proximity to the particular location 414, the server 102 transmits a signal to generate the augmented reality scene embedded with the communication message to a user device 408 of the user 406 (“Lekh”).

The user 406 (“Lekh”) also sends a communication message 412 to the user 402 (“Sunil”), thereby updating the communication thread. The augmented reality based communication thread can only be accessed by the user 402 “Sunil” or the user 406 “Lekh” at the particular location 414. At the same time, the server 102 also updates the association data by associating the updated communication thread with the particular location 414. Thus, the communication thread is placed at a location relevant for this communication as selected by at least one of a user device associated with the user “Sunil” or the user “Lekh”. In some cases, the user “Sunil” or the user “Lekh” manually selects a location for placing augmented reality based communication threads.

FIG. 4B is an exemplary user interface depicting a map view 420 of communication threads accessible to a user at multiple locations, in accordance with an embodiment of the present disclosure.

In one embodiment, the user (for example, user 108b) queries for accessible communication threads at nearby locations or multiple distant locations on his user device using the location-relevant communication enabled application 116. In one embodiment, the server 102 displays available one or more communication threads associated with a particular location to the user device of the user 108b when the user device accesses an augmented reality scene of the particular location through the location-relevant communication enabled application available in the user device. In one embodiment, the available one or more communication threads are shown in thumbnail form.

Based on the query, the user device displays the map view 420 including locations (such as, location 1 422....location n 426) which are associated with

accessible communication threads

424, and 428. In particular, the server 102 is configured to load back the accessible communication threads to the user 108a. In some embodiments, the user 108a queries for accessible communication threads for a particular location. In response, the user device 110a receives a list of accessible communication threads for the user 108a corresponding to the particular location from the server 102 and selects a specific communication thread from the list.

FIG. 5 represents an example representation of a communication thread 500 accessible to one or more authorized users, created at a particular location in an augmented reality scene, in accordance with an embodiment of the present disclosure. The communication thread includes a group of linked messages. The group of linked messages is at least one of: text, video, audio, animation, a plurality of computer readable instructions by a VR device to provide human sensory output and any combination thereof.

As described above, initially, the server 102 is configured to receive an augmented reality scene of a location from a user device associated with an authorized user (i.e., thread creating user) and a communication thread from the user device. The server 102 is configured to place the communication thread within the augmented reality scene and generate an association data. Further, the server 102 is configured to configure the association data to virtually place the augmented reality scene for another authorized user upon receipt of a request from the another authorized user for accessing the communication thread with the location.

In this example, the communication thread 500 can also be in the form of chats. The first chat message 502 is from user "Anil" and the second chat message 504 is from user "Sunil", and the third chat message 506 is from the user "Anil". Thus the first, second and third chat messages make the group of linked messages, thereby creating the communication thread 500. The communication thread 500 is placed virtually as an augmented reality at a relevant location. Alternatively the same communication thread 500 can also be placed at multiple locations. For example, the communication thread 500 can be placed at an office of user "Anil" as well as at an office of the user "Sunil". The access to the communication thread by at least one user is authenticated.

FIG. 6 illustrates a block diagram 600 of plurality of the communication threads in the form of virtually placed augmented reality scenes placed in the proximity of about the same location and where multiple users can access said communication threads, in accordance with an example embodiment of the present disclosure.

In one embodiment, there can be a plurality of communication threads among users at same location. As shown in the FIG. 6, there are two different communication threads T1 (see, 602) and T2 (see, 604), which are placed virtually as an augmented reality scene at a same location L1. The communication thread T1 includes a chain of communication messages m1, m2....mn which are accessible to a first user 606 and a second user 608. Exactly at the same location L1, the communication thread T2 (see, 604) includes a chain of communication messages n1, n2...nn which are accessible to a third user 610 and a fourth user 612. In one embodiment, the communication thread T2 may be accessible to the first user 606 when any one of the third user 610 and the fourth user 612 grants access permissions to the first user 606 for the communication thread T2.

FIG. 7 represents a flow chart 700 of a process flow of an operation of a user device with location identifying circuit, in accordance with an example embodiment of the present disclosure. The sequence of operations of the flow chart 700 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped together and performed in form of a single step, or one operation may have several sub-steps that may be performed in parallel or in sequential manner.

As mentioned previously, a communication thread is virtually placed by a thread creating user at a particular location in an augmented reality environment. The communication thread can be accessed by one or more authored users as per access setting performed by the thread creating user.

At 702, a user (such as, the user 108c) sends a request to the server 102 via his user device for accessing the communication thread associated with the particular location. The request includes, but is not limited to, location data associated with the user and user device identifier. In one case, the user may input or select the location manually on the user device.

At 704, the server 102 checks or authenticates access grants for the user 108c for accessing the communication thread on his/her user device 110b. In particular, the server 102 determines whether the user is at or in proximity to the particular location or not, based on the location data received from the user device 110b. Further, the server 102 verifies access permissions of the communication thread for the user 108c. Once the access is granted, the server 102 determines the communication thread associated with the particular location based on an association data stored in the communication thread database 104. In case, the access is not granted, the process flow stops at 712.

At 706, the user device 110b receives a signal for generating an augmented reality scene including the communication thread on the user device 110b. When the user 108c points his/her user device 110b to the particular location, the user can view the augmented reality scene with the communication thread. In one embodiment, the communication thread may be displayed as hanging in the air or simply in front of user’s field of view. The communication thread includes location relevant messages for the user 108c.

More illustratively, the data corresponding to the communication thread is loaded back from the server 102 to the user device 110b which is pointing to the particular location. Thereafter, the user device 110b displays the communication thread that is superimposed on the user's view (captured using a camera of the user device) on a display of the user device 110b. In other embodiment, the communication thread is superimposed in form of sound on speaker or human sensory stimulus based output or combinations thereof.

At 708, the user 108c appends a communication message to the communication thread using his/her user device 110b.

At 710, the user device 110b uploads the communication message appended in the communication thread to the server 102. Thus, the communication thread includes a group of linked messages. In one embodiment, the group of linked messages is created based on a Merkle tree of hash or at least one digest value for a communication message. The at least one digest value of the communication message is used to arrive at a hash value for the communication message which is itself combined with a hash value for a preceding communication message to generate the communication thread.

At 712, the server 102 updates the stored communication thread associated with the particular location in the association data.

FIG. 8 represents a flow chart 800 of a process flow of determining an operation of a user device without the location identifying circuit, in accordance with an example embodiment of the present disclosure. The sequence of operations of the flow chart 800 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped together and performed in form of a single step, or one operation may have several sub-steps that may be performed in parallel or in sequential manner. In the example embodiment, the user device is not enabled with the location identifying circuit.

Initially, the user 108b selects one or more locations manually on the location-relevant communication enabled application 116 installed on the user device. At 802, the user device sends a request to access the communication threads associated with the one or more locations to the server 102.

At 804, the access grants for the user 108b for accessing the communication threads on his/her user device are checked based on user profile of the user 108b. In particular, the server 102 extracts access permission information set by thread creating users for the communication threads of the one or more locations and identifies a set of locations with accessible communication threads for the user 108b from the one or more locations. In one embodiment, the server 102 displays available one or more communication threads associated with a particular location to the user device when the user device accesses an augmented reality scene of the particular location through a location-relevant communication enabled application 116 available in the user device.

At 806, the user device receives information of accessible communication threads for the set of locations of the one or more locations based on the step 804. The user device displays the accessible communication threads for the set of locations.

At 808, the user 108b selects a location from the set of locations where the communication thread needs updation. Based on the selected communication thread, the user device virtually places an augmented reality scene of the location with the communication thread on the display of the user device.

At 810, the user 108b updates the communication thread at the selected location and the user device uploads the updated communication thread to the server 102 which is configured to map the updated communication thread corresponding to the selected location.

FIG. 9 represents a flow chart 900 of a process flow of location relevant communication system, in accordance with an example embodiment of the present disclosure. The sequence of operations of the flow chart 900 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped together and performed in form of a single step, or one operation may have several sub-steps that may be performed in parallel or in sequential manner.

At 902, the server 102 receives a request for accessing a communication thread associated with a location from a user 108c. The request includes, but is not limited to, location data associated with the user and user device identifier. In one case, the user 108c may input or select the location manually on the user device 110b.

At 904, the server 102 checks access permissions of the communication thread for the user 108c. In case, at 906, if the access permissions are not granted for the user 108c, the server 102 notifies the user 108c indicating access denial for the communication thread.

At 908, when the access permissions are granted for the user (i.e., authorized user), the server 102 checks whether location access for the communication thread is enabled or not. In other words, the server 102 determines whether the communication thread associated with the particular location can be accessed from a different location or not.

When the location access for the communication thread is enabled, the authorized user 108c sends the location data corresponding to the particular location, where the authorized user is physically present at the particular location.

When the location access for the communication thread is not enabled, the user 108c may not need to be present at the particular location for accessing the communication thread and the user 108c may input the particular location on the user device 110b.

When the location access for the communication thread is not enabled, the server 102 directly grants access to the communication thread to the user 108c (process goes to 914). In other words, the server 102 sends one or more instruction sets to the user device 110b to generate an augmented reality scene of the particular location including the communication thread in the user device 110b when the location input is received from the user device of the authorized user 108c and/or an augmented reality scene of the particular location is received from a camera of the user device of the authorized user 108c through a location-relevant communication enabled application 116 available at the user device.

At 910, when the location access for the communication thread is enabled, the server 102 determines whether the user 108c is present at the particular location or not. If the user 108c is not present at the particular location, at 912, the server 102 notifies the user 108c indicating access denial for the communication thread.

At 914, when the user 108c is present at the particular location, the server 102 sends a signal to the user device 110b to generate an augmented reality scene of the particular location including the communication thread in the user device 110b while the user 108c points his/her user device to the particular location. More illustratively, the user 108c is transmitted with one or more instruction sets for generating the augmented reality scene with the communication thread.

FIG. 10 is a simplified block diagram of a server 1000, in accordance with an embodiment of the present disclosure. The server 1000 is an example of the server 102 of FIG. 1. The server 1000 includes a processing system configured to extract programming instructions from a memory to provide various features of the present disclosure. The components of the server 1000 provided herein may not be exhaustive and the server 1000 may include more or fewer components than those depicted in FIG. 10. Further, two or more components may be embodied in one single component, and/or one component may be configured using multiple sub-components to achieve the desired functionalities. Some components of the server 1000 may be configured using hardware elements, software elements, firmware elements, and/or a combination thereof.

The server 1000 may include a request handler 1002, a communication interface 1004, user profile and permission database 1006 and a communication thread database 1008, which are configured to communicatively coupled via a bus 1010.

Via the communication interface 1004, the request handler 1002 receives a request for accessing a communication thread associated with a location from a user device of a user (for example, user 108b). The request includes location data of the location and user device identifier from the user device of the user 108b.

The request handler 1002 processes the request and runs a query to determine a communication thread associated with the location in the communication thread database 1008. The communication thread database 1008 is configured to store one or more association data which is configured to map a set of locations to corresponding communication threads. The request handler 1002 also checks access permission of the communication thread for the user 108b by running a query to the user profile and permission database 1006. The user profile and permission database 1006 is configured to store user profile data of users for whom the access permissions are granted for one or more communication threads.

FIG. 11 is a simplified block diagram of a user device without location identifying circuit capable of implementing at least some embodiments of the present disclosure. For example, the user device 1100 may correspond to the user device 110a (without location identifying circuit) associated with the user 108a who creates or requests for location relevant communication threads in an augmented reality environment. The user device 1100 is depicted to include one or more applications (e.g., “location-relevant communication enabled application”). The applications can be an instance of an application downloaded from the server 102 or a third-party server. The location-relevant communication enabled application is managed by the server 102.

It should be understood that the user device 1100 as illustrated and hereinafter described is merely illustrative of one type of device and should not be taken to limit the scope of the embodiments. As such, it should be appreciated that at least some of the components described below in connection with the user device 1100 may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of the FIG. 11. As such, among other examples, the user device 1100 could be any of a mobile electronic device, for example, cellular phones, tablet computers, laptops, mobile computers, personal digital assistants (PDAs), mobile televisions, mobile digital assistants, or any combination of the aforementioned, and other types of communication or multimedia devices.

The illustrated user device 1100 includes a processor 1102 (e.g., a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, image processing, input/output processing, power control, and/or other functions.

The illustrated user device 1100 includes a memory 1104, for example, a non-removable memory and/or removable memory. The user device 1100 can support one or more input devices 1106 and one or more output devices 1108. Examples of the input devices 1106 may include, but are not limited to, a touch screen / a display screen (e.g., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone (e.g., capable of capturing voice input), a camera module (e.g., capable of capturing still picture images and/or video images) and a physical keyboard. Examples of the output devices may include, but are not limited to, a speaker and a display. Other possible output devices can include piezoelectric or other haptic output devices. Some devices can serve more than one input/output function. For example, the touch screen and the display can be combined into a single input/output device.

The disclosed methods with reference to FIGS. 1 to 11, or one or more operations of the

methods

700, 800 and 900 may be implemented using software including computer-executable instructions stored on one or more computer-readable media (e.g., non-transitory computer-readable media, such as one or more optical media discs, volatile memory components (e.g., DRAM or SRAM), or nonvolatile memory or storage components (e.g., hard drives or solid-state nonvolatile memory components, such as Flash memory components)) and executed on a computer (e.g., any suitable computer, such as a laptop computer, net book, Web book, tablet computing device, smart phone, or other mobile computing device). Such software may be executed, for example, on a single local computer or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a remote web-based server, a client-server network (such as a cloud computing network), or other such network) using one or more network computers. Additionally, any of the intermediate or final data created and used during implementation of the disclosed methods or systems may also be stored on one or more computer-readable media (e.g., non-transitory computer-readable media) and are considered to be within the scope of the disclosed technology. Furthermore, any of the software-based embodiments may be uploaded, downloaded, or remotely accessed through a suitable communication means. Such a suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.

Although the disclosure has been described with reference to specific exemplary embodiments, it is noted that various modifications and changes may be made to these embodiments without departing from the broad spirit and scope of the disclosure. For example, the various operations, blocks, etc. described herein may be enabled and operated using hardware circuitry (for example, complementary metal oxide semiconductor (CMOS) based logic circuitry), firmware, software and/or any combination of hardware, firmware, and/or software (for example, embodied in a machine-readable medium). For example, the apparatuses and methods may be embodied using transistors, logic gates, and electrical circuits (for example, application specific integrated circuit (ASIC) circuitry and/or in Digital Signal Processor (DSP) circuitry).

Particularly, the server 102 and its various components such as the computer system and the database may be enabled using software and/or using transistors, logic gates, and electrical circuits (for example, integrated circuit circuitry such as ASIC circuitry). Various embodiments of the disclosure may include one or more computer programs stored or otherwise embodied on a computer-readable medium, wherein the computer programs are configured to cause a processor or computer to perform one or more operations. A computer-readable medium storing, embodying, or encoded with a computer program, or similar language, may be embodied as a tangible data storage device storing one or more software programs that are configured to cause a processor or computer to perform one or more operations. Such operations may be, for example, any of the steps or operations described herein. In some embodiments, the computer programs may be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g., magneto-optical disks), CD-ROM (compact disc read only memory), CD-R (compact disc recordable), CD-R/W (compact disc rewritable), DVD (Digital Versatile Disc), BD (BLU-RAY® Disc), and semiconductor memories (such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash memory, RAM (random access memory), etc.). Additionally, a tangible data storage device may be embodied as one or more volatile memory devices, one or more non-volatile memory devices, and/or a combination of one or more volatile memory devices and non-volatile memory devices. In some embodiments, the computer programs may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g., electric wires, and optical fibers) or a wireless communication line.

Various embodiments of the invention, as discussed above, may be practiced with steps and/or operations in a different order, and/or with hardware elements in configurations, which are different than those which are disclosed. Therefore, although the invention has been described based upon these exemplary embodiments, it is noted that certain modifications, variations, and alternative constructions may be apparent and well within the spirit and scope of the invention.

Although various exemplary embodiments of the invention are described herein in a language specific to structural features and/or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as exemplary forms of implementing the claims.

Claims

A method for facilitating location relevant communication, the method comprising:
receiving, by a server, a request for accessing a communication thread associated with a location, the request comprising at least a location data of the location from a user device of a user;
determining, by the server, the communication thread associated with the location based, at least in part, on an association data stored in a communication thread database, the communication thread database storing one or more association data comprising mappings of a set of locations with corresponding communication threads; and
transmitting, by the server, one or more instruction sets to the user device to generate an augmented reality scene of the location comprising the communication thread in the user device.
The method as claimed in claim 1, wherein the association data is stored in the communication thread database, prior to receiving the request for accessing the communication thread, based at least on:
receiving, by the server, the location data from a thread creating user;
receiving, by the server, the communication thread created by the thread creating user, wherein a user device associated with the thread creating user is configured to virtually place the communication thread at the location in the augmented reality scene
mapping, by the server, the location data and the communication thread to form the association data; and
storing, by the server, the association data in the communication thread database.
The method as claimed in claim 2, wherein the thread creating user is different from the user sending the request for accessing the communication thread, wherein the communication thread is a group of linked messages, and wherein the method further comprises:
receiving, by the server, a communication message from the user device of the user in response to the group of linked messages in the communication thread, the communication message received via the augmented reality scene generated via the user device; and
updating, by the server, the communication thread stored in the communication thread database in response to receipt of the communication message from the user device.
The method as claimed in claim 2, wherein the user is same as the thread creating user, wherein the communication thread is a group of linked messages, and wherein the method further comprises:
receiving, by the server, an input from the user device of the user to update the communication thread, the input received via the augmented reality scene generated at the user device; and
updating, by the server, the communication thread stored in the communication thread database in response to receipt of the input from the user device.
The method as claimed in claim 4, further comprising:
receiving, by the server, an updated communication thread from the user device; and
updating, by the server, the association data by associating the updated communication thread with the location.
The method as claimed in claim 5, further comprising:
transmitting, by the server, a notification to at least one or more authorized users and the thread creating user associated with the communication thread for indicating a change in the association data upon updating the association data.
The method as claimed in claim 2, wherein receiving the location data is performed based at least on:
a manual selection of the location in the user device;
identification of the location by a location identifying circuit of the user device; or
selection of the location with a pre-configuration in the user device.
The method as claimed in claim 2, wherein the communication thread is a group of linked messages, comprising one or more of:
text;
image;
video;
audio;
a computer-readable content; or
a plurality of computer readable instructions by an augmented reality (AR) device to provide human sensory output.
The method as claimed in claim 1, further comprising electronically enabling the user to perform:
appending the communication thread;
deleting the communication thread;
modifying the communication thread;
retrieving the communication thread;
editing the communication thread;
modifying the location associated with the communication thread;
replicating the communication thread at an another location;
merging a plurality of the communication threads at at least one of the locations; or
modifying an access of the communication thread.
The method as claimed in claim 1, further comprising displaying, by the server, available one or more communication threads associated with a particular location to the user device when the user device accesses an augmented reality scene of the particular location through a location-relevant communication enabled application available in the user device.
A server for facilitating location relevant communications, the server comprising:
a communication interface;
a communication thread database;
a memory comprising executable instructions; and
a processor communicably coupled to the communication interface, the processor configured to execute the executable instructions to cause the server to at least perform:
receive a request for accessing a communication thread associated with a location, the request comprising at least a location data of the location from a user device of a user,
determine the communication thread associated with the location based, at least in part, on an association data stored in the communication thread database, the communication thread database storing one or more association data comprising mappings of a set of locations with corresponding communication threads, and
transmit one or more instruction sets to the user device to generate an augmented reality scene of the location comprising the communication thread in the user device.
The server as claimed in claim 11, wherein the association data is stored in the communication thread database, prior to receiving the request for accessing the communication thread, based at least on:
receive the location data from a thread creating user;
receive the communication thread created by the thread creating user, wherein a user device associated with the thread creating user is configured to virtually place the communication thread at the particular location in the augmented reality scene;
map the location data and the communication thread to form the association data; and
store the association data in the communication thread database.
The server as claimed in claim 12, wherein the thread creating user is different from the user sending the request for accessing the communication thread, wherein the communication thread is a group of linked messages, and wherein the server is further caused to:
receive a communication message from the user device of the user in response to the group of linked messages in the communication thread, the communication message received via the augmented reality scene generated via the user device; and
update the communication thread stored in the communication thread database in response to receipt of the communication message from the user device.
The server as claimed in claim 12, wherein the user is same as the thread creating user, wherein the communication thread is a group of linked messages, and wherein the server is further caused to:
receive an input from the user device of the user to update the communication thread, the input received via the augmented reality scene generated via the user device; and
update the communication thread stored in the communication thread database in response to receipt of the input from the user device.
The server as claimed in claim 12, wherein the communication thread is a group of linked messages, comprising one or more of:
text;
image;
video;
audio;
a computer-readable content; or
a plurality of computer readable instructions by an augmented reality (AR) device to provide human sensory output.
A method comprising:
receiving, by a server, an augmented reality scene of a location from a user device;
receiving, by the server, location data of the location from the user device;
receiving, by the server, a communication thread from the user device;
placing, by the server, the communication thread within the augmented reality scene to form an association data comprising mapping of the communication thread and the location; and
storing, by the server, the association data in a communication thread database for providing access to the association data to one or more authorized users of the communication thread upon receipt of a request for accessing the communication thread associated with the location.
The method as claimed in claim 16, further comprising:
configuring, by the server, the association data to virtually place the augmented reality scene for an authorized user of the one or more authorized users upon receipt of the request from the authorized user for accessing the communication thread associated with the location; and
transmitting one or more instruction sets comprising the configured association data to the authorized user.
The method as claimed in claim 17, wherein the authorized user is transmitted with the one or more instruction sets when the server receives a location data corresponding to the location from a user device of the authorized user when the authorized user is physically present at the location.
The method as claimed in claim 17, wherein the authorized user is transmitted with the one or more instruction sets when:
a location input is received from a user device of the authorized user; or
an augmented reality scene of the location is received from a camera of the user device of the authorized user through a location-relevant communication enabled application available at the user device.
The method as claimed in claim 16, wherein the communication thread is a group of linked messages, comprising one or more of:
text;
image;
video;
audio;
a computer readable content; or
a plurality of computer readable instructions by an augmented reality (AR) device to provide human sensory output.