EP3491589A1 - Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users - Google Patents
Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of usersInfo
- Publication number
- EP3491589A1 EP3491589A1 EP17749318.6A EP17749318A EP3491589A1 EP 3491589 A1 EP3491589 A1 EP 3491589A1 EP 17749318 A EP17749318 A EP 17749318A EP 3491589 A1 EP3491589 A1 EP 3491589A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- user
- realm
- computing device
- server
- location
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
Definitions
- the present invention relates to providing individual and/or synchronized virtual tours through locations within a realm for a group of users within a networked environment, and more particularly, to allowing users to either select their own locations or rely on other users to select locations for experiencing the realm.
- the house hunter may view the images on his or her own.
- other parties such as real estate agents, cannot share the house hunter's experience of the realm and cannot control what the house hunter views. In this manner, each individual may have a unique, personal experience of the realm that is not readily communicated with others.
- a method coordinates experiences of a realm for a group of users, including at least an administrative user and at least one participating user.
- the method includes transmitting, from the server to an administrative user, a session identifier associated with the realm.
- the method also includes receiving, by the server, a user acceptance to join a session using the session identifier so that the at least one participating user can experience the realm.
- the method also includes transmitting to a computing device of the participating user, by the server, executable code including data describing a series of locations in the realm.
- the series of locations comprises at least a first location and a second location, and the first location is associated with a first panoramic image and the second location is associated with a second panoramic image.
- the executable code is configured to receive instructions from at least one computing device of at least one user within the group to display the realm at at least one selected location, and display, on the computing device, data corresponding to the realm at at least one of the locations selected by at least one of the users within the group.
- the executable code is further configured to receive instructions from the server regarding locations selected by a user within the group.
- the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user.
- the request may include a location in the realm selected by the administrative user.
- the method may include transmitting, by the server, the location in the realm selected by the administrative user to a computing device of the participating user.
- the location causes the executable code running on the computing device of the participating user to display data corresponding to the location selected by the administrative user.
- the method may include transmitting, by the server to a computing device of the participating user, locations in the realm selected by the users of the group. Additionally, the method may include receiving, by the server from the administrative user, a request to synchronize an experience of the realm for the participating user with an experience of the realm for the administrative user. In response to the request from the administrative user, the method may include transmitting, by the server to the computing device of the participating user, an instruction to display data corresponding to the locations in the realm selected by the administrative user.
- the computing device of the participating user may create at least one virtual image representing a view of the realm between a location selected by the participating user and the location selected by the administrative user. Also, the computing device may display data corresponding to the virtual image and the location selected by the administrative user to simulate motion in the realm from the location selected by the participating user to the location selected by the administrative user.
- the method may include receiving, by the server from the computing device of the administrative user, an updated location selected by the administrative user.
- the method may also include transmitting, by the server to the computing device of the participating user, the updated selected location so that the computing device of the participating user displays data corresponding to the updated selected location in the realm.
- the instruction to synchronize the experiences of the participating user and the administrative user causes the computing device of the participating user to disregard instructions from the participating user.
- the method may include receiving, by the server from a first user in the group, a request to synchronize an experience of the realm for a second user in the group with an experience of the realm for the first user.
- the selected location in the realm includes a view direction.
- the method may include receiving, by the server from an administrative user, a request to view a list of users who have joined the experience of the realm. Also, the method may include transmitting, by the server to a computing device of the administrative user, the list of users for display.
- the method includes receiving, by the server from the administrative user, a request to view a history of selected locations in the realm for a user in the group of users.
- the method may include transmitting, by the server to a computing device of the administrative user, the history of selected locations in the realm for the user.
- the method may include receiving, by the server from the administrative user, a request to follow the view of the participating user.
- the method may also include sending, by the server to the computing device of the participating user, an instruction to provide, on a continual basis, updates on the location in the realm selected by the participating user.
- the method may further include forwarding, by the server to the computing device of the administrative user, the updated location selected by the participating user.
- the updated location causes the executable code running on the computing device of the administrative user to display data corresponding to the realm at the updated selected location of the participating user.
- the method also includes transmitting, by the server to a computing device of the administrative user, locations in the realm selected by the users of the group.
- the executable code on the computing device of the administrative user may be configured to receive a request to follow the view of the participating user and display data corresponding to the locations in the realm received from the server that are selected by the participating user.
- the method may include receiving, by the server from an administrative user, an instruction to display a second realm to the group of users. Additionally, the method may include transmitting, by the server to each computing device for the group of users, executable code including data describing a series of locations in the second realm.
- Fig. 1 depicts a block diagram representing a tour platform including at least one server that provides individual and synchronized experiences of a realm for a group of users;
- FIG. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm
- Fig. 2B is an exemplary flow diagram depicting the initiation of the session of Fig. 2A, in which users bypass the server to update one another regarding their locations in the realm;
- FIG. 3 depicts exemplary relationships between images in a set of images for a realm
- Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user;
- Fig. 4B is an exemplary flow diagram depicting one user bypassing the server to directly command and control the experience of the realm for another user;
- Fig. 5 is an exemplary flow diagram depicting the manner in which the image data for one user is updated, based on the location of another user;
- FIG. 6 A is an exemplary flow diagram depicting a user's selection of a new realm for a session, and the propagation of that realm to the other users in the group;
- Fig. 6B is an exemplary flow diagram depicting the initiation of the session of Fig. 6A, in which users bypass the server to update one another regarding their locations in the realm;
- Fig. 7 is an exemplary flow diagram for creating a session history in real time.
- Fig. 8 depicts a platform for an exemplary apparatus that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images;
- Fig. 9 depicts an isometric view of a receptacle for the mobile computing device when mounted on a rotating member of a platform;
- Fig. 10 depicts exemplary elements of the platform of Fig. 8;
- Figs. 11-13 depict side, front, and top views of the receptacle and rotating member, illustrating the configuration of the receptacle to offset the mobile computing device from the axis of rotation of the member;
- Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform of the apparatus of Fig. 8.
- a "realm" means a three-dimensional space. Parts of a realm need not be contiguous - realms may include discontinuities. While a realm may represent an actual physical space, any part of a realm may be imaginary.
- a "set” includes at least one member.
- FIGS. 1-7 the following section describes various embodiments relating to Systems and Methods for Enabling Shared Experiences of a Realm for Users in a Group. Reference numerals recited in this section relate to FIGs. 1-7. The fact that the same reference numeral is used in the section below relating to an
- the inventors have developed systems and methods that enable users in a group to experience the same realm concurrently, and also to synchronize, either by sharing or commanding and controlling, their experiences of the realm.
- the systems include a tour platform, with at least one server.
- the tour platform stores sets of images, each set corresponding to a different realm.
- a user selects a realm and creates a session for experiencing the realm.
- the tour platform transmits code, which includes the set of images for the realm, to the computing devices of the users.
- the devices store and execute the code, thereby enable the user to interact with the images to experience the realm.
- Local execution of the code improves the speed, performance, and overall quality of the user's experiences. Moreover, this local execution enables the tour platform to synchronize experiences of the realm for two or more users efficiently simply by transmitting the locations of interest within the realm to the computing devices of the users, instead of the full images corresponding to the locations.
- the users' computing devices may process the locations in conjunction with the set of images for the realm to identify the view (e.g., image data) to display to the user.
- the systems and methods described herein decrease the likelihood of disruption in a user's experience, due to errors in image data transmission or lapses in connectivity over the network.
- Fig. 1 depicts a block diagram representing a tour platform 100 including at least one server 105 that provides individual and synchronized experiences of a realm for a group of users via each user's respective computing device 110. Although operations described herein are discussed with respect to the server 105, any given operation may be performed by one or more servers in a group of servers.
- the server 105 connects to the user's computing devices 110 over a wide area network 115.
- the server 105 stores sets of images, either on the server 105 itself or on another entity (e.g., another server, a database 106), and each set corresponding to a different realm.
- the server 105 also stores code that, when executed on a computing device 110, enables a user to interact with a set of images.
- users may transmit sets of images corresponding to realms to the server 105 for storage (for the sake of simplicity, user activity discussed herein will be described in reference to the user initiating the activity, even though the user's computing device may perform the activity itself).
- the set of images may include information such as the relationships between the images and the criteria for switching from one image to another.
- any given image in the set may have embedded objects, which the user may interact with.
- Fig. 2A is an exemplary flow diagram depicting the initiation of a session in which a group of users experience the same realm.
- a user may send a request to create a session to the server 105 (step 201).
- the request identifies at least one particular realm to associate with the session, and as the session creator, the user has administrative privileges, described in further detail below (session creators are also referred to herein as
- the server 105 creates the session (step 205), and associates the administrative user and the realm(s) with the session.
- the administrative user sends invitations to at least one other user to join the session (step 210).
- the invitations include an identifier for the session, such as an
- the administrative user sends the invitations to the server 105 (step 215), and the server forwards the invitations to the users (step 220).
- the administrative user may send the invitations directly to the computing devices of the users (step 220).
- a user may accept an invitation to join a session (step 225) by, for example, accessing the URL in an invitation, which directs the computing device 110 of the user to the server 105.
- the server 105 adds the user to the session (step 230).
- the server 105 transmits the set(s) of images
- the computing device 110 stores the set of images and executable code.
- each image in the set of images is a panoramic image.
- the computing device 110 displays image data from one panoramic image at a time. To provide a more natural experience for the user, the computing device 110 displays only a subset of the panoramic image, which corresponds to the user's view of the represented scene. Thus, any given subset of the panoramic image may be associated with a unique view direction.
- the computing device 110 may initially display the panoramic image from a default view direction. For example, each image in the set may be stored in association with a default view direction.
- the computing device 110 identifies a subset of the panoramic image to display, based on the default view direction. Thereafter, the user is allowed to change the view direction. For example, the user may drag the panoramic image to pan to the left or right, or tilt upwards or downwards. In response to the user control, the computing device 110 updates the view direction and the corresponding subset of the panoramic image to display. [0045] Furthermore, the images in the set may be interrelated in a predetermined manner. Examples of such relationships are depicted in Fig. 3. The relationships constrain the sequence in which the user may view the images for a realm. For example, when the computing device 110 loads the set of images depicted in Fig. 3, the computing device 110 may display the image labeled "1" first.
- the user may only opt for image "2".
- the computing device 110 displays image "2"
- the user is permitted to switch solely to either image "1" or "3". Should the user traverse the images to select image "7”, the user would be permitted to select image "6", "8", or "10" to view.
- the user may not be permitted to change images unless the view direction for the displayed image encompasses a pathway between the locations in the realm represented by the displayed image and another image. For example, suppose images "1" and "2" represented adjacent rooms separated by a doorway. When the user is exploring the scene in image "1", to provide a natural experience of the realm, the user is not be permitted to select image "2" unless the view direction for image "1" encompasses the door separating the rooms. In this manner, the user would not be permitted to select image "2” unless the view direction falls within a predetermined range of view directions, during which the door would be visible. In some embodiments, the computing device 100 would not display an icon, such as an arrow, for changing images until the view direction for the displayed image meets this criterion.
- an icon such as an arrow
- the computing device 110 When the user sends the computing device 110 an instruction to display another image, the computing device 110 creates and displays transitional images to simulate motion between the locations represented by the two images in the set.
- the server 105 may create the transitional images and send them to the computing device 110 to display. Exemplary systems and methods for creating transitional images are described in Application No. 11/271, 159, which is incorporated by reference in its entirety.
- the transitional images may represent the pathway between the view direction for the prior image, and a default view direction for the selected image, which may depend on the identity of the prior image. For example, an image may have multiple default view directions, and referring to Fig.
- image “2" may have one default view direction if a user has selected the image from image “1” and another default view direction if a user has selected the image from image "3". If a user is viewing image “1” and selects image “2", the computing device 110 uses the current view direction for image “1” and the default view direction for image “2” when the user is transitioning from image “1", and creates transition images to simulate motion between the two images.
- the images in the set are not interrelated. Instead, the user may switch from one image in the set to any other image. For example, the user may invoke a control for changing images, and in response, the computing device 110 may display thumbnails representing the images in the set. When the user selects a thumbnail, the computing device 110 may display the corresponding image. In these embodiments, each image may include a default view direction, and the device 110 loads the image data for the default view direction. Because the images are not interrelated, the computing device 110 may switch from one image to another, instead of creating transitional images. Alternatively, the computing device 110 may transition between images by, for example, morphing one image into another. In some embodiments, a user may configure settings provided by the code to select the manner in which the computing device 110 displays images sequentially.
- the computing device 110 sends updates about the state of the user's location to the server 105 (step 240).
- the computing device 110 may update the server 105 regarding the image on display and the view direction therein.
- Each communication may include an identifier for the session, an identifier for the user (e.g., the user's computing device 110), an identifier for the image in the set that is being viewed, a yaw angle for the view direction, a pitch angle for the view direction, or any combination thereof.
- the server 105 receives updated locations in the realm for each user in the session, and the server 105 may store the updates for each user.
- the computing device 110 may send updates directly to another user and/or all of the other users in the group.
- the computing device 110 sends updates about the state of the user's location directly to the other users in the group (step 240'), thereby bypassing the server 105.
- the server 105 transmits each update from each user to all of the other users in the group (step 245).
- the server 105 may send updates solely to the administrative user. When the administrative user receives this information, his or her computing device 110 may store the information, thereby saving a history of the users' locations for the session (step 250).
- the tour platform 100 allows users to synchronize other users' experiences of the realm.
- any given user also referred to herein as the
- commanding user may require another user (also referred to herein as the "following user") to follow his or her experience of the realm.
- the computing device 110 of the following user displays the realm according to the location of the commanding user, and in some embodiments, the computing device 110 may become unresponsive to instructions from the following user to select an image or view direction in the realm.
- a user may elect to follow the experience of the realm for another user. Thus, the user voluntarily becomes a "following user", but may opt to stop following at any time.
- Fig. 4A is an exemplary flow diagram depicting one user commanding and controlling the experience of the realm for another user (also referred to herein as
- the commanding user may invoke the option to command and control other users.
- the computing device 110 may display an icon for commanding and controlling other users' experience, and in response to user selection of this icon, the computing device 1 10 may display the users in the group for the session.
- the commanding user may select individuals to be following users (step 405), and transmit an instruction to command and control the experiences of the selected users to the server 105 (step 410).
- the administrative user may be allowed to command and control all of the other users in the group, but a non-administrative user may be permitted to command and control a more limited number of users.
- the server 105 sends instructions to the selected users to follow the commanding user (step 415). In Fig.
- the computing device 110 transmits instructions to command and control directly to the computing devices 110 of the selected users (step 410'), thereby bypassing the server 105.
- the computing device 110 of a following user may confirm receipt of the instruction to the server 105 (step 420).
- the computing device 110 may display a notification to the following user that he or she is now following the commanding user, and may also begin disregarding any commands that the following user attempts to input (e.g., selection of an image to view, selection of a view direction).
- the computing device 110 of a following user begins displaying the realm according to the commanding user's experience (step 425).
- the instruction from the server 105 includes a location of the commanding user.
- the computing device 110 obtains the location of the commanding user in the course of the commanding user's broadcast of his or her updated location.
- the server 105 may transmit each update from each user to all of the other users in the group (step 245).
- the user's computing device 110 may ignore these updates, or store the updates to create a local copy of the session history.
- his or her computing device 110 may begin using the updated information associated with the commanding user to determine the image data in the set of images to display.
- a user When a user becomes a following user, the computing device 110 may jump to the location of the commanding user directly. Alternatively, the computing device 110 may create transitional images to simulate motion between the following and commanding users' locations.
- Fig. 5 is an exemplary flow diagram depicting the manner in which a following user's computing device 110 updates the image data that it displays, based on the location of the commanding user. The computing device 110 may execute code to perform these steps when it first begins following the commanding user, and any time the
- commanding user changes his or her location thereafter.
- the computing device 110 of the following user may receive an update to the commanding user's location (step 505) from the server 105.
- the update may identify an image in the set and a view direction.
- the computing device 110 compares the image in the update with the image that it is currently displaying. If the images are the same (step 510), using the view direction associated with the image data on display and the newly received view direction, the computing device 110 creates and displays transitional images to simulate motion from the current view direction to the new one (step 515).
- the computing device 110 displays the image data for the newly received view direction until the device 110 receives another update (step 520).
- the computing device 110 retrieves the new image from memory, as well as a default view direction (step 525). If the image has multiple default view directions, the computing device 110 may select the default that is closest to the view direction in the update and display image data associated with this default. The computing device 110 creates and displays transitional images to simulate motion from the default view direction to the view direction in the update (step 530). The computing device 110 displays the image data for the updated view direction until the device 110 receives a further update (step 520).
- the computing device 110 of a following user may continue performing the steps in Fig. 5 until the commanding user releases control. While the commanding user compels other users to following his or her experience of the realm, his or her computing device 110 displays an icon for releasing control of other users. If the commanding user selects this icon, the computing device 110 sends an instruction to the server 105 to terminate the control over the other users, and the server 105 forwards the commands to the computing devices 110 of those users.
- a computing device 110 may display a notification to its user that the commanding user has terminated its control.
- the computing device 110 disregards location updates from the commanding user, and becomes receptive to instructions from the user, again.
- the computing device 110 remains at the most recent location selected by the commanding user, and allows the following user to navigate from there.
- the computing device 110 may have stored the most recently selected location of the following user prior to the instruction to follow the commanding user, and after the commanding user releases control, the computing device 110 restores the selected location of the user.
- a user may command and control other users' experiences of the realm, in some embodiments, the user may simply share his or her experience, while allowing the other users to retain control.
- the sharing user may perform similar steps to those described in Fig. 4, but with instructions to share the user's experience with the targeted users.
- the computing devices 110 of the recipients reserve an area of the display for depicting the experience of the sharing user (e.g., a "picture-in-picture" display), and the devices 110 execute code both for updating the image data displayed in this reserved area, and updating the image data for locations selected by the users.
- the devices 110 may downsample the images in the set to determine the image data to display for the experience of the sharing user, but process the images received from the server 105 to determine the image data to display for the user of the computing device 110.
- a user may opt to follow another user.
- the computing device 110 may include a control for following a user, and when this control is activated, the device 110 displays a list of users to follow. By selecting a user, the computing device 110 begins displaying image data corresponding to the followed user's location, whether the device 110 receives the location as part of the followed user's broadcast or sends a request to the server 105 to receive the location of the followed user whenever this location is updated.
- the user can opt to either replace the image data on its computing device 110 with the experience of the followed user, or display the experience on a reserved area, thereby viewing different locations in the realm simultaneously. While the user is following another user, the computing device 110 displays an "unfollow" control, whose selection reverts the display data to that chosen by the user.
- the administrative user may wish to select a different realm for the session.
- the administrative user may be a real estate agent, and upon completing a tour of a property with a client, the real estate agent may wish to show another property.
- the client may express disinterest with a property, and the parties may select another property to determine if that property will fit the client's needs more appropriately.
- Fig. 6A is an exemplary flow diagram depicting the administrative user's selection of a new realm for a session, and the propagation of that realm to the other users in the group.
- the administrative user activates an icon on the computing device 110 to change realms.
- the computing device 110 displays representations of realms for the session, such as thumbnails of the first images for each of the realms.
- the administrative user selects one of the realms, and the computing device 110 loads the set of images associated with the selected realm (step 610).
- the computing device 110 displays the first image in this set, based on the default view position for the image and/or beginning of a tour of the new realm.
- the computing device 110 sends an update to the server 105 regarding the new realm for the session (step 615).
- the server 105 sends an update to the other users in the group regarding the new realm for the session (step 620).
- the computing device 110 send the update directly to the other users in the group (step 615'), thereby bypassing the server 105.
- their computing devices 110 process the update (step 625), retrieve the set of images for the new realm from memory, and display the first image in this set (step 630).
- the users may be informed of realms for the session in real-time, and obtain new sets of images from the server 105 whenever the administrative user selects a new realm for the experience.
- the code may identify the realms via alphanumeric identifiers, by way of example, but may not contain their sets of images (nevertheless, the code may include the set for the first realm of the session).
- the computing device 110 may send a request with the alphanumeric identifier to the server 105, and in response, the server 105 may return the images. In this manner, the computing devices 110 do not store sets of images for realms until such sets are needed.
- the server 105 may store a user history of the session.
- Fig. 7 is an exemplary flow diagram for creating a session history in real time. The steps are similar to those described in Fig. 4. However, each time a user sends an updated location to the server 105, the server 105 records the change (step 720), and may also store the time that the update was received. In this manner, the server 105 creates a log of every location that each user visited, and the length of time that the user remained at that location. Thus, the server 105 may preserve the history of the session.
- the administrative user may run analytics on the session history to ascertain the other users' interests. For example, the administrative user may instruct the server 105 to determine the percentage of time that the users spent in each image, and well as particular sections of each image. If a realm corresponds to a house, analytics may reveal the percentage of time that a user spent in the kitchen, master bedroom, bathroom, and living room. Analytics may also determine that, while the user was in the kitchen, the user spent part of the time examining the appliances, but a significant amount of time enjoying the view into the backyard. Similarly, while the user remained in the basement, the user may have spent little time looking at the heating system, but a significant amount of time looking at the foundation. In this manner, the administrative user may discern items that interest or concern a user based on their location, and select other houses (and realms) accordingly.
- the inventors have developed an apparatus and methods that enable anyone with a mobile computing device (e.g., a smartphone) to obtain high quality panoramic images.
- the apparatus is configured to house a mobile computing device, and the apparatus further leverages features of existing panoramic image capturing software (also referred to herein as "panoramic software") on the device to ensure that the device captures excellent panoramic images.
- panoramic software also referred to herein as "panoramic software”
- the apparatus uses the panoramic software's reliance on the device's accelerometer(s) to determine whether the device is in motion.
- the apparatus may ensure that the mobile computing device captures the images needed to create a panoramic image, but only when the apparatus has halted motion at the designated positions.
- FIG. 8 Elements of an exemplary apparatus 100 that leverages features of panoramic image capturing programs on mobile computing devices to facilitate the capture of panoramic images are depicted in Figs. 8 and 9.
- the apparatus 100 includes a receptacle 110 that is configured to house a mobile computing device (not shown).
- a user may mount the receptacle 110 on a platform 120 by coupling the receptacle 110 to its member 125.
- the platform may include a motor 130 that is coupled to the member 125, as depicted in Fig. 10.
- the motor 130 and member 125 are coupled by a drivetrain, collectively represented by the gearbox 126 and drive gear(s) 127, though other
- the motor 130 By driving the member to different positions, the motor 130 consequently drives the mobile computing device in the receptacle 110 to different positions, as well.
- a user inserts his or her mobile computing device in the receptacle 110 and mounts the receptacle 110 on the platform 120 (alternatively, the receptacle 110 may be permanently affixed to the platform 120, and the user may merely insert the device therein). Then, the user initiates the panoramic software of the mobile computing device 190 and activates the motor 130 to begin driving the member 125.
- the user activates the motor by depressing a control, such as a button or any other physically tactile component (not shown).
- the control may activate the motor 130 by detecting that the mobile computing device has been coupled to the platform 120.
- control may be a sensor, or sensor system, that detects the presence of a mounted receptacle 110, a mobile computing device within the receptacle 110, or both, and which activates the motor 130 in response to such detection.
- the sensor may monitor the weight applied to the member 125 and activate the motor 130 when the weight exceeds a threshold.
- the motor 130 drives the member 125, and consequently, the mobile computing device, to move to different positions, and the motor 130 also halts motion at the positions for predefined periods of time.
- the motor's 125 activities are coordinated with the panoramic software of the mobile computing device.
- the motor 130 may drive the member 125 to move at a velocity that is sufficiently high so as to be detectable by the mobile computing device 190. Because the device 190 determines that it is in motion, the device 190 refrains from capturing images for the panoramic image. Once the motor 130 halts, the motor 130 remains stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device. In response to this stillness for a threshold period of time, the mobile computing device prepares its camera (e.g., focusing, color balancing, etc.) according to the scene in view, and captures an image.
- its camera e.g., focusing, color balancing, etc.
- the motor 130 drives the member 125 on a continual basis.
- the configuration of the drivetrain (e.g., gearbox 126 and drive gears 127) and the resultant movement of the member 125 is coordinated with the panoramic software of the mobile computing device, in lieu of the motor 130.
- the drivetrain moves the member 125 to different positions, and also halts motion at the positions for predefined periods of time.
- the mobile computing device 190 recognizes that it has remained stationary for a sufficiently long period of time to trigger the panoramic software of the mobile computing device 190, the mobile computing device 190 prepares its camera according to the scene in view, and captures an image.
- the apparatus 100 provides a period of stillness that is coordinated with the period of time that mobile computing device uses to recognize its physically still state and to capture an image.
- the apparatus 100 resumes driving the member 125 and mobile computing device to its next image capture position.
- the apparatus 100 and mobile computing device do not communicate with one another regarding their own operations. At no point does the mobile computing device recognize that it has been coupled to an apparatus 100. Moreover, even if a mobile computing device has not been mounted on the apparatus 100, if a user activates the motor 130, the motor 130 and/or drivetrain stills drive the member 125 through its sequence of positions, halting motion therebetween. Thus, the apparatus 100 and mobile computing device operate independently of one another, even though the apparatus 100 tailors its parameters to the features of panoramic software on the mobile computing device.
- the motor 130 and/or drivetrain may drive the member 125 angularly or linearly.
- the member 125 may be a turntable that rotates around an axis 128.
- the motor 130 and/or drivetrain may alternate between driving the member 125 to rotate by a particular number of degrees and halting the member 125 for a predefined period of time. Because the apparatus 100 facilitates capture of a panoramic image, the apparatus 100 may rotate the mobile computing device by a total of 360 degrees, throughout the sequence of its motions.
- the apparatus 100 may move the member 125 and mobile computing device to ten (10) positions for capturing images for a panoramic image.
- the motor 130 and/or drivetrain drives the member 125 to rotate thirty-six (36) degrees before halting for a predefined period of time.
- the motor 130 and/or drivetrain drives the member 125 to rotate by another thirty-six (36) degrees before halting.
- the motor 130 and/or drivetrain repeats this sequence until the member 125 has been rotated a full three hundred and sixty (360) degrees.
- the motor 130 and/or drivetrain may halt motion altogether, and the panoramic software of the mobile computing device may recognize that the camera can no longer be expected to move, and the device begins aggregating the captured images into a single panoramic image.
- this example describes angular increments of thirty-six (36) degrees for a total of 10 positions for capturing images, any other angular increment, and corresponding number of positions, may be used.
- the member 125 may be a rotary table. Furthermore, the receptacle 110 and member 125 may be configured such that, when a mobile computing device 190 is inserted into the receptacle 110, the device 190 is offset radially from the axis of rotation 128 of the member 125 by a predefined distance, so as to align the entrance pupil 192 of the lens system 195 with the axis 128.
- the motor 130 and/or drivetrain drives the member 125 to rotate around its axis 128, the lens system 195 rotates around the axis 128 while the device 190 rotates around the axis 128 at an offset.
- the offset ensures that the mobile computing device 190 captures images suited for panoramic images, and the offset may be selected to optimize the quality of the images for this purposes. In some embodiments, the offset may be less than about 0.5 inches.
- Figs. 11 and 12 depict two exemplary views of a receptacle 110 mounted on a member 125 that illustrates a radial offset between the device 190 and the axis of rotation 128 of the member 125, which ensures alignment between the axis 128 and the lens system 195.
- Fig. 13 depicts an overhead view of the receptacle 110 demonstrating that the imaging system of a mobile computing device would align with the entrance pupil 192 of the receptacle 110.
- the member 125 may progress along a linear path predefined by a groove in the platform.
- the motor 130 and/or drivetrain drives the member 125 to move down the path by a particular distance, and halts the member 125 for a predetermined period of time before driving the member 125 further down the path.
- the member 125 may move at a constant velocity, although in some embodiments, the motor 130 and/or drivetrain may vary the velocity of the member 125.
- the member 125 may be coupled to the motor 130 via a set of gears.
- Figs. 14-17 depict top, side, bottom, and isometric views of an exemplary set of gears that may be used in the platform 120 of the apparatus 100.
- the motor 130 drives a gearbox 805 to rotate a first gear 810.
- the teeth of the first gear 810 are engaged with the teeth of a second gear 815, which in turn are engaged with the teeth of a third gear 820 attached to the member 125 (in this case, a rotary table).
- a third gear 820 attached to the member 125 (in this case, a rotary table).
- rotation of the first gear 810 via the motor 130 drives the rotation of the rotary table 125.
- the configuration of teeth for at least one of the gears may determine the positions that the member 125 and mobile computing device will assume.
- one of the gears may have teeth of different length.
- motion of the member 125 begins, and motion halts when the shorter teeth of the gear fail to engage with the other gear in the set.
- the position where the member 125 halts depends on the position of the shorter teeth on the gear, as well as the gear's relationship to others in the set (e.g., relative size).
- the amount of time that the member 125 remains stationary may depend on the form factor of the engaged teeth.
- the positions where the member 125 halts motion, as well as the duration of those periods, can be controlled by selecting the size of the gear, the angular frequency of the larger teeth, and the form factors of the larger teeth, among other parameters.
- activating the motor 130 may cause the motor 130 to drive the set of gears for a predetermined period of time, but the configuration of the gears, both individually and as a set, define the positions that the member 125 and mobile computing device attain, and the length of time they remain in such positions.
- the platform 120 may include a processor and a memory for storing computer instructions.
- the processor may load and execute computer instructions from the memory, and these instructions may control the manner in which the motor 130 drives the member 125. For example, based on the instructions, the processor may alternate between driving the member 125 at a particular velocity for a period of time, and halting all motion. Because the parameters for motion and stillness are encoded in the computer instructions, these embodiments require simpler sets of gears than the embodiments that rely exclusively on the set of gears to determine positions for the member 125 and the length(s) of time for halting motion at the positions.
- the lens system 192 When the receptacle 110 has a lens system 192, the lens system 192 must be positioned so that the lens of an inserted mobile computing device 190 aligns with the lens system 192 on the receptacle 110. As shown in Fig. 13, the receptacle 110 may be configured to position a mobile computing device 190 to ensure this alignment. Enabling the device to leverage its own lens and the lens system 192 of the receptacle 110 can improve the quality of images obtained for panoramic images. As discussed above, and depicted in Fig. 11, the lens system 192 of the receptacle 100 must also be positioned so that its entrance pupil 192 is aligned with the axis of rotation 128 of the member 125.
- the embodiments depicted in Figs. 9 and 11-13 illustrate a lens system 195 affixed to the receptacle 110, and positioned to align with the lens of the mobile computing device
- the receptacle 110 does not have an attached lens system 195.
- the mobile computing device includes the sole lens (or lenses) used for image capture.
- the receptacle 110 would still be configured to position the entrance pupil of the lens of the mobile computing device according to the same principles guiding the positioning of the lens system 192 affixed to a receptacle 110.
- the receptacle 110 includes an open sleeve 111 with arms 112, 113 configured to receive a mobile computing device 190.
- a user may insert the device into the sleeve 111, and the arms 112, 113 hold the device in place.
- the receptacle 110 may include a slot 111 whose front and back faces define a space for housing the device.
- the receptacle 110 may include clamps that hold the device in place.
- the receptacle 110 may include flexible molded components that define at least part of the perimeter of the space for housing the device. When a user presses the device into the space, the component flex to accept and secure the device.
- the receptacle 110 may be mounted on the platform 120 in any manner desired.
- the receptacle 110 is coupled to the member 125 by sliding a portion of the receptacle 110 into a groove of the member 125.
- any other method of mounting or coupling may be used.
- the receptacle 110 may include one or more pins, and the member 125 may include one or more holes that receive the pins of the receptacle 110.
- a portable device for facilitating capture of panoramic images on a mobile computing device comprising:
- a receptacle configured to house a mobile computing device
- a platform comprising (1) a member, with an axis of rotation, configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis, and (2) a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
- P5. The portable device of claim PI, wherein the motor is coupled to a processor that executes code to drive the member.
- P7 The portable device of claim P6, wherein the motor drives the member to rotate by a predetermined number of degrees.
- the drivetrain comprises at least one drive gear with teeth of different lengths, wherein the teeth of different lengths determine the different velocities at which the member is driven.
- PI 1 The portable device of claim PI, wherein the predefined distance of radial offset between the mobile computing device and the axis of rotation of the member is less than 0.5 inches.
- PI 2 A method for facilitating capture of panoramic images on a mobile computing device, the method comprising:
- driving the member comprises: driving, by the motor, the member from the first location to the second location at a non-zero angular velocity.
- driving the member comprises: driving, by the motor, the member to rotate from the first location to the second location.
- driving the member comprises: driving, by the motor, the member to rotate by a predetermined number of degrees.
- driving the member comprises: driving, by the motor, a drivetrain coupled to the member, wherein the drivetrain comprises at least one drive gear with teeth of different lengths and positions of the teeth of different lengths determine when the member halts.
- a computer-readable medium having computer program code non-transitorily stored thereon, the computer program code for facilitating capture of panoramic images on a mobile computing device, the program code comprising:
- program code for monitoring motion of the mobile computing device including substantially zero and substantially non-zero velocities of the mobile computing device
- program code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.
- a portable device facilitates capture of panoramic images on a mobile computing device.
- the device includes a receptacle configured to house a mobile computing device.
- the device also includes a platform with a member, with an axis of rotation, that is configured to interface with the receptacle to offset radially the mobile computing device from the axis of rotation by a predefined distance, such that an entrance pupil of a lens system of the mobile computing device is aligned with the axis.
- the platform also includes a motor that drives the member at different velocities so that the mobile computing device captures an image for a panoramic image when a velocity of the member is substantially zero for a threshold period of time.
- the velocities may be angular velocities.
- the member may be a circular member.
- the motor may be a mechanical motor.
- the motor may be coupled to a processor that executes code to drive the member.
- the motor may drive the member to alternate between rotating and remaining still.
- the motor may drive the member to rotate by a predetermined number of degrees.
- the predetermined number of degrees may be thirty-six (36).
- the predefined distance of radial offset between the mobile computing device and the axis of rotation of the member may be less than 0.5 inches.
- a method facilitates capture of panoramic images on a mobile computing device.
- the method initiates, by a mobile computing device, a program to capture images of a scene for a panoramic image when the mobile computing device is stationary.
- the method also captures, by a mobile computing device, a first image of a scene at a first location.
- the method further drives, by a motor, a member configured to interface with a receptacle containing the mobile computing device from the first location to a second location.
- the method additionally halts, by the motor, further motion of the member for a predetermined period of time.
- the method determines, by a sensor of the mobile computing device, a lack of motion of the mobile computing device for a threshold period of time. In response to the lack of motion, the method captures, by the mobile computing device, a second image of the scene at the second location. Lastly, the method creates the panoramic image from the first and second images of the scene.
- the method may drive, by the motor, the member from the first location to the second location at a non-zero angular velocity.
- the method may drive, by the motor, the member to rotate by a predetermined number of degrees.
- a computer- readable medium has computer program code non-transitorily stored thereon, and the computer program code facilitates capture of panoramic images on a mobile computing device.
- the program code includes code for monitoring motion of the mobile computing device, including substantially zero and substantially non-zero velocities of the mobile computing device.
- the program code also includes code for capturing at least one image of a scene in response to determinations that the mobile computing device has remained at a substantially zero velocity for a threshold period of time.
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201662368471P | 2016-07-29 | 2016-07-29 | |
| US15/224,077 US11153355B2 (en) | 2016-07-29 | 2016-07-29 | Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users |
| US15/224,068 US20180035048A1 (en) | 2016-07-29 | 2016-07-29 | Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device |
| PCT/US2017/044342 WO2018022977A1 (en) | 2016-07-29 | 2017-07-28 | Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP3491589A1 true EP3491589A1 (en) | 2019-06-05 |
Family
ID=59558518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP17749318.6A Withdrawn EP3491589A1 (en) | 2016-07-29 | 2017-07-28 | Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP3491589A1 (en) |
| CA (1) | CA3071398A1 (en) |
| WO (1) | WO2018022977A1 (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070271338A1 (en) * | 2006-05-18 | 2007-11-22 | Thomas Anschutz | Methods, systems, and products for synchronizing media experiences |
| US8661353B2 (en) * | 2009-05-29 | 2014-02-25 | Microsoft Corporation | Avatar integrated shared media experience |
| GB2511721A (en) * | 2012-12-06 | 2014-09-17 | Nec Corp | Communication system |
-
2017
- 2017-07-28 EP EP17749318.6A patent/EP3491589A1/en not_active Withdrawn
- 2017-07-28 CA CA3071398A patent/CA3071398A1/en not_active Abandoned
- 2017-07-28 WO PCT/US2017/044342 patent/WO2018022977A1/en not_active Ceased
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018022977A1 (en) | 2018-02-01 |
| CA3071398A1 (en) | 2018-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11786821B2 (en) | Instant streaming of a mobile user interface to a device of a secondary user without application install | |
| US11575722B2 (en) | Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users | |
| EP2561466B1 (en) | Approaches for device location and communication | |
| US8539085B2 (en) | Networked computer system for communicating and operating in a virtual reality environment | |
| US9798382B2 (en) | Systems and methods of eye tracking data analysis | |
| US20200394754A1 (en) | Course profiling and sharing | |
| WO2021118962A1 (en) | Cross reality system with simplified programming of virtual content | |
| US8588809B2 (en) | Managing public resources | |
| EP3469804A1 (en) | Game play companion application | |
| CN105979133A (en) | Tracking shooting method, mobile terminal and system | |
| EP3491589A1 (en) | Systems and methods for providing individual and/or synchronized virtual tours through a realm for a group of users | |
| EP4276413A1 (en) | Map generation system and method | |
| US9519987B1 (en) | Managing character control in a virtual space | |
| KR102904246B1 (en) | Industrial robot control method based on user's remote movement and device utilizing the same | |
| CN120393406A (en) | Task perspective adjustment method and device, storage medium and electronic device | |
| US20250196000A1 (en) | Techniques for transitioning between locations in a virtual environment | |
| US20180035048A1 (en) | Apparatus and Method for Facilitating Capture of Panoramic Images on a Mobile Computing Device | |
| Mohammadi | Esplorare il concetto di Fiducia in un Team di Robot Aerei | |
| JP2023170707A (en) | System and method of controlling system | |
| WO2023009112A1 (en) | Augmented reality content rendering | |
| CN110505682A (en) | Image display device control information transmission method, system and computer-readable medium based on Internet of Things | |
| JP2003187268A (en) | Image processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20190211 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| AX | Request for extension of the european patent |
Extension state: BA ME |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SMARTER SYSTEMS, INC. |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20200407 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
| 18D | Application deemed to be withdrawn |
Effective date: 20201020 |