HK1179074B - Device linking - Google Patents

Description

Device linking

Technical Field

The invention relates to device linking.

Background

A user may interact with a variety of different devices on a given day. For example, a user may interact with a desktop PC, a laptop computer, a mobile communication device (e.g., a mobile phone), a game console, and so forth. However, traditional interactions with devices are often disjointed, such that interactions with one device are disjointed from interactions with another device. Furthermore, even if techniques are subsequently developed that attempt to correct this problem, these techniques are often complex and inefficient, and thus users often choose to forgo this functionality.

Disclosure of Invention

Device linking is described. In one or more implementations, data describing characteristics of a plurality of devices associated with a user account of a network service is maintained at the network service. A communication is formed for receipt by one of the plurality of devices, wherein the communication includes a portion of data related to another one of the plurality of devices and the portion of data is suitable for discovery by the receiving device of the another one of the plurality of devices to initiate a local area network connection between the two devices.

In one or more implementations, data identifying another computing device associated with a user account is received from a network service at a computing device associated with the user account. In response to the computing device determining that the other computing device is available via a local area network connection, the computing device forms a local area network connection with the other computing device. In response to the computing device determining that the other computing device is unavailable via the local area network connection, the computing device forms a non-local area network connection with the other computing device.

In one or more implementations, availability of a device is discovered through communication with a network service to support a companion experience, the availability being determined through association of the device with a user account. As a result of the discovery, data received from the network service is used to initiate a local area network connection between the computing device and the device, where the local area network connection can be used to transfer data involved in the peer experience.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

Drawings

The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference numbers in different instances in the description and the figures may indicate similar or identical items. The entities represented in the figures may indicate one or more entities and thus references to each entity in the singular or plural may be made interchangeably in the discussion.

FIG. 1 is an illustration of an environment in an example implementation that is operable to perform the device linking techniques described herein.

FIG. 2 is an illustration of a system in an example implementation showing the computing devices and service providers of FIG. 1 in greater detail.

FIG. 3 is a flow diagram depicting a procedure in an example implementation in which a network service is configured to mediate connections between devices.

FIG. 4 is a flow diagram depicting a procedure in an example implementation in which a computing device is configured to communicate with another computing device utilizing a local network connection and/or a remote network connection.

FIG. 5 is a flow diagram depicting a procedure in an example implementation in which a companion experience is supported through device linking.

FIG. 6 illustrates an example system that includes the computing device described with reference to FIG. 1.

Fig. 7 illustrates various components of an example device that can be implemented as any type of computing device described with reference to fig. 1-4 to implement embodiments of the techniques described herein.

Detailed Description

Overview

Conventional techniques used to link devices together typically involve multiple manual steps performed by a user. Moreover, these steps are often complex and thus users have traditionally not benefited from these techniques even when they are available.

Device linking techniques are described. In one or more implementations, techniques are described in which different types of devices may work in conjunction, such as using a mobile communication device to support interaction with a game console. Various technologies are discussed herein that can be used to link devices together, for example, to support such interaction. Examples of such a technique include utilizing: the use of "clouds" and local connections for performing set-up, the use of local and remote connections, support for fallback functions, etc. Further discussion of this and other techniques may be found in relation to the following sections.

In the following discussion, an example environment is first described in which the techniques described herein may be employed. Example processes that may be performed in this example environment, as well as other environments, are then described. Accordingly, execution of example processes is not limited to the example environment, and the example environment is not limited to execution of example processes.

Example Environment

FIG. 1 is an illustration of an environment 100 in an example implementation that is operable to employ techniques described herein. The illustrated environment 100 includes an example of two computing devices 102, 104 that may be configured in various ways. The computing devices 102, 104 may be configured, for example, as legacy computers (e.g., desktop personal computers, laptop computers, etc.), mobile stations, entertainment devices, gaming consoles communicatively coupled to display devices (e.g., televisions, mobile communication devices (e.g., wireless telephones, tablet computers)), netbooks, and so forth, as further described with respect to example operating environments and devices. Thus, the computing devices 102, 104 may range from full resource devices with substantial memory and processor resources (e.g., personal computers, game consoles) to low-resource devices with limited memory and/or processing resources (e.g., traditional set-top boxes, hand-held game consoles). In the illustrated implementation, the computing device 102 is configured as a gaming console and the other computing device 104 is configured as a mobile communication device, although other implementations are also contemplated, as described above.

Computing devices 102, 104 are each shown to include an input/output module 106, 108, respectively. The input/output modules 106, 108 represent functionality related to recognizing input and/or supplying output by a respective computing device. For example, the input/output modules 106, 108 may be configured to receive input from a keyboard, a mouse to identify a gesture and cause an operation corresponding to the gesture to be performed, and so on. The inputs may be detected by the input/output modules 106, 108 in a variety of different ways.

For example, the input/output module 106 may be configured to receive one or more inputs via touch interaction with a hardware device (e.g., the illustrated controller 110). The touch interaction may involve pressing a button, moving a joystick, moving across a trackpad, using a touch screen of a display device (e.g., the computing device 102 detects a finger of a user's hand or a stylus), and so forth.

The input/output modules 106, 108 may utilize the identification of the input to interact with a user interface output by the respective computing devices 102, 104 to interact with games, applications, browse the internet, change one or more settings of the computing devices 102, 104, and so forth. Various other hardware devices are also contemplated that involve touch interaction with the device. Examples of such hardware devices include cursor control devices (e.g., mice), remote controls (e.g., television remote controls), mobile communication devices (e.g., wireless telephones (shown as computing device 104) configured to control one or more operations of computing device 102), and other devices involving a touch on the part of a user or object.

The input/output modules 106, 108 may also support a Natural User Interface (NUI), such as to recognize interactions that do not involve touch. For example, the computing devices 102, 104 may utilize an input device to detect input without the user touching the particular device, such as by recognizing audio input using a microphone. For example, the input/output modules 106, 108 may be configured to perform speech recognition to recognize a particular utterance (e.g., a spoken command), as well as to recognize a particular user that provided the utterance.

In another example, the input/output modules 106, 108 may be configured to recognize gestures, presented objects, images, and the like by using a camera. For example, a camera may be configured to include multiple lenses so that different viewpoints may be captured and thus depth determined, as shown for computing device 102 in a game console configuration. For example, different viewpoints may be used to determine a relative distance from the input device and thus may be used to determine a change in the relative distance. The various viewpoints may be used as depth perception by the respective computing devices 102, 104. Naturally, other images may also be used without depth sensing, such as a camera of the computing device 104 configured as a mobile communication device. These images may also be used to provide various functions, such as techniques to identify particular users (e.g., through facial recognition), objects, perform searches, and so forth.

The input-output modules 106, 108 may utilize these inputs to perform skeletal mapping along with feature extraction for particular points of the human body (e.g., 48 skeletal points) to track one or more users (e.g., 4 users simultaneously) for motion analysis. For example, the input/output modules 106, 108 may analyze the captured images to identify one or more movements made by the user, including what body parts were used to make the movement and which user made the movement. An example is illustrated by identifying the location and movement of one or more fingers of the user's hand 112 and/or identifying movement of the user's hand 112 as a whole. The motion may be identified by the input/output modules 106, 108 as a gesture that initiates a corresponding operation.

The computing devices 102, 104 are also shown to include respective linking modules 114, 116. The linking modules 114, 116 represent functionality of the respective devices for initiating and managing one or more network connections between the devices. These connections may be used to support a variety of different functions, such as a companion experience. For example, a computing device 104 configured as a mobile communication device may interact with a computing device 102 configured as a game console to supplement the user experience. This may include using computing device 104 as a game controller, outputting an electronic program guide to control the output of broadcast content by computing device 102, and so forth. Thus, interaction with computing device 104 may be used to control one or more operations performed by computing device 102, and vice versa. For example, the computing device 102 may provide supplemental content for output by the computing device 104.

The linking modules 114, 116 may include a variety of different functions for initiating and managing network connections. For example, the linking modules 114, 116 may include functionality for forming a local area network connection 118 between devices (e.g., a local WiFi connection) and/or forming a remote connection involving a network 120 (e.g., "through the cloud") by utilizing a service provider 122 accessible via the internet. Thus, in this second example, the service provider 122 is also shown to include a linking module 124 that represents functionality of the service provider 122 that is also used to support device linking functionality.

For example, the link modules 114, 116 may utilize a remote connection of the network 120 to contact the service provider 120 to perform device discovery, e.g., "locate" the device with which it is communicating. This data is then used to set up the local area network connection 118 between these devices to support the companion experience described above. In another example, such a connection may be maintained in whole or in part through a remote connection involving network 120 (e.g., the internet or other wide area network). Thus, the link modules 114, 116 may utilize a variety of different types of connections and techniques to form the connections, further discussion of which may be found in relation to the following figures.

Generally, any of the functions described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), or a combination of these implementations. The terms "module," "functionality," and "logic" as used herein generally represent software, firmware, hardware, or a combination thereof. In the case of a software implementation, the module, functionality, or logic represents program code that performs specified tasks when executed on a processor (e.g., CPU or CPUs). The program code can be stored in one or more computer readable memory devices. The features of the techniques described below are platform-independent, meaning that the techniques may be implemented on a variety of commercial computing platforms having a variety of processors.

For example, the computing devices 102, 104 may also include entities (e.g., software) that cause hardware of the computing devices 102, 104 to perform operations, such as processors, functional blocks, and so forth. For example, the computing devices 102, 104 may include computer-readable media that may be configured to maintain instructions that cause the computing devices, and more particularly hardware of the computing devices 102, 104, to perform operations. Thus, the instructions are used to configure hardware to perform operations and in this way cause hardware transformations to perform functions. The instructions may be provided by the computer-readable medium to the computing device 102 through a variety of different configurations.

One such computer-readable medium configuration is a signal bearing medium and thus is configured to transmit instructions (e.g., as a carrier wave) to the hardware of the computing device, e.g., over a network. The computer-readable medium may also be configured as a computer-readable storage medium and thus is not a signal bearing medium. Examples of computer-readable storage media include Random Access Memory (RAM), Read Only Memory (ROM), optical disks, flash memory, hard disk memory, and other storage devices that may use magnetic, optical, and other technologies for storing instructions and other data.

FIG. 2 illustrates a system 200 that illustrates computing devices 102, 104 and service provider 122 in more detail. The connection between the computing devices 102, 104 to support the companion experience may be initiated and maintained in various ways. For example, each of the computing devices 102, 104 may be associated with a user account of the network service of the service provider 122. Thus, a user may simply log into the user account of the service provider 122 by providing credentials via the network 120 without involving additional login information, key codes, or the like. These credentials may then be processed by the account manager module 202 of the service provider 122 to authenticate the user. For example, this authentication may be used to access a variety of different services of the service provider 122 (and other service providers) through a "one time" login, such as a music service, a messaging service, a calendar service, a contacts service, and so forth.

Once authenticated, the functionality of the linking module 124 may be exposed, such as for forming a connection between devices. For example, the linking module 124 may be configured to maintain data describing network connection details that may be used to form a network connection between devices. This may include data describing the details of the local network connection 118, such as by using an identifier, network name, etc. to support a Wi-Fi connection. This data may also describe remote connection details for access via network 120 (e.g., the internet), such as IP addresses, supported bandwidths, location information, network access types, and so forth.

This data may be communicated to service provider 122 in various ways and at various times. For example, the data may be communicated as part of authentication, may be stored from a previous communication, may be provided in response to a request received from the service provider 122 (e.g., after authentication is completed), and so forth. Thus, the link modules 114, 116 may communicate a variety of different data that may be used to form a connection.

In one or more implementations, various settings for controlling whether to provide such data may be exposed on the respective linking modules 114, 116. For example, configuration settings for making the respective computing device discoverable may be exposed, which may be set to "on" by default, although other examples are also contemplated.

Additionally, another configuration setting may be used to control whether the computing device is to maintain a live connection with the service provider 122, which may be set to "off" by default. This may be used to reduce resource consumption (e.g., of network 120 and/or service provider) so that service provider 122 is not forced to maintain device connection features for devices that do not wish to maintain device connection features. For example, this setting may initially be set to "off". However, once a connection is attempted, this setting may be switched "on" automatically and without user intervention to maintain a "ready" open connection to perform the linking described herein.

To initiate the connection, the computing devices 102, 104 may first "discover" each other in various ways. For example, the link modules 114, 116 may be configured to first determine whether another device is available via a local area network connection 118, such as via Wi-Fi, bluetooth, or other wired or wireless network. This discovery may be configured to utilize data previously stored by the respective link modules 114, 116, such as identification of a particular network identifier of the respective computing device 102, 104, network, and other information, although other examples are also contemplated.

If no device is so discovered, the link modules 114, 116 may communicate with the service provider 112 to discover if another device is available for connection. For example, the computing devices 102, 104 may communicate data indicating the location of the device, data that may be used to discover the device through a local connection, and so on. The data may indicate a particular location, such as being in a particular room, utilizing GPS coordinates, and other location determination functions. Further, this information may be used to determine the type of connection to establish, such as to establish a remote connection via network 120 if local area network connection 118 is unavailable, e.g., devices are separated from each other by a distance greater than that supported by local area network connection 118.

For example, computing device 104 may communicate with link module 124 of service provider 122 via network 120 to determine whether other devices (e.g., computing device 102) registered to the user's account are available for linking. The service provider 122 may then return an answer, which may include additional local area network connection information (e.g., wireless or wired subnets) for these devices. The link module 116 of the computing device 104 may then use this information to search the local area network in an attempt to discover one or more other devices. If found, the computing devices 102, 104 may negotiate a direct link for communication via the local area network connection 118, which may support more efficient communication than is supported via the network 120 in one or more cases. For example, the local area network connection 118 may support a higher bandwidth than a remote connection via the network 120. Furthermore, cost considerations may also be used as part of the decision process regarding which network to use, e.g., a Wi-Fi network versus a mobile phone network with an upper usage limit.

If not found, the computing devices 102, 104 may communicate via the network 120 in various ways. For example, the communication may pass through the service provider 122 as an intermediary. Thus, in this example, the communications may utilize the internet or other wide area network to connect the devices to one another. In another example of a remote connection, tunneling may be supported to pass communications, such as by direct communication via network 120 by the respective link modules 114, 116 utilizing the IP addresses of other devices without the service provider 122 actively acting as an intermediary.

Various other examples are also contemplated, such as hybrid modes in which different communications are communicated via different networks. For example, such hybrid modes may be used to support the delivery of commands via network 120 and the delivery of content via local area network connection 118, and vice versa. This division of communications may be performed for various reasons, such as due to limitations in the topology of the particular network connections supported by the respective networks.

In some cases, the characteristics of the physical connection may change during use. Accordingly, the link modules 114, 116, 124 may be configured in a variety of different ways to address these changes. For example, the linking module 114, 116, 124 may be configured to notify the user (e.g., via a user interface) of the change. Additionally, the link modules 114, 116, 124 may be configured to adjust (e.g., disable) features that do not work well in this state, such as reduced resolution, communications intensive functions, features not supported by the network, and so forth.

Further, the link modules 114, 116, 124 may be configured to cache commands, which may be used to improve efficiency and handle intermittent connectivity issues. This caching may be performed at the computing devices 102, 104 and at the service provider 122. Various other examples are also contemplated.

For example, the linking modules 114, 116, 124 may be configured to support automatic fallback recovery. For example, the local network connection 118 may degrade or become disconnected, such as due to the computing device 104 moving away from the computing device 102, network interference, and so forth. In these cases, the linking modules 114, 116, 124 may cause the connection to be made via the network 120 instead, may decide to employ a hybrid format as described above, and so on. The same is true in reverse, because if the reliability of the network 120 is reduced, the local area network connection 118 may be utilized to support communication between devices automatically and without user intervention.

This change may also be used to switch networks in response to determining that another one of the networks becomes available. For example, the computing device 104 may initially communicate with the computing device 102 over the internet, such as when the computing device 104 is located at a distance that does not support the local area network connection 118. In response to determining that the computing device 104 is now within range of the local area network of the computing device 102, the link modules 114, 116 may automatically communicate over the local area network connection 118. As noted above, various considerations may be taken into account when using this functionality, such as the cost considerations discussed above. Thus, device linking may be supported with a variety of different functionality, which may also be used to support a variety of additional functionality, such as the companion experience described above.

The link modules 114, 116, 124 may also support various other functions. For example, as described above, the connection may be bidirectional such that each of the devices may transmit data to and receive data from the other device. This functionality can be utilized in various ways. For example, the computing device 102 may be configured to notify the computing device 104 of the current state of content output. The computing device 104 can then utilize this information to provide functionality such as locating related content, performing an internet search based on one or more scenes associated with the related content, and so forth. The same is true in reverse, as the computing device 104 can communicate to the computing device 102 a state that can be used by the device to support functionality, such as continuing playback of content by the computing device 102 at a current point corresponding to content output by the computing device 104.

In another example, the link modules 114, 116, 124 may also support various different functional encryption algorithms to secure both communications via the local area network connection 118 and communications remotely via the network 120, among others. Further, while the Internet is described with respect to network 120, the inventive technique may also support a variety of different types of networks, such as using a single domain, as part of an enterprise, an intranet, and so forth. Further discussion of device linking techniques may be found in relation to the following procedures.

Example procedure

The following discussion describes device linking techniques that may be implemented utilizing the above-described systems and devices. Aspects of each of the procedures may be implemented using hardware, firmware, or software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In portions of the following discussion, reference will be made to the environment 100 of FIG. 1 and the system 200 of FIG. 2.

FIG. 3 is a flow diagram depicting a procedure 300 in an example implementation in which a network service is configured to mediate connections between devices. Data describing characteristics of a plurality of devices associated with a user account of a network service is maintained at the network service (block 302). For example, the link module 124 of the service provider 122 may receive data from the computing devices 102, 104 associated with the user's account. Such data may be received in response to a selection of a setting at a respective device to permit discovery of the device.

A communication is formed for receipt by one of the plurality of devices, wherein the communication includes a portion of data related to another one of the plurality of devices and the portion of data is suitable for discovery of the other one of the plurality of devices by the receiving device to initiate a local area network connection between the two devices (block 304). For example, the communication may include data that may be used to locate the device locally, such as a wired or wireless sub-network via which the other device may be accessed through a local area network connection. The communication may also include data, such as an IP address, that may be used to remotely locate the device. This data can then be used to form connections that can be used to support various functions, such as the companion experience described above.

Fig. 4 is a flow diagram depicting a procedure 400 in an example implementation in which a computing device is configured to communicate with another computing device utilizing a local network connection and/or a remote network connection. Data identifying another computing device associated with the user account is received from a network service at a computing device associated with the user account (block 402). As described above, the data may describe the device in various ways, such as by a network address, the name of the device, and so forth.

In response to the computing device determining that the other computing device is available via a local area network connection, the computing device forms a local area network connection with the other computing device (block 404). For example, where available, the computing device 102 may form a local wireless connection (e.g., Wi-Fi) with the computing device 104.

In response to the computing device determining that the other computing device is unavailable via a local area network connection, the computing device forms a non-local area network connection with the other computing device (block 406). Continuing with the previous example, if computing device 104 is unavailable via local area network connection 118, computing device 102 may form a network connection via network 120 (e.g., the internet or other wide area network). Various other examples are also contemplated.

FIG. 5 is a flow diagram depicting a procedure 500 in an example implementation in which a peer experience is supported through device linking. The availability of a device is discovered through communication with a network service to support a companion experience, the availability being determined through association of the device with a user account (block 502). For example, a computing device 104 configured as a mobile communication device (e.g., a wireless telephone) may be used by the service provider 122 to communicate to determine whether a device, such as the computing device 102 configured as a game console, is available.

As a result of the discovery, data received from the network service is used to initiate a local area network connection between the computing device and the device, where the local area network connection can be used to transfer data involved in the peer experience (block 504). For example, the computing device 104 may receive the following data: the data describes the wired or wireless networks via which the computing device 102 is available. Various other examples are also contemplated, examples of which may be found in relation to the following implementation examples.

Implementation examples

Implementation examples of the previously described techniques are described below. In one or more peer experience scenarios, the user can use the device to browse a video catalog or the like and then pick a movie, rent it and play it on the console. During the movie the user can use the mobile communication device or other devices to control it, e.g. play/pause, fast forward and rewind, etc. The game console may also be configured to notify the device of what is happening on the console, such as the current movie status, title changes on the console, and so forth. From the device, the user can start the title on the console, for example to get the title ID of the title running on the console.

In terms of communication between devices, message exchanges may fall into various categories, examples of which include:

operation: how do i trigger work on another device?

Notification: how do i be informed of a status change on another device?

There are various notifications that may occur in the system:

activity title change: a new title is started. This notification occurs when a new title is launched on the console (whether via controller input or companion command).

Media state change: some aspects of the playhead (playhead) status have changed, such as the content ID, playback rate, playhead position, or play/pause status. This notification occurs both periodically to maintain synchronization of the position variables among the devices, and instantaneously whenever a change occurs based on user input (e.g., a stop button is pressed).

There are various operations that may be issued in the system:

start-up title: a console title is launched, optionally using a command line argument that indicates which piece of media content is to be displayed. This command may be issued by a companion device (also referred to as a "companion" in the following discussion) upon selection of a new piece of content from the guide or search results.

Obtain the campaign title: the console is queried for the currently running title. This command may be invoked when a peer first connects to the console to obtain the initial title ID and whenever the client explicitly refreshes this information (e.g., returns from sleep). The result of this command contains the same information as the active title change notification.

Send input: an input command is sent to the console. This command is issued by the peer each time a transport control (e.g., play, pause, stop) is clicked.

Obtain the media status: the console is queried for the current media state. This may be invoked when a peer first connects to the console to obtain an initial media state and whenever the client needs to explicitly refresh this information (e.g., return from sleep). The result of this command contains the same information as the media state change notification.

Media status

In this example, the primary data structure used in both the protocol and the API is a media state structure. This structure represents the current playhead status and content ID played within the media application/title. The media state may be derived from the console media API and includes the following fields/attributes:

scourttransportstate (search transport state) is an enumeration taken from the console media API:

additional enumerated values for indicating that media is not playing (e.g., a game is running) may be configured as follows:

// no media played

SCOURTRANSPORTSTATE_NOMEDIA=-1,

When this value is used, the remaining fields of the media state are undefined.

TransportCapabilities is a flag enumeration indicating what operations a media player may perform:

communication

A communication stack for enabling companion scenarios may combine local area low latency TCP and UDP messaging with cloud-based services to support security and device discovery and communication between devices without line-of-sight IP connectivity.

The communication may be coordinated through a cloud (e.g., a web service). The console registers with the companion service for discovery by the companion device. The companion device uses the companion service to determine which device it can communicate with. If line-of-sight IP connectivity exists between the console and the companion device, subsequent communications between the device and the console may occur through local TCP and UDP messaging without a service intermediary. If there is no line-of-sight IP connectivity between the console and the companion device, communication may occur via the companion service, but with higher latency. The companion application may adapt its user interface based on whether a low-latency communication stack is available, thereby disabling the "meaningless" feature when relying on cloud-based messaging.

Device discovery/pairing/authorization may occur through a companion service. The system may perform this as follows:

1. the companion service uses an authenticated network ID corresponding to the login ID.

2. A given device communicates with the console on the device that the current user is logged into. Guest pairing/authorization using an invitation code or other higher level user interface may also be supported.

The console may re-register with the companion service when the set of logged-in users changes. Part of this registration may include a set of active users, the IP address of the console, and the TCP port used to listen for local peer commands. Upon registration, the companion service may return a secure session key that the console may use to securely sign and encrypt messages on the local subnet.

When a companion device attempts to join the session, it contacts the companion service, which then returns both the network address of the console to which the current user is logged on and the secure session key that can be used to sign and seal messages on the local subnet.

Communication with the service may be performed through HTTPS. If line-of-sight IP connectivity is available, then for the command, subsequent communications may occur using TCP/IP (using the TCP/IP address of the console); and for notifications, subsequent communications may occur using UDP broadcasts (using the IP subnet address of the console). If line-of-sight IP connectivity is not available, subsequent communications may occur via the companion service.

In one or more implementations, the following is possible: TCP connectivity to the console is possible but UDP broadcast to the device is not possible because the console and the companion device are separated by an IP router. In this case, the companion device may receive the notification via the companion service, but still issue commands (and receive their responses) through direct TCP to the console.

Safety feature

In addition to the security provided over the local area sub-network (e.g., WEP or WPA over Wi-Fi), communications in the system may be protected as follows.

Peer-to-peer service

Communication between the companion device and the service may be performed through HTTPS, which may be authenticated using a network ID corresponding to a valid console ID. For example, the mobile communication device may obtain an authentication (e.g., SAML) token from a linking service (e.g., XBL), which it then presents to a companion service. The companion service then issues one or more security tokens for subsequent invocations of the service. For example, one token may be used as a subsequent invocation of the service, and another token may be used by the console and the mobile communication device to authenticate messages.

Console to companion service

Communication between the console and the service may be performed through HTTPS. When the set of XBL users logged onto the console changes, the console can obtain a SAML token from the XBL, which it then presents to the companion service. The companion service then issues a security token for subsequent invocations of the service.

Peer-to-peer device to console/console-to-peer device

Once the companion devices or consoles authenticate against the companion service, the devices may then establish a secure session to communicate with each other. A session may be considered a security context that may be used by multiple devices to communicate. Each session may include:

1. a session id (guid) tracked by the service that uniquely identifies this communication session.

2. A 128-bit session key that is used to sign and encrypt messages sent over the local subnet. The console may re-authenticate to the companion service whenever the set of users logged into the console changes, and may generate a new session key for the session if a previous user logs out.

Messages sent between devices on the local subnet may be integrity protected and encrypted. The HMAC-SHA1 may be used to provide integrity protection, while AES-128 in CBC mode may be used to perform encryption. The sequence number may be used to implement replay protection. The recipient may maintain a 'high water' number and reject messages with lower numbers.

Console implementation

The vast majority of the communication stack of the companion can be implemented in the console operating system, with a minimal set of APIs exposed to each title.

Console API

The companion API may be called by each title. This API may be referred to as "LrcSetMediaState". LrcSetMediaState is called by the media player title to pass the playhead status or the content ID has changed. This function may be called as follows:

1. in response to an explicit change in content ID (e.g., changing from playing a first movie to playing a second movie within the same console title/application)

2. In response to processing a transfer control request (e.g., stop pressed, play rate changed due to FF/REW).

3. Periodically invoked as the playhead state progresses due to normal playback, including reaching the end of the stream or buffering the beginning or end.

Implementation of this API may cache the data passed in the last call to satisfy subsequent requests for playhead status without disrupting execution of the application or consuming title resources.

An implementation of this API may implement heuristics to determine when media state change notifications are actually sent based on the type of change that occurred. In general:

1. a change in field rather than location may trigger a notification to be sent on the next available opportunity.

2. Changes made only to the location field may not trigger notification transmission. Instead, the console operating system may send periodic media state change notifications, and the next may pick the last change in location. For periodic changes on the local subnet, these changes may be sent every 10 seconds. For periodic changes on the cloud, these changes may be sent every 30 seconds.

The signature of the API is as follows:

the function returns S _ OK if successful and E _ FAIL if failed.

LrcGetInput/LrcGetInputWithSeek

The lrcgetlnput/LrcGetInputWithSeek API is designed to be called as part of the title's input polling routine. LrcGetInput is designed to be called from a title that cannot support a seek command for getting a control command from a companion device. LrcGetInputWithSeek is designed for titles that can support the "seek" operation.

If there is an input event, the function returns ERROR _ SUCCESS. If there is no input event, the function returns ERROR _ EMPTY (NULL).

The pdwUserIndex (user INDEX pointer) is a pointer to the INDEX of the check-in user (e.g., player) associated with the device, which may be a value ranging from 0 to XUSER _ MAX _ COUNT-1 (maximum user number-1), or set to XUSER _ INDEX _ ANY (ANY user INDEX) to get the next available input event from ANY user.

Returning, the variable pointed to by the pdwUserIndex may contain an index of the player associated with the device that was the source of the input event. This is useful in the case where the variable pointed to by the pdwUserIndex contains an XUSER _ INDEX _ ANY on the input.

The dwFlags parameter may be XINPUT _ FLAG _ ANYDEVICE (input FLAGs ANY device) or XINPUT _ FLAG _ angyuser (input FLAGs ANY user) (in the case of a pdwuserldex having a value XUSER _ INDEX _ ANY).

The pKeystroke parameter may be a non-null pointer to a XINPUT keytroke structure.

The pSeekPos (search location) parameter may be a non-null pointer to ULONGLONG.

For LrcGetInput, if the function returns ERROR _ SUCCESS, the structure referenced by pKeystroke may contain XINPUT _ KEYSTROKE data for this input event.

For LrcGetInputWithSeek,

1. if the input is a seek command, ULONGLONGG referenced by pSeekPos may contain the desired location (in 100 ns), and the structure referenced by pKEystroke may be undefined.

2. If the input is not a seek command but the function returns ERROR _ SUCCESS, ULONGLONG referenced by pSeekPos may be-1 and the structure referenced by pkeustroke may contain XINPUT _ KEYSTROKE data for this input event.

For both APIs, the Human Interface Device (HID) code corresponding to the input is standard hardware code. The userldex can be set to the correct index based on the current user of the companion device (which can be zero to three). In the case where a new key is not pressed (and in the case of GetInputWithSeek, there is no seek information), these APIs return ERROR _ EMPTY. If the pdwUserIndex contains an ID for the input, but the ID does NOT have a corresponding login user, then these APIs return ERROR _ DEVICE _ NOT _ CONNECTED (DEVICE NOT CONNECTED).

Implementation of companion components within a console

At boot time, the console creates a TCP listening socket on the dynamic port between X and Y to support the incoming connection for commands. The snoop queue length is one (1) and the console can receive one incoming connection at a time to save resources. This means that after servicing an incoming command request and sending a corresponding response, the console can close the TCP connection before accepting the next call on the listening socket.

When sending notifications over the local subnet, the console can create a UDP socket, make a call to sendto, and then close the socket. Note that the use of both TCP and UDP is optimized to reduce the number of open sockets, which is a proper optimization of the code running in the console. The protocol may be designed to allow implementations to hold a TCP connection open more than one message exchange. In addition to the above-described socket usage, the console may consume one additional socket to interact with the companion service.

Upon each login change involving the XBL account/profile login or logout, the console contacts the companion service, indicating that the set of users on the console has changed. This call also registers the console's local IP address and the TCP port that is used to listen for incoming command requests. In addition, pending COMET-like HTTP requests may remain "docked" on the service to respond to incoming requests from non-line-of-sight IP devices. This request is retransmitted once every 30 seconds and terminates when the login set on the console changes.

Console resource consumption

The total socket usage from the console is:

1 statistically allocated outbound TCP socket for HTTP communications with the service, which is used for both login set registration and COMET event pull.

1 statistically distributed TCP snoop sockets for local subnet requests

1 dynamically allocated TCP stream sockets for servicing incoming local subnet requests

1 dynamically allocated UDP sockets for sending Notification messages

This means that a minimum of two sockets are used and a maximum of 4 (if UDP notifications are allowed to be sent before sending a TCP response) or 3 (if notifications are deferred to be sent until the TCP connection is torn down).

Protocol overview

The protocol supports call operations on the console using a direct TCP connection initiated from the companion device to the console. The protocol design supports multiple pending requests per TCP connection, and out-of-order response delivery, however, the console implementation of the present invention can close the connection after the first response is sent.

The protocol uses UDP broadcasts from the console to the companion devices to support sending notifications. The following message formats may be securely sent over TCP or UDP using the signing/encryption rules described below.

Message format

The message may be encoded in a binary big endian format on the network. Each message field may be aligned by their natural boundaries (i.e., WORD on a 2-byte boundary, DWORD on a 4-byte boundary, etc.). The fixed length string is encoded as UTF-8 text ending with '\ 0' and does not contain a leading Unicode BOM, which can be stripped by the writer.

The secure framing protocol is defined for use in both TCP connections and UDP payloads. The formats of these messages include:

1. a fixed length message header containing version information, security data, address information, and a message ID.

2. A variable length message body containing message type specific data. The length of the message body is indicated by a field in the message header.

3. A fixed length message trailer containing an HMAC-SHA1 signature over the message header and body.

Message header

The message starts with a 32 byte message header, whose contents are as follows:

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength = < number of bytes in the rest of the message >

DWORD sequence number = initialization vector for replay detection, correlation with reply, and for encryption, for each message >

DWORD ProtocolVersion=0x00000001

DWORD To = < device id To which the message is intended, 0xFFFFFFFF indicates broadcast >

DWORD From = < device id From which message comes, for addressing response >

DWORD MessageKind = < see below >

DWORD MessageType = < see below >

The "To" and "From" message header fields are used To support replay detection, since each companion device has its own sequence number. Without the "From" field in the request, the controller would be unable to determine which client sent the message and therefore unable to determine the correct sequence number. Without the "To" field in the response, an attacker could potentially replay a message intended for one device To a different device.

There are two authentication fields in the message header: messagekidd and MessageType. The messagekidd field indicates that the message is:

[0x00000001] request message, whether it is used to request an operation to be performed (e.g., command, query, connection management), or whether it is

[0x00000002] response message, whether it conveys the result of an operation performed in response to a particular request message, or whether it is a response message

[0x00000003] Notification message that conveys a state change event

The MessageType field identifies the format and semantics of a given operation or notification. An example list of supported message types is:

[0x80000001] JoinSession (request and response)

[0x80000002] LeaveSession (request and response)

[0x00000001] GetActiveTitleId (request and response)

[0x00000002] LaunchTitle (request and response)

[0x00000003] SendInput (request and response)

[0x00000004] GetMediaAndTitleState (request and response)

[0x00000005] NonMedia TitleStateNotification

[0x00000006] MediaTitleStateNotification

Response messages have a to additional field in their message header.

DWORD ResponseTo = < this corresponding request sequence number >

DWORD ResultCode = < HRESULT-based status code >

The "response" message begins with a four byte result code that is treated as a HRESULT. Specifically, the value 0x00000000/S _ OK indicates successful execution of the requested operation. A specific result code is defined for each response message type.

Message trailer

The message ends with 20 bytes.

BYTE [20] HMAC = < HMAC-SHA1 on header signature (HeaderSignature), message length (MessageLength), sequence number (sequence number) and encrypted versions of the subsequent message header fields and the entire message body >

Message text

This section defines the format and semantics of the specific message types that the protocol can support. The bytes after the message header field sequence number and before the message tail are encrypted.

JoinSession request message

The message is sent from the companion device to the console to (a) ensure that the protocol versions match and (b) obtain initial sequence numbers for inbound and outbound messages. The JoinSession request/response may occur before any additional messages from the companion device are sent to the console over the local subnet.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength = < number of bytes in the rest of the message >

DWORD sequence number =0xnnnnnnn < random initial value >

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000001< request >

DWORD MessageType=0x80000001<JoinSession>

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

JoinSession response message

The message is sent from/to the companion device to (a) ensure that the protocol versions match and (b) convey an initial sequence number for inbound and outbound messages.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength = < number of bytes in the rest of the message >

DWORD sequence number =0xnnnnnnn < always sequence number +1 of the request >

DWORD ProtocolVersion=0x00000001

DWORD To = < device id To which the message is intended, same as From in the JoinSession request message >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000002< response >

DWORD MessageType=0x80000001<JoinSession>

DWORD ResponseTo=0xnnnnnnnn

DWORD ResultCode = < see below >

DWORD SupportedProtocolVersion=0xnnnnnnnn

DWORD ClientSequenceNumber = < sequence number available to client for next request >

DWORD Notification sequence number = < sequence number available to the server for the next UDP Notification message >

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

If the result code (ResultCode) is S OK (0), then the requested protocol version is supported. Also:

supported protocol version contains the protocol version number supported by the server.

ClientSequenceNumber (client sequence number) contains the sequence number that the client can use for the next message it sends to the server.

Notification sequence number (Notification sequence number) contains the sequence number of the next notification message to be sent by the server over UDP.

If the result code is E _ VERSION _ MISMATCH (0x8 hhhhh), the session has not been joined and only the SupportedProtocolVersion field is valid. If the result code is E _ TOO _ MANY _ CONNECTIONS (0x8 hhhhh), the session has not been joined and none of SupportedProtocolVersion, ClientSequenceNumber, and Notification SequenceNumber are valid.

Getactivtitleid request message

This message is sent from the companion device to the console to query the console for the active title ID.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000001< request >

DWORD MessageType=0x00000001<GetActiveTitleId>

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

Getactivttleid response message

This message is sent from/to the companion device to/from the console in response to the getactivtitleid request message, and indicates a title currently running.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id To which the message is intended, same as From in GetActiveTitleId request message >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000002< response >

DWORD MessageType=0x00000001<GetActiveTitleId>

DWORD ResponseTo=0xnnnnnnnn

DWORD ResultCode=0x00000000

DWORD TitleId=0xnnnnnnnn

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

The Title Id is a console Title Id of a Title currently running on the console.

LaunchTitle request message

This message is sent from the companion device to the console to initiate the title with the specified command line argument.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000001< request >

DWORD MessageType=0x00000002<LaunchTitle>

DWORD TitleId

DWORD Launch ParameterLength; (unused)

BYTE [900] LaunchParameter (UTF-8 text ending with Null)

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

The Title Id is a console Title Id of a Title currently running on the console. The LaunchParameter field typically identifies the content that is to be played once the title is launched. The exact interpretation of this field is title specific.

LaunchTitle response message

This message is sent from the console to the companion device/from the companion device to the console to indicate header start success/failure.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id To which the message is intended, same as From in the LaunchTitle request message >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000002< response >

DWORD MessageType=0x00000002<Launch>

DWORD ResponseTo=0xnnnnnnnn

DWORD ResultCode=0x00000000

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent message header fields and encrypted version of the entire message body >

SendInput request message

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000001< request >

DWORD MessageType=0x00000003<SendInput>

DWORD Validfields =0x01-VirtualKey,0x02-SeekPos,0x 03-both

DWORD VirtualKey = < virtual key from XDK >

ULONGLONG SeekPosition

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and encrypted version of subsequent field >

The ValidFields field has the value 0x01 if the request contains a keystroke, the value 0x02 if the request contains a seek command, and the value 0x03 if it contains both. VirtualKey (virtual key) is equivalent to its definition in XINPUT _ keystone. The seek position is used to convey a seek command. If the request message does not indicate a seek, then this field has a value of 0 xFFFFFFFFFFFFFFFFFFFFFF (-1).

SendInput response message

This message is sent from the console to/from the companion device to the console to indicate success/failure of the SendInput operation.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id To which the message is intended, same as From in SendInput request message >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000002< response >

DWORD MessageType=0x00000003<SendInput>

DWORD ResponseTo=0xnnnnnnnn

DWORD ResultCode=0x00000000

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent header fields and encrypted version of the entire message body >

GetMediaAndTitleState command message

This message is sent from the companion device to the console to query the status of the media on the console.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000001< request >

DWORD MessageType=0x00000004<GetMediaAndTitleState>

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and subsequent header fields and encrypted version of the entire message body >

GetMediaAndTitleState response message

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To = < device id for which the message is intended, same as From in GetMediaAndItterState request message >

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000002< response >

DWORD MessageType=0x00000004<GetMediaAndTitleState>

DWORD ResponseTo=0xnnnnnnnn

DWORD ResultCode=0x00000000

DWORD TitleId

ULONGGLONG Duration (in 100 ns)

ULONGGLONG Position (100 ns as unit)

ULONGGLONG MinSeek (in 100 ns)

ULONGGLONG MaxSeek (in 100 ns)

FLOAT Rate (playback Rate, 1.0= = normal)

DWORD TransportState (see ScourTransportState enumeration in the Console API below)

DWORD TransportCapabilities (see TransportCapabilities enumeration in the Console API below)

DWORD MediaAssetIdLength; (unused)

BYTE [256] MediaAssetId (UTF-8 text ending with Null)

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and encrypted version of subsequent field >

If the result code is S _ OK (0) and the TransportState is not SCOURTRANSPORTSTATE _ NOMEDIA, then the other media status fields (Duration, Position, …, MediaAssetId) are all valid. If the result code is S _ OK (0) and the TransportState is SCOURTRANSPORTSTATE _ NOMEDIA, then there is no current media on the console and the remaining media status values are undefined.

Non-media title status notification message

The nonimanditlestatNotification message indicates that a non-media-enabled console title (e.g., a game) is currently running on the console. The nonimanditlestatNotification message is sent by the console via UDP broadcast when:

1. titles of non-enabled media (e.g., games) are running. And

2. since the update time interval (10 seconds), no nona media title state notification or mediatitle state notification has been sent. Or the title of the non-enabled media has been started.

This message does not need to be sent from the console to the cloud because the title change forces re-authentication to the cloud, which conveys the title ID. This message may be sent from the cloud to the companion device.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To=0xFFFFFFFF

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000003< Notification >

DWORD MessageType=0x00000005<NonMediaTitleStateNotification>

DWORD TitleId

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and encrypted version of subsequent field >

Mediatitlestateinotification message

The mediatitlestatNotification message indicates that a media-enabled console title (e.g., a title related to a video streaming service) is currently running on the console. This message conveys the console title ID as well as the current content ID and the playhead status.

The mediatitlestateinotification message is sent by the console via UDP broadcast when:

1. a title (e.g., game) of the enabled media is running. And

2. since the update interval (10 seconds), no nonamediastatetitlenotification or mediatitlestatentification has been sent. Or that the title of the enabled media has been started or that the transport control command has been processed (e.g., played, stopped) by the title.

The mediatitlestateinotification message is sent by the console to the cloud when:

1. a title (e.g., game) of the enabled media is running. And

2. since the update interval (30 seconds), no noni mediatitlestatatenotification or mediatitlestatnotification has been sent. Or that the title of the enabled media has been started or that the transport control command has been processed (e.g., played, stopped) by the title.

This message may be sent from the cloud to the computing device.

DWORD HeaderSignature=0xBEDABEDA

DWORD MessageLength=0xnnnnnnnn

DWORD SequenceNumber=0xnnnnnnnn

DWORD ProtocolVersion=0x00000001

DWORD To=0xFFFFFFFF

DWORD From = < device id From which message comes >

DWORD MessageKind =0x00000003< Notification >

DWORD MessageType=0x00000006<MediaTitleNotification>

DWORD TitleId

ULONGGLONG Duration (in 100 ns)

ULONGGLONG Position (100 ns as unit)

ULONGGLONG MinSeek (in 100 ns)

ULONGGLONG MaxSeek (in 100 ns)

FLOAT Rate (playback Rate, 1.0= = normal)

DWORD TransportState (see ScourTransportState enumeration in the Console API below)

DWORD TransportCapabilities (see TransportCapabilities enumeration in the Console API below)

DWORD MediaAssetIdLength; (unused)

BYTE [256] MediaAssetId (UTF-8 text ending with Null)

BYTE [20] HMAC = < HMAC-SHA1 on header signature, message length, sequence number and encrypted version of subsequent field >

If the transportState is not SCOURTRANSPORTSTATE _ NOMEDIA, the other media status field (Duration, Position, …, MediaAssetId) is valid. If the transportState is SCOURTRANSPORTSTATE _ NOMEDIA, then there is no media currently on the console and the remaining media state values are undefined.

Console and cloud communication

When a user logs in to the console, the console reports the user information to the cloud so that the cloud can know who logged in to the console. The console also tells the cloud its local subnet IP address. When the user logs off, the console reports to the cloud.

Notification model

The console may use a unicast method to announce certain changes on the console like title changes, media state changes, etc. A socket established between the device and the console may be used to do this.

On the companion device side, the runtime library may provide notification capabilities. That is, the link module may register whatever events it is interested in, and the runtime layer may notify the application of these events when they occur.

Runtime libraries on the device side

Runtime libraries may be used on each supported device.

Here are example APIs that may be supported.

1.bool JoinSession()

This API may connect the device to a console host specified by "hostpipaddress".

And returning a value: return TRUE if the connection is good. Otherwise, false is returned.

Example uses: JoinSession ();

after pairing is successful, it can get the console's local subnet IP address from the cloud. It may also obtain security keys from the cloud that can be used to secure communications with the console.

2.DisconnectSession()

This API may close the connection between your device and the console to which it is currently connected. Note that: the runtime uses this API to clean up session data; the socket with the console is closed. Of course, the console may know when the device goes to sleep. It can close the socket.

3.TitleInfo[]GetAvailableTitles()

This API may provide you with a list of titles that the living room companion experience currently supports.

For example, here is one of the possible returns of this function:

4.unsigned int GetCurrentRunningTitleId()

this API returns you the titleID of the title currently running on the console currently connected to.

5.void Launch(unsigned int TitleId,string parameter)

This API can use the given parameters specified in "parameter" to launch the application specified by "TitleId".

Titleid- -the title ID of the application you want to launch. The caller gets the friendly application name by calling "getlistofavailablenotes ()".

Parameter — you want the parameters passed to the title during startup.

6.void SendControlCommand(CommandType key)

This API sends console control commands to the console to which it is currently connected.

7. Notification API

These APIs are used to enable devices to receive notification events from the console like status changes, title changes, etc. on the console. Although specific examples are described, it should be apparent that the present discussion and the appended claims are not necessarily limited to these examples.

Example systems and devices

FIG. 6 illustrates an example system 600 including the computing device 102 described with reference to FIG. 1. The example system 600 enables a ubiquitous environment for a seamless user experience when running applications on a Personal Computer (PC), a television device, and/or a mobile device. Services and applications run substantially similarly in all three environments to get a common user experience when transitioning from one device to the next when using an application, playing a video game, watching a video, etc.

In the example system 600, multiple devices are interconnected through a central computing device. The central computing device may be local to the plurality of devices or may be located remotely from the plurality of devices. In one embodiment, the central computing device is a cloud of one or more server computers connected to the plurality of devices through a network, the internet, or other data communication link. In one embodiment, the interconnect architecture enables functionality to be delivered across multiple devices to provide a common and seamless experience to users of the multiple devices. Each of the multiple devices may have different physical requirements and capabilities, and the central computing device uses a platform to enable an experience to be delivered to the device that is both tailored to the device and yet common to all devices. In one embodiment, a class of target devices is created and experiences are customized for the generic class of devices. The class of devices may be defined by physical characteristics, type of use, or other common characteristics of the devices.

In various implementations, computing device 102 may assume a variety of different configurations, such as for computer 602, mobile device 604, and television 606 uses. Each of these configurations includes devices that may have generally different constructs and capabilities, and thus the computing device 102 may be configured according to one or more of the different device classes. For example, the computing device 102 may be implemented as the computer class 602 device that includes personal computers, desktop computers, multi-screen computers, laptop computers, netbooks, and so forth.

The computing device 102 may also be implemented as a mobile class 604 device that includes mobile devices such as mobile phones, portable music players, portable gaming devices, tablet computers, multi-screen computers, and the like. The computing device 102 may also be implemented as a television class 606 device that includes devices having or connected to generally larger screens in casual viewing environments. These devices include televisions, set-top boxes, game consoles, and the like. The techniques described herein may be supported by these various configurations of the computing device 102 and are not limited to the specific examples described herein.

Cloud 608 includes and/or represents a platform 610 for content services 612. The platform 610 abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud 608. The content service 612 may include applications and/or data that may be used when computer processing is executed on a server that is remote from the computing device 102. The content service 612 may be provided as a service on the internet and/or over a subscriber network such as a cellular or Wi-Fi network. An example of this is shown as including linking module 114 on the computing device. As described above, these techniques may also utilize the "cloud," for example, by implementing linking module 124 as part of platform 310, as described below.

The platform 610 may abstract resources and functionality to connect the computing device 102 with other computing devices. The platform 610 may also be used to abstract scaling of resources to provide a corresponding level of scaling to requirements encountered by a content service 612 implemented via the platform 610. Thus, in an interconnected device embodiment, implementation of functions described herein may be distributed throughout the system 600. For example, the functionality may be implemented in part on the computing device 102 and via the platform 610 that abstracts the functionality of the cloud 608.

Fig. 7 illustrates components of an example device 700 that may be implemented as any of the types of computing devices described with reference to fig. 1, 2, and 6 for implementing the embodiments described herein. Device 700 includes a communication device 702 that enables wired and/or wireless communication of device data 704 (e.g., received data, data being received, data scheduled for broadcast, data packets of the data, etc.). The device data 704 or other device content can include configuration settings of the device, media content stored on the device, and/or information associated with a user of the device. Media content stored on device 700 can include any type of audio, video, and/or image data. Device 700 includes one or more data inputs 706 via which any type of data, media content, and/or inputs can be received, such as user-selectable inputs, messages, music, television media content, recorded video content, and any other type of audio, video, and/or image data received from any content source and/or data source.

Device 700 also includes communication interfaces 708 that can be implemented as any one or more of a serial and/or parallel interface, a wireless interface, any type of network interface, a modem, and as any other type of communication interface. The communication interfaces 708 provide a connection and/or communication links between device 700 and a communication network by which other electronic, computing, and communication devices communicate data with device 700.

Device 700 includes one or more processors 710 (e.g., any of microprocessors, controllers, and the like) which process various computer-executable instructions to control the operation of device 700 and to implement embodiments of the techniques described herein. Additionally or alternatively, device 700 can be implemented with any one or combination of hardware, firmware, or fixed logic circuitry that is implemented in connection with processing and control circuits which are generally identified at 712. Although not shown, device 700 can include a system bus or data transfer system that couples the various components within the device. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures.

Device 700 also includes computer-readable media 714, such as one or more memory components, examples of which include Random Access Memory (RAM), non-volatile memory (e.g., any one or more of a read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and a disk storage device. A disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and/or rewriteable Compact Disc (CD), any type of a Digital Versatile Disc (DVD), and the like. The device 700 may also include a mass storage media device 716.

Computer-readable media 714 provides data storage mechanisms to store the device data 704, as well as various device applications 718 and any other types of information and/or data related to operational aspects of device 700. For example, an operating system 720 can be maintained as a computer application with the computer-readable media 714 and executed on processors 710. The device applications 718 can include a device manager (e.g., a control application, software application, signal processing and control module, code that is native to a particular device, a hardware abstraction layer for a particular device, etc.). The device applications 718 also include any system components or modules to implement embodiments of the techniques described herein. In this example, the device applications 718 include an interface application 722 and an input/output module 724 that are shown as software modules and/or computer applications. Input/output module 724 represents software for providing an interface with devices such as a touch screen, track pad, camera, microphone, etc. that are configured to capture input. Alternatively or in addition, the interface application 722 and the input/output module 724 can be implemented as hardware, software, firmware, or any combination thereof. Further, the input/output module 724 may be configured to support multiple input devices, such as separate devices that capture visual and audio inputs, respectively.

The device 700 also includes an audio and/or video input-output system 726 that provides audio data to an audio system 728 and/or video data to a display system 730. The audio system 728 and/or the display system 730 can include any devices that process, display, and/or otherwise render audio, video, and image data. Video signals and audio signals may be communicated from device 700 to an audio device and/or a display device via an RF (radio frequency) link, S-video link, composite video link, component video link, DVI (digital video interface), analog audio connection, or other similar communication link. In an embodiment, the audio system 728 and/or the display system 730 are implemented as external components to device 700. Alternatively, the audio system 728 and/or the display system 730 are implemented as integrated components of the example device 700.

Conclusion

Although the invention has been described in language specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed invention.

Claims (10)

1. A method implemented by one or more service provider computing devices implementing a network service, the method comprising:

associating a plurality of devices with a user account of the network service;

maintaining, at the network service, data describing characteristics of the plurality of devices (302); and

forming a communication for receipt by one of the plurality of devices, wherein the communication includes a portion of data maintained at the network service relating to another one of the plurality of devices, the portion of data adapted for discovery by the receiving device of the another one of the plurality of devices to initiate a local area network connection between the two devices to facilitate a companion experience between the devices (304).

2. The method of claim 1, wherein the communication further comprises another portion of data related to the another one of the plurality of devices and adapted to discover the another one of the plurality of devices to initiate a remote network connection between the two devices.

3. The method of claim 1, wherein the portion of data describes a wired or wireless subnetwork via which the other device is accessible over the local area network connection.

4. The method of claim 1, wherein the portion of data is usable by the receiving device to form a remote network connection with the other device in response to determining that the other device is not accessible via the local area network connection.

5. The method of claim 1, wherein the portion of data is usable by the receiving device to form a hybrid network connection with the other device that includes the local area network connection and a remote network connection.

6. The method of claim 1, wherein the portion of data is usable by the receiving device to perform a fallback operation to switch between the local network connection and the remote network connection in response to determining the unavailability of the local network connection or the remote network connection.

7. The method of claim 1, wherein the portion of data is usable by the receiving device to support encryption of communications between the receiving device and the other device.

8. A system for device linking, the system comprising:

means for receiving data associated with a user account from a network service, the data identifying another computing device associated with the user account, the computing device and the another computing device associated with the user account prior to receipt;

means for, in response to determining that the other computing device is available via a local area network connection, forming, by the computing device, the local area network connection with the other computing device using data received from the network service to facilitate a companion experience between the computing device and the other computing device; and

means for, in response to determining that the other computing device is unavailable via a local area network connection, forming, by the computing device, a non-local area network connection with the other computing device using data received from the network service to facilitate a companion experience between the computing device and the other computing device.

9. The system of claim 8, wherein the data describes a wired or wireless subnet via which the other computing device is to be enabled to be accessed over the local area network connection.

10. The system of claim 8, wherein the data is provided to the network service by the other computing device.