WO2025131286A1 - Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service - Google Patents
Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service Download PDFInfo
- Publication number
- WO2025131286A1 WO2025131286A1 PCT/EP2023/087339 EP2023087339W WO2025131286A1 WO 2025131286 A1 WO2025131286 A1 WO 2025131286A1 EP 2023087339 W EP2023087339 W EP 2023087339W WO 2025131286 A1 WO2025131286 A1 WO 2025131286A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- user device
- privacy
- session
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/107—Network architectures or network communication protocols for network security for controlling access to devices or network resources wherein the security policies are location-dependent, e.g. entities privileges depend on current location or allowing specific operations only from locally connected terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/105—Multiple levels of security
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1069—Session establishment or de-establishment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1073—Registration or de-registration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1083—In-session procedures
- H04L65/1089—In-session procedures by adding media; by removing media
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/02—Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W12/00—Security arrangements; Authentication; Protecting privacy or anonymity
- H04W12/60—Context-dependent security
- H04W12/63—Location-dependent; Proximity-dependent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/80—Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
Definitions
- the present disclosure relates to providing communication services through user terminals of a wireless communications system.
- a user can be provided a communication service through one or more input and/or output (I/O) user devices which are proximately located to the user and connected to a cloud-based user terminal emulator. Which I/O user devices are selectable for use to provide the service to the user can depend upon their user interface (UI) capabilities being able to satisfy requirements of the service.
- UI user interface
- Example types of UI capabilities of I/O user devices include display devices, cameras, microphones, keyboards, speakers, sensors, etc. Proximity of I/O user devices to the user can be determined based on which I/O user devices can receive a beacon signal transmitted by a user tag being transported by the user.
- I/O user devices It can become problematic from a privacy perspective to provide user terminal emulation services through I/O user devices. For example, a user may desire to conduct a videoconference to discuss company proprietary information using UI capabilities of one or more I/O user devices, but privacy may not be maintained if other users begin using the same or other proximately located I/O user devices.
- Some embodiments disclosed herein are directed to an input and/or output (I/O) user device handler (IODH) that includes at least one processor and at least one memory storing program code that is executable by the at least one processor to perform operations.
- the operations include obtaining a session privacy requirement for a communication session.
- the operations determine whether the first I/O user device does not satisfy a privacyproximity rule defined for a second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O user interface capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device.
- the operations control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy -proximity rule.
- a user can receive and initiate communication services without the necessity of a traditional all- inclusive feature-rich user terminal.
- the operations can emulate a user terminal using the networked second I/O user device that is proximately located to a user tag transported by a second user, and where the second I/O user device individually or combined with one or more other I/O user device(s) has/have user interface (UI) capabilities to provide an I/O user interface for the second user to interface with the communication service function (e.g., a user terminal emulation application of a user terminal emulation server) to provide a communication service.
- the communication service function e.g., a user terminal emulation application of a user terminal emulation server
- the IODH can protect privacy of the second user's communications from being monitored (eavesdropped-on) by a first user who is requesting a session between the first I/O user device which is proximately located to the second I/O user device and having an I/O user interface capability (e.g., microphone, camera, etc.) that does not satisfy the session privacy requirement for the ongoing communication session of the second I/O user device.
- an I/O user interface capability e.g., microphone, camera, etc.
- Some related embodiments disclosed herein are directed to an IODH having at least one processor and at least one memory storing program code that is executable by the at least one processor to perform operations.
- the operations include obtaining a session privacy requirement for a communication session. Based on a request for the communication session between a first I/O user device of a set of I/O user devices served by the I/O user device handler and a communication service function of a network entity, the operations determine whether a second I/O user device of the set does not satisfy a privacy-proximity rule based on location relative to the first I/O user device and based on having an I/O user interface capability that does not satisfy the session privacy requirement for the communication session with the first I/O user device. The operations control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule.
- IODH can protect privacy of the first user's communications from being monitored (eavesdropped- on) by a second user who is using a second I/O user device.
- the IODH can allow the communication session with the first I/O user device to be setup if the second I/O user device satisfies the privacy-proximity rule or otherwise may prevent setup of the requested communication session or allow setup but modify what privacy-affecting I/O user interface capabilities (e.g., microphone, camera, etc.) are allowed to be used through communication session by the first and/or second I/O user device.
- I/O user interface capabilities e.g., microphone, camera, etc.
- Figure 1 illustrates a system with a user terminal emulation server that operationally integrates sets of I/O user devices that are proximately located to users to logically form virtualized user terminals providing communication services in accordance with some embodiments of the present disclosure
- Figure 2 illustrates a block diagram illustrating the user terminal emulation server communicating with various elements of a cellular system to provide communication services in accordance with some embodiments of the present disclosure
- Figure 3 illustrates a block diagram illustrating the user terminal emulation server communicating in a different manner with various elements of a cellular system to provide communication services in accordance with some other embodiments of the present disclosure
- FIGS 4 and 5 illustrate combined flowcharts of operations and related data flows between a user tag, I/O user devices, an Extensible Authentication Protocol (EAP) authenticator, and a user terminal emulation server which may include an EAP server in accordance with some embodiments of the present disclosure;
- EAP Extensible Authentication Protocol
- Figure 6 illustrates a combined flowchart of operations and related data flows between a user tag, I/O user devices, a 3GPP key agreement function system, and a user terminal emulation server in accordance with some embodiments of the present disclosure
- Figure 7 illustrates a block diagram of hardware circuit components of an I/O user device which are configured to operate in accordance with some embodiments
- Figure 8 illustrates a block diagram of hardware circuit components of a user terminal emulation server and/or an I/O device handler (IODH) that are configured to operate in accordance with some embodiments of the present disclosure
- Figure 9 illustrates a block diagram of hardware circuit components of an EAP authenticator or AKMA server that are configured to operate in accordance with some embodiments of the present disclosure
- Figure 10 illustrates a block diagram of hardware circuit components of a core network node that are configured to operate in accordance with some embodiments of the present disclosure
- Figure 11 illustrates a block diagram of hardware circuit components of a radio network node that are configured to operate in accordance with some embodiments of the present disclosure
- Figures 12A and 12B illustrate combined flowcharts of operations for monitoring and controlling privacy of communications through I/O user devices with communication service functions in accordance with some embodiments of the present disclosure
- Figure 13 illustrates a flowchart of operations that can be performed by a IODH to monitor and control privacy of communications through I/O user devices in accordance with some embodiments of the present disclosure
- Figure 14 illustrates another flowchart of operations that can be performed by a IODH to monitor and control privacy of communications through I/O user devices in accordance with some other embodiments of the present disclosure.
- Embodiments are directed to enabling a user to receive and initiate communication services without the necessity of a traditional all-inclusive feature-rich user terminal.
- Operations emulate a user terminal using a I/O user device that is proximately located to a user tag transported by a user, and where the I/O user device individually or combined with one or more other I/O user device(s) has/have user interface (UI) capabilities that are required or preferred to provide an I/O user interface for the user to interface with a communication service function (e.g., a videoconference service, media streaming service, etc. which interfaces with the I/O user device(s) through a user terminal emulation application of a user terminal emulation server) to provide a communication service.
- a communication service function e.g., a videoconference service, media streaming service, etc. which interfaces with the I/O user device(s) through a user terminal emulation application of a user terminal emulation server
- Further operations are directed to enabling monitoring
- example system components and operations are described for performing user terminal emulation as a cloud computing service in accordance with some embodiments.
- Dynamic allocation of I/O user device capabilities whenever and wherever the I/O user devices are in the proximity of a user enables efficient and flexible use of existing hardware, such as televisions, conference phones, laptops, surveillance cameras, connected household appliances, connected cars, etc., that is capable of providing necessary UI functionality to user during a communication service.
- the user thereby has reduced or no need to carry an expensive and all-inclusive user terminal, e.g., smart phone, that includes all necessary UI capabilities, display device, keyboard, speakers, etc.
- the user may instead carry a hardware device which operates to identify the user, referred to as a "UserTag” or “user tag", over a wireless or wired (e.g., smartcard reader) communication interface, such as a near field communication (NFC) interface, to one or more of the I/O user devices.
- a wireless or wired (e.g., smartcard reader) communication interface such as a near field communication (NFC) interface
- NFC near field communication
- a user terminal emulation server can operate to provide a user terminal, which can also be referred to as a SoftUE or a user terminal emulation application that is run by the user terminal emulation server.
- Figure 1 illustrates a system with a user terminal emulation server 100 that can use one or more I/O user devices 130 that is/are proximately located to users to logically emulate a user terminal providing a communication service in accordance with some embodiments of the present disclosure.
- the user terminal emulation server 100 may use the UI capability of a single I/O user device 130 or operationally integrate the UI capabilities of a set of the I/O user devices 130 to logically emulate a user terminal providing communication services in accordance with some embodiments of the present disclosure.
- the user terminal emulation server 100 may be a cloud resource that is networked and remote from the I/O user devices 130, or may be more proximately located on a shared network with the I/O user devices 130. Alternatively, functionality of the user terminal emulation server 100 may be performed by an application hosted on one or more of the I/O user devices 130.
- the user terminal emulation server 100 is configured to communicate with the I/O user device(s) 130 proximately located to a user who can use the UI capabilities of the proximate I/O user device(s) 130 to be provided a communication service (e.g., voice call, videoconference, electronic mail, text messaging, extended reality meeting, etc.).
- a communication service e.g., voice call, videoconference, electronic mail, text messaging, extended reality meeting, etc.
- Users may carry a hardware tag, a.k.a. "UserTag", "user tag” or “UT”, which is capable of transmitting a unique user identifier through a communications interface, such as a near-field communications interface (e.g., Bluetooth, BLE, NFC, RFID, etc., or combinations thereof), for receipt by one or more of the I/O user devices 130 which are proximately located to the user.
- a communications interface such as a near-field communications interface (e.g., Bluetooth, BLE, NFC, RFID, etc., or combinations thereof)
- a near-field communications interface e.g., Bluetooth, BLE, NFC, RFID, etc., or combinations thereof
- One type of user tag can be a low-complexity standalone electronic device having limited operational capability for transmitting an identifier through a near-field communications interface and performing authentication operations such as described herein.
- Another type of user tag can have more operational capability (e.g., processing and memory hardware resources), such as a smartphone or smartwatch having cellular connectivity that transmits a cellular identity (e.g., from a SIM card, a USIM on a UICC, or an embedded SIM on an embedded UICC or on an integrated embedded UICC) and / or an application identity through a cellular interface or a near-field communications interface and is configured to perform authentication operations such as described herein.
- a user tag may be a device that does not require human interaction in order to interact with an I/O user device, and may lack a user interface.
- a user tag may be configured to interact with one or more types of Internet of things (loT) devices, such as a camera, sensor, actuator, or other electronic device having Internet or other wireless connectivity.
- LoT Internet of things
- the user identifier may alternatively or additionally be operationally identified through biometrics operations performed by, e.g., one or more of the I/O user devices 130.
- the biometrics operations may include, without limitation, voice recognition, image/face recognition, eye recognition, fingerprint recognition, gait recognition, gesture recognition, or a combination thereof.
- the user identity may be determined based on credentials provided by the user when, e.g., logging into an application or account.
- the user identity may be provided by a cell phone using information from the subscription SIM and proximity of the cell phone to one or more of the I/O user devices 130 can be determined using the phone’s NFC capability.
- a user identifier, a user tag identifier, and a user terminal emulation application 110 can be logically associated with each other in a database 120 during a user registration process or as part of another setup process.
- a user may obtain an account login identifier (serving as the user identifier) that is registered in the database 120 as being associated with a user tag identifier for a physical user tag that has been provided to (e.g., purchased by) the user and being associated with a user terminal application 110 that emulates a user terminal having defined capabilities (e.g., a cell phone providing cellular and over-the-type voice-over-IP communication services).
- the user terminal emulation server 100 may maintain in the database 120 network addresses of I/O user devices 130 and UI capabilities of the I/O user devices 130.
- the database 120 is illustrated as residing in the example server 100, in some other embodiments information described below as residing in the database 120 may alternatively or additionally be stored within an I/O user device handler (IODH) 212 in Figure 2 and/or the user terminal emulation applications 110.
- the capabilities of the I/O user devices 130 may be logically arranged in the database 120 based on the type of UI capability provided, e.g., display device, microphone, speaker, physical/virtual keyboard, and may be further arranged based on a quality of service provided by the UI capability.
- a different instantiation of the user terminal emulation application 110 may be hosted by the server 100 for each user who is to be provided communication services (i.e., illustrated user terminal emulation applications #1-#N corresponding to users 1-N).
- the user terminal emulation application 110 may perform registration of the user with the network entity 150 and setup of a communication service with a user responsive to communication requests.
- the operation to register the network address of the user terminal emulation application and the identity of the user with the network entity can include registering the network address of the user terminal emulation application 110 and the identity of the user with a network server of a VoIP communication service provider.
- the operation to register the network address of the user terminal emulation application and the identity of the user with the network entity can include registering the network address of the user terminal emulation application 110 and the identity of the user with a Home Subscriber Server (HSS) 211, Unified Data Management (UDM), or other network node of a core network operated by a cellular communication service provider.
- HSS Home Subscriber Server
- UDM Unified Data Management
- the user terminal emulation server 100 may receive the registration messages from the I/O user devices using the Session Initiation Protocol (SlP)ZSession Description Protocol (SDP), where each of the registration messages identifies the network address and the UI capability of one of the I/O user devices.
- the communication request may be received from the network entity 150 using the SIP/SDP, and the operation to provide communication sessions between the user terminal emulation application 110 and each of the I/O user devices in the set, and between the user terminal emulation application 110 and the requesting user terminal may be performing using the SIP/SDP.
- a registration message from an I/O user device can include, for example, an IP address and port number, MAC address, fully qualified domain name (FQDN), and/or another network address, and can further include information identifying the UI capability of the I/O user device.
- the I/O user device may power-up and responsively communicate the registration message to the user terminal emulation server 100.
- the user terminal emulation server 100 receives a communication request from the network entity 150 for establishing a communication service between the user and a requesting user terminal, e.g., a cellular phone, computer with Microsoft Teams application, etc. Responsive to the communication request, the user terminal emulation server 100 identifies one or more of the I/O user devices 130, which may be registered in the database, that are proximately located to a location of the user and are determined, based on the UI capabilities identified by the database 120 for the set of I/O user devices and based on content of the communication request, to satisfy a capability rule for being individually usable or combinable to provide an I/O user interface for the user to interface with the user terminal emulation application 110 to provide the communication service.
- a requesting user terminal e.g., a cellular phone, computer with Microsoft Teams application, etc. Responsive to the communication request, the user terminal emulation server 100 identifies one or more of the I/O user devices 130, which
- the user terminal emulation server 100 provides one or more communication sessions between the user terminal emulation application 110 and the one or more I/O user devices 130 and between the user terminal emulation application 110 and the requesting user terminal via the network entity 150.
- the communication request that is received by the user terminal emulation application 110 may contain an indication of a minimum UI capability that must be provided to the user during the communication service, such as: speaker only; combination of speaker and microphone; display only; combination of display device, speaker, and microphone; etc.
- a UI capability rule which can be used by the server 100 to determine whether a communication service can be provided and by which set of I/O user devices, may thereby be defined based on the minimum UI capability that is indicated by the communication request.
- the user terminal emulation server 100 then routes communication traffic between at least one of the I/O user devices in the set and the requesting user terminal via the network entity 150.
- the user terminal emulation server 100 selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the database 120 for the one of the I/O user devices, and then routes the data of the data type toward the network address of the selected one of the I/O user devices.
- the server 100 may also combine data streams that are received from the I/O user devices in the set, and route the combined data streams towards the requesting user terminal, e.g., via the network entity 150.
- the user terminal emulation server 100 may be responsible for tracking which I/O user devices are proximately located to a present location of the user.
- the server 100 can receive presence reports from individual ones of the I/O user devices containing their network address and an identifier of a user tag which is determined by the I/O user device to be proximately located thereto.
- an I/O user device may read a user tag through an NFC communication interface and/or may perform other operations to detect presence of a user and to identify a user tag transported by the user.
- the server 100 updates the database 120 to indicate which user tag identifiers are proximately located to which of the I/O user devices.
- a set of I/O user devices 130 has been determined by the IODH 212 (Fig. 2) to be proximately located to a location of a first user carrying UserTag#!, and further determined to have UI capabilities that are combinable to satisfy the UI capability rule for providing a combined I/O user interface for the first user to use during a requested communication service.
- IODH 212 (Fig. 2) responsively uses that set of I/O user devices 130 to provide a combined I/O user interface for use by the first user during a communication service via Application #1 and network entity 150 between the first user and another user terminal.
- IODH 212 (Fig. 2) to be proximately located to a location of a second user carrying UserTag#2, and to further have UI capabilities that are combinable to satisfy the UI capability rule for providing a combined I/O user interface for the second user to use during a requested communication service.
- IODH 212 responsively uses that set of I/O user devices 130 to provide a combined I/O user interface for use by the second user during a communication service via Application #2 and network entity 150 between the second user and yet another user terminal.
- Figure 1 also illustrates that another set of I/O user devices 130 is not proximately located to either UserTag#! or UserTag#2.
- the communication request which is requesting the establishment of communication service session ("communication session" or "session") with an identified user may be initiated by the network entity 150 using the network address of the user terminal emulation application and identity of the user which were earlier registered with the network entity 150.
- the communication request may additionally or alternatively be generated by one of the I/O user devices 130 responsive to a command received from a proximately located user.
- a user tag may operate automatically or through action of the user to initiate communications with one of the I/O user devices 130.
- a user may operate a user interface provided by one of the I/O user devices 130 to initiate, e.g., a combined audio and video call with another user.
- the speaker device and the microphone device are each identified as belonging to the set of I/O user devices that are determined to be proximately located to the location of the user (e.g., UserTag#! and are further determined, based on the UI capabilities identified by the database 120, to satisfy the UI capability rule for used individually or combined to provide a combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service.
- further operations are performed to route a microphone stream received from the microphone device toward the requesting user terminal (e.g., via network entity 150).
- the operations select the speaker device based on matching an audio characteristic of the audio stream to the speaker capability identified by the database 120 for the speaker device, and then route the audio stream toward the network address of the speaker device.
- the UI capabilities of devices may correspond to different UI functions and/or circuits provided by the same electronic device.
- the speaker device and the microphone device may correspond to a speaker function and circuit and to a microphone function and circuit, respectively, of a same smartphone, television, or other electronic device.
- the streams may then be routed to an application programming interface (API) of the targeted functions and/or circuits.
- API application programming interface
- the example embodiment may include, when a display device is one of the I/O user devices in the set capable of displaying a received video stream, the operations update the database 120 based on content of registration messages to identify network addresses of the display device, and to identify UI capabilities of the display device as having a display capability.
- the display UI capabilities may identify a screen display size, aspect ratio, pixel resolution, video frame rates supported, whether display device supports shared user support via split screen configuration, and/or other operational characteristics.
- the display device is also identified as among the set of I/O user devices that determined, based on the UI capabilities identified by the database 120, to satisfy the UI capability rule for being used individually or combined to provide the combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service.
- the set of I/O user devices is further selected based on each of the I/O user devices satisfying a rule for being proximately located to the location of the user.
- further operations Based on determining that the speaker device, the display device, and the microphone device satisfy the UI capability rule, further operations respond to receipt of video stream as communication traffic from the requesting user terminal by selecting the display device based on matching a video characteristic of the video stream to the display capability identified by the database 120 for the display device, and then routing the video stream toward the network address of the display device.
- the operations for routing the audio stream and the video stream toward the network addresses (or APIs) of the speaker device and the display device, respectively may include when audio data and video data are received within a same stream from the requesting user terminal through a first communication session: separating the audio data from the video data; routing the audio data toward the network address (or API) of the speaker device through a second communication session; and routing the video data toward the network address (or API) of the display device through the second communication session or a third communication session.
- the example embodiment may include, when a camera device is one of the I/O user devices in the set capable of providing a camera stream, the operations update the database 120 based on content of a registration message to identify a network address of the camera device and to identify a UI capability of the camera device as having a camera capability.
- the camera UI capabilities may identify a camera pixel count, image quality, light sensitivity, and/or other operational characteristics.
- the camera device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user and is further determined, based on the UI capability identified by the database 120, to satisfy the UI capability rule for being used individually or combined with the other I/O user devices in the set to provide the combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service. Based on determining that the camera device satisfies the UI capability rule, further operations are performed to route the camera stream received from the camera device toward the requesting user terminal, e.g., via the network entity 150.
- the operations for routing the microphone stream received from the microphone device and the camera stream received from the camera device toward the requesting user terminal can include: receiving the microphone stream from the microphone device through a first communication session; receiving the camera stream from the camera device through the first communication session or a second communication session; combining the microphone stream and camera stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity 150.
- the example embodiment may include, when a keyboard device is one of the I/O user devices in the set capable of outputting key selection data responsive to key selections by a user among keys of the keyboard device, the operations can update the database 120 based on content of a registration message to identify a network address of the keyboard device and to identify a UI capability of the keyboard device as having a keyboard capability.
- the keyboard device capabilities may identify a number of keys (buttons) that are user selectable, indication of whether the keyboard is a physical keyboard or a touch sensitive input device, and/or other keyboard capabilities.
- the keyboard device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user and is further determined, based on the UI capability identified by the database 120, to satisfy the UI capability rule for being used individually or combined with the other I/O user devices in the set to provide the combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service. Based on determining that the keyboard device satisfies the UI capability rule, further operations are performed to identify commands formed by the key selection data received from the keyboard and to perform operations that have been predefined as being triggered based on receipt of the identified commands.
- the operations for routing the key selection data received from the keyboard device and microphone stream received from the microphone device may include: receiving the key selection data from the keyboard device through a first communication session receiving the microphone stream from the microphone device through the first communication session or a second communication session; combining the key selection data and the microphone stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity 150.
- FIG. 2 is a block diagram illustrating the user terminal emulation server 100 as an element of an operator service node 202 within a cellular system 200.
- the communication service function of the network entity 150 may be provided by the operator service node 202 or may be reached through external infrastructure 240, e.g., the Internet or private network.
- the server 100 may, for example, be implemented in the radio access network 220 to provide edge computing with faster responsiveness or may be implemented within another node of the cellular system 200.
- the user terminal emulation server 100 can include the IODH 212, a control function (CF) 214, the instantiated user terminal emulation applications 110, and a service gateway (GW) 216.
- a user terminal emulation application 110 may perform one or more user applications which are provided by a smart phone, such as a Netflix application, Meta (Facebook) application, Microsoft Teams application, Internet browser application, etc.
- the user terminal emulation server 100 may perform operations to manage the I/O user devices, such as to handle maintenance of the database 120, perform registration of I/O user devices to be available for use by the user terminal emulation applications 110, and manage mobility through operations for setting up and performing handover of communication services through I/O user devices.
- the user terminal emulation server 100 may operate to identify the IP address of a user terminal emulation application, e.g., which encapsulates a Microsoft Teams application, for a subscriber with a service provider, e.g., a Microsoft Teams server.
- the IODH 212 may be located outside the user terminal emulation server 100 in another network node of the system.
- the CF 214 may be responsible for assigning an IP address to each user terminal emulation application 110.
- the IP address to be assigned by the CF 214 may be received from the core network 210 functionality such as a PDN-GW.
- the service GW 216 may interconnect the user terminal emulation server 100 to a PSTN network, packet data network gateway of a 3GPP (3 rd Generation Partnership Project) system, etc.
- the cellular system 200 can include a Core Network 210 having a Home Subscriber Server (HSS) 211, a Policy and Charging Roles Function (PCRF), gateway (GW) and Mobility Management Entity (MME) providing control signaling related to mobile terminal mobility and security for the radio access.
- HSS 211 contains subscriber-related information and provides support functionality for user authentication and user access to the system.
- the PCRF enables QoS control per data flow and radio bearer, by setting QoS rules for each data flow, based on operator set policies and subscriber information.
- the GW can include a Serving GW (S-GW) and a Packet Data Network GW (PDN-GW), where the S- GW interconnects the core network 210 with the radio access network 220 and routes incoming and outgoing packets for the I/O user devices 232 and/or 130 and the user terminals 230.
- the PDN-GW interconnects the core network 210 with external infrastructure 240, such as the Internet, and allocates IP-addresses and performs policy control and charging.
- Some I/O user devices 232 having cellular communication capability can communicate via, e.g., eNBs or other radio access nodes of a Radio Access Network 220 with the operator service node 202 via the core network 210.
- the user terminal emulation server 100 may handle set up of a communication service between a selected set of the I/O user devices that are proximate to a user and a remote user terminal 230 (e.g., smart phone) via the cellular system 200.
- a user terminal emulation application 110 may be instantiated or otherwise activated responsive by an incoming call (service, session) targeting the user tag.
- the user terminal emulation application 110 can identify subscriptions associated with the user tag (e.g., registered in a user account) and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the incoming communication session.
- the user terminal emulation application 110 may ask the IODH 212 to identify which I/O user devices 130 are proximately located to the user tag, and may further ask the IODH 212 to determine or may determine itself whether the identified I/O user devices 130 are usable individually or combinable to satisfy the UI capabilities specified by the incoming communication session.
- the user terminal emulation application 110 and/or the IODH 212 may receive an ACK or NACK back on whether a sufficient set of I/O user devices 130 can be used to provide the communication service. If ACK, then the IODH 212 also sets the state of the I/O user devices 130 in the set to in-use to avoid another user terminal emulation application 110 attempting to utilize the same I/O user devices 130 as which are presently in use.
- the signaling may be implicit when the IODH 212 already has presence information and/or presence is determined based on content of IP packets sent from the I/O user device to the IODH 212.
- the IODH 212 may be executed by the user terminal emulation server as part of the user terminal emulation application, by the I/O user devices, and/or by another computing node of the system.
- the user terminal emulation application corresponding to the specific user i.e., the user tag
- the IODH 212 may operate to receive the notifications from the I/O user devices proximately located to the user tag.
- Further UI capability discovery (synchronization) communications may optionally be performed between the IODH 212 and the I/O user devices, or the IODH 212 may be per-configured with knowledge of the UI capabilities of the I/O user devices.
- the I/O user devices are associated to the user in the database 120, along with associated indications of combinable UI capabilities provided by the set of I/O user devices which are proximately located to the user tag.
- One or more of the I/O user devices may be selected for default call reception ACK/NACK.
- the user via the user tag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service.
- the user may receive or accept a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
- An incoming session (e.g., video call) from a requesting user terminal which is directed to the user (user tag) arrives at the user terminal emulation server for the user carrying the user tag.
- the individual or combinable UI capabilities of the available I/O user devices is compared to the UI requirements of the incoming session.
- the user terminal emulation server may renegotiate the required UI capabilities (e.g., QoS) of the incoming session.
- the user terminal emulation server prompts, via one or more of the available I/O user devices (e.g., a pre-selected answer device), the user carrying the user tag to provide a session request answer (ACK/NACK).
- ACK/NACK session request answer
- the user responds through the pre-selected answer device to accept (ACK) or reject (NACK) the incoming session, to provide signaling to the user terminal emulation server.
- operations route an audio stream from the requesting user terminal to one of the I/O user devices in the set that has a speaker capability via one or more sessions, and routes a video stream from the requesting user terminal to another one of the I/O user devices in the set that has a display capability via one or more sessions.
- a data stream that is received from one of the I/O user devices in the set through a one or more sessions is routed toward the requesting user terminal.
- two or more data streams are received through one or more sessions from the I/O user devices, they can be combined into a combined data stream that is routed toward the requesting user terminal.
- the user terminal emulation server or IODH 212 may perform operations to continuously monitor presence of the I/O user devices to determine when one or more of I/O user devices is no longer proximately located to the user such that it can no longer be included as part of the combined UI be provided during the ongoing communication session.
- the user terminal emulation server or IODH 212 may substitute the UI capability of another I/O user device to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence.
- This use case involves a user, with a user tag, being proximately located to I/O user devices 130 having different UI capabilities when an outgoing call (communication session) is received by the user terminal emulation server.
- the I/O user devices 130 are associated to the identified user via the user terminal emulation server 100 which handles all communications sessions for the user while the associated I/O user devices 130 are managed by an IODH 212.
- a user terminal emulation application 110 may be instantiated or otherwise activated responsive by an outgoing call being requested by a user carrying the user tag.
- the user may initiate an outgoing call through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
- the user terminal emulation application 110 can identify subscriptions associated with the user tag and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the outgoing call.
- the user terminal emulation application 110 may ask the IODH 212 to identify which I/O user devices 130 are proximately located to the user tag, and may further ask the IODH 212 to determine or may determine itself whether the identified I/O user devices 130 are individually useable or combinable to satisfy the UI capabilities specified by the outgoing call.
- the user terminal emulation application 110 and/or the IODH 212 may receive an ACK or NACK back on whether one or a set of I/O user devices 130 can be used to provide the communication service. If ACK, then the IODH 212 also sets the state of the one or more I/O user devices 130 in the set to in-use to avoid another user terminal emulation application 110 attempting to utilize the same I/O user device(s) 130 or one or more capabilities of that I/O user device(s) 130 which are presently in use. In case of NACK, the user terminal emulation application 110 and/or the IODH 212 can take different actions to setup a reduced UI capability communication service with the user depending on user settings, e.g. only allow sound instead of the preferred sound and video responsive to when no display device is presently available for use (e.g., when presently used by another user terminal emulation application 110 or when none is proximately located to the user tag).
- a user tag enters a room and signals its presence to any proximately located and capable I/O user device in the room using a discovery beacon signal.
- one or more of the I/O user devices determines presence of the user tag by polling, such as by periodically transmitting discover beacon signals that trigger responsive signaling by the user tag.
- the I/O user devices that receive signaling indicated presence of the user tag report to the IODH 212 along with a network address or other identifier of the I/O user device (e.g., IP address, port number, MAC address, FQDN, etc.) or API of an application function on the I/O user device.
- the IODH 212 may be executed by the user terminal emulation server as part of the user terminal emulation application, by the I/O user devices, and/or by another computing node of the system.
- the user terminal emulation application corresponding to the specific user i.e., the user tag
- the user terminal emulation application corresponding to the specific user is updated with respect to the detected user's presence.
- the IODH 212 may operate to receive the notifications from the I/O user devices proximately located to the user tag. Further UI capability discovery (synchronization) communications are performed between the user terminal emulation server or the IODH and the I/O user devices.
- the I/O user devices are associated to the user in the database 120, along with associated indicated service subscriptions and combinable UI capabilities provided by the set of I/O user devices which are proximately located to the user tag. One or more of the I/O user devices may be selected for use in making a call or initiating another service.
- the user via the user tag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service.
- the user may initiate a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
- a user carrying the user tag uses the UI of one of the I/O user devices to trigger an outgoing call (e.g., video call), which triggers signaling of the outgoing call to the user terminal emulation server.
- the IODH 212 and/or the user terminal emulation application queries the user (e.g., displays a message, generates a sound, etc.) through one of the I/O user devices proximately located to the user to request the user to select among available types of communication methods that can be presently used for the outgoing call.
- One of the I/O user devices provides responsive signaling to the IODH 212 or user terminal emulation server 100 indicating the user's selected type of communication method for the outgoing call.
- the user terminal emulation server communicates an outgoing session stream request to the network entity 150, where the request may include an identifier of the calling user, identifier of the user terminal of the called user, and a quality of service for the communication session.
- the user terminal emulation server receives a communication session acceptance (ACK) or denial (NACK) from the network entity 150.
- ACK communication session acceptance
- NACK denial
- the user terminal emulation server may attempt to renegotiate the requested communication session such as at a lower quality of service.
- the user terminal emulation server selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the database 120 for the one of the I/O user devices, and then routes the data of the data type toward the network address of the selected one of the I/O user devices or API of a selected application function on the I/O user device.
- the I/O user devices transmit data streams through one or more sessions to the user terminal emulation server, which may combine the data streams into a combined data stream that is routed toward the called user terminals via the network entity 150.
- the user terminal emulation server or IODH may continuously monitor presence of the I/O user devices to determine when one or more of I/O user devices is no longer proximately located to the user such that it can no longer be included as part of the combined UI be provided during the ongoing communication session.
- the user terminal emulation server or IODH may substitute the UI capability of another I/O user device to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence.
- EAP Extensible Authentication Protocol
- AKMA Authentication and Key Management for Applications
- the secure association enables the user terminal emulation server 100 to utilize the UI capabilities of those I/O user device(s) 130 to obtain a communication service.
- the cloud-based service may be a phone service, videoconference service, streaming media service, video on-demand service, audio on-demand service, web service, digital assistant service, service technician service and/or any other service which may operate to communicatively connect to physical resources in the proximity of a user.
- the operations receive an indication of an I/O user interface capability of the first I/O user device 130 through the secure channel connection with the first I/O user device 130.
- the operations communicate with the first I/O user device 130 to use the I/O user interface capability to provide at least part of the communication service for a user.
- EAP-Based Authentication and related operations are now discussed below.
- EAP is an authentication framework which supports multiple authentication methods.
- EAP typically runs directly over data link layers such as Point-to-Point Protocol (PPP) or IEEE 802, without requiring IP.
- PPP Point-to-Point Protocol
- EAP provides its own support for duplicate elimination and retransmission but is reliant on lower layer ordering guarantees. Fragmentation is not supported within EAP itself; however, individual EAP methods may support fragmentation.
- EAP is a two-party protocol spoken between the EAP peer and server.
- keying material is generated by EAP authentication algorithms, known as "methods”. Part of this keying material can be used by EAP methods themselves, and part of this material can be exported.
- EAP methods can also export associated parameters such as authenticated peer and server identities and a unique EAP conversation identifier, and can import and export lower-layer parameters known as "channel binding parameters", or simply "channel bindings”.
- An example, flow of an EAP exchange for access authentication (802. lx used for WLAN/LAN), can include the device attaching to an authenticator point (AP). This means an (e.g.) 802.11 association is established between device and AP.
- the AP requests the identity of the device with an EAP identity request message.
- the device replies with its identity in an EAP response message.
- the AP which works as an Authenticator, forwards the identity in a RADIUS or DIAMETER access request message to the authentication server.
- the authentication server replies with a challenge for the device and indicating the EAP method to be used.
- the AP forwards the request in an EAP request message to the device.
- the device responds to the EAP message with an EAP response message.
- the AP forwards the response in a RADIUS or DIAMETER message to the authentication server.
- EAP messages are exchanged between the device and the authentication server until the authentication server has authenticated the device using the chosen method.
- the authentication server sends a RADIUS or DIAMETER access accept message, containing a pairwise master key (PMK) to the AP.
- the AP keeps the PMK and forwards the success in an EAP success message to the device.
- the device generates the corresponding PMK.
- the device is authenticated and the AP and device can use the PMK to configure access security.
- EAP can also be run towards Authenticators other than WLAN APs, and the Authenticator can be co-located with/part of the Authentication server.
- FIG. 4 illustrates a combined flowcharts of operations and related data flows between the user terminal emulation application 110 and elements of an I/O user device (IOD) domain 410, such as I/O user devices 130, a lookup service/IODH, and an Extensible Authentication Protocol (EAP) authenticator 400 in accordance with some embodiments of the present disclosure.
- IOD I/O user device
- IOD Extensible Authentication Protocol
- an assumption in the EAP-based approach is that the user tag is transported with the user and contains circuitry which is configured to operate as an EAP client.
- the user tag therefore has at least limited communication and computational capabilities.
- An example user tag can be any type of mobile electronic device, such as tablet computer, mobile phone, or less complex device with NFC, RFID, Bluetooth, or other short- range communication such as capacitive communication coupling.
- the IODS 130 are registered with their (local) IODH 212, which functions as a lookup service for IOD A ... IOD x in the local IOD domain 410, i.e., the IODH 212 has information characterizing IOD A ... IOD x which may include an identifier, authentication credentials (e.g.
- the user terminal emulation application 110 may be executed within the IOD Domain 410, such as with the EAP Authenticator 400 and/or the IODH 212. Executing the user terminal emulation application on the same computing node or on a locally networked node as the EAP authenticator and/or IODH can simplify and reduce system resources consumed to exchange information and other communications therebetween.
- IOD x in the IOD domain 410 can be determined based on access control policies which can reside in the IODH 212 and may vary depending on use case scenario, e.g., semi-public domain, such as a hotel vs. private domain such as an enterprise office complex.
- operations establish 405 ( Figure 4) and 511 ( Figure 5) a secure channel connection with a first IOD 130 using a session identifier and an identifier associated with the first IOD to determine a first IOD specific key generated from a master key, where the first IOD specific key and the session identifier being used for secure communication of messages with the first IOD.
- a user who is transporting a user tag 101 wants to utilize a proximately located IOD A.
- the user may trigger IOD A, by pushing a button on the user tag 101, triggering a command responsive to the user tag seeing a Service Set IDentifier (SSID) or Bluetooth (BT) address of IOD A, and/or IOD A.
- SSID Service Set IDentifier
- BT Bluetooth
- the user may activate or initiate attention from IOD A using the user tag over NFC or RFID (i.e., very close proximity), and/or by the user pushing a button on the IOD A to initialize a bootstrapping process 501.
- the user tag can broadcast a beacon signal which can be heard by an IOD and which will trigger the authentication.
- the user tag sends 502, via the IOD (target), an attach request to the system where the IOD is connected.
- the attach request is forwarded 503 by the IOD to the EAP Authenticator 400 in the system.
- the EAP authenticator 400 may be a local process in OD A or in the IODH 212 of the user terminal emulation server 100, or may reside as a separate entity in the IOD domain 410.
- the attach request may therefore be processed by the IOD itself, i.e., where multiple EAP Authenticators are present in the system, be processed by the IODH 212, or be processed by the EAP authenticator 400.
- the EAP authenticator 400 responds 504 with an EAP identity request, which is communicated back to the user tag 101.
- the user tag 101 responds 505 with an EAP response, which carries the identity of the user tag 101.
- the identity identifies the user tag 101 (e.g., holder of user tag, such as the user) and points to the user terminal emulation application 110 associated with the user tag 101.
- the identity may be ⁇ hash(user_pub_key)>@ ⁇ user_terminal emulation application_address>.
- the hash of the public key provides a more compact identity of the public key of the user tag 101, and may be included with a network address of the user terminal emulation application 110.
- the user terminal emulation application 110 may provide a communication service function corresponding to, for example, an over-the-top Voice Over Internet Protocol (VoIP) service, Netflix service, Facebook service, Microsoft Teams meeting service, Internet browser service, a cellular communication service, etc.
- VoIP Voice Over Internet Protocol
- the user tag 101 When the user tag 101 has been issued to the user, it may have also been associated with the corresponding user terminal emulation application 110.
- the user tag 101 in addition to its own credentials (e.g., public key pair), can be configured to have reachability information of the user terminal emulation application 110, e.g., a Fully Qualified Domain Name (FQDN), as well as credentials for authentication of the user terminal emulation application 110 (e.g., public key of user terminal emulation application 110).
- FQDN Fully Qualified Domain Name
- the user terminal emulation application 110 may have been configured with the credentials of the user tag 101 (e.g., public key of user tag 101).
- the EAP authenticator 400 identifies the user terminal emulation application 110 based on the realm part of the identifier and forwards the request to the user terminal emulation application 110, which is here acting as the EAP server (referred to as "EAP server/user terminal emulation application 110" for brevity).
- the EAP authenticator 400 first establishes 506a secure connection between itself and the EAP server/user terminal emulation application 110.
- the EAP messages are passed over that secure channel between authenticator and EAP server/user terminal emulation application 110.
- the secure connection may be a Transport Layer Security (TLS) session.
- TLS Transport Layer Security
- the EAP Authenticator 400 knows it needs to talk with the EAP server/user terminal emulation application 110 based on the identity (realm part of identity) received 506b in EAP Identity Response. It also knows it needs to have a secure channel with the EAP server/user terminal emulation application 110.
- the EAP Authenticator 400 creates the secure session and then forwards 506b the Identity response to the EAP server/user terminal emulation application 110.
- the communication between the EAP Authenticator 400 and the EAP server/user terminal emulation application 110 function of the user terminal emulation application 110, while using EAP messages, may be carried by DIAMETER or RADIUS protocol.
- the user terminal emulation application/EAP server 110 and the user tag 101 will perform an EAP exchange 507 to authenticate the user tag 101 to the user terminal emulation application 110.
- the EAP server/user terminal emulation application 110 may also be authenticated to the user tag 101. The authentication may require multiple messages to be exchanged between the user tag 101 and the EAP server/user terminal emulation application 110 via the EAP authenticator 400.
- the EAP server/user terminal emulation application 110 can send 508 a final EAP SUCCESS message to the EAP authenticator 400.
- the EAP SUCCESS message also carries the master key, e.g., pairwise master key (PMK), and the session-ID.
- the PMK key is the master key for the session-ID, the PMK key can be used to derive more keys for lODs, e.g., IOD X, added to this session.
- the user tag 101 at this stage may derive the master key, e.g., PMK key, but it will not (necessarily) be used by the user tag 101.
- the master key can be used if the user wants to authorize additional IOD, e.g., IOD X, assuming the user is more in control of which lODs are added to the session by having to actively (possibly even physically) add more lODs to the EAP server/user terminal emulation application 110.
- the authenticator generates 509 an IOD specific key K iodA from the received keying material to be used for streaming data between user terminal emulation application 110 and the target IOD A.
- Generating the IOD specific key K iodA may include using the received keying material as is, or may include performing a computational operation on the received key material such as by hashing a concatenation of the keying material and the identifier of the target IOD A and/or using another key derivation function (KDF) based on PMK and IOD specific information.
- KDF key derivation function
- the EAP authenticator 400 provides 510 the IOD specific key K iodA to the IOD A, together with the session-ID exported by the EAP server/user terminal emulation application 110 and the address (e.g. FQDN) of the user terminal emulation application 110.
- the IOD A can use the received data to establish 511 a secure channel (e.g., TLS) between itself and the user terminal emulation application 110.
- a secure channel e.g., TLS
- IOD A the first IOD specific key
- IOD A the EAP server/user terminal emulation application 110
- session-ID an identifier for the EAP server/user terminal emulation application 110 can use to lookup the context from where it can derive the corresponding K iodA.
- IOD A could re-use the TLS session established in operation 506.
- a more scalable solution is enabled by using a uniform operation for all IODS connecting to the EAP server/user terminal emulation application 110 so as to not have special cases in an implementation.
- the IOD A can use the session-ID to indicate to the EAP server/user terminal emulation application 110 the session /keying material/authentication context to which the connection request relates.
- the EAP server/user terminal emulation application 110 locates context and associate keying material based on received session-ID.
- the IOD A is to connect with the EAP server/user terminal emulation application 110 using a secure channel so the EAP server/user terminal emulation application 110 can stream or receive data from the IOD A, such that the IOD A needs to have keying material which it can use to establish a secure connection and the user terminal emulation application 110 needs to verify that the IOD A is authorized to send data.
- the session ID that was sent in operation 508 is what the IOD A can send to the EAP server/user terminal emulation application 110 so it knows the request relates to the authentication session that was setup for the session ID, and then locates its copy of the PMK key and any other keys negotiated during the authentication.
- the IOD uses its own identifier (e.g. IOD A) as a kind of username. The IOD A thereby tells the EAP server/user terminal emulation application 110 who it is.
- the EAP authenticator 400 has derived an IOD A specific key based on the PMK key, e.g., by concatenating the IOD A ID and the PMK key and then hashing the concatenated string, and may truncate the hash value to a defined length.
- the EAP server/user terminal emulation application 110 needs to know the ID for the IOD A so it can derive the same IOD A specific key provided to IOD A in, e.g., step 510.
- the EAP server/user terminal emulation application 110 uses the IOD identifier to derive IOD A specific key (K iodA).
- the IOD A uses the received key K iodA as the password/authentication credential to authenticate to the EAP server/user terminal emulation application 110.
- the IOD A and the EAP server/user terminal emulation application 110 share the same IOD A specific key which is used as a shared secret that the EAP server/user terminal emulation application 110 and IOD A use to authenticate each other and establish a secure channel therebetween.
- the EAP server/user terminal emulation application 110 verifies, via PSK-based authentication, that the IOD indeed possesses a valid session key (K_iodA) and is thus authorized to connect to the EAP server/user terminal emulation application 110 and exchange data with it.
- K_iodA session key
- the IOD A after successful authentication, can indicate 512 its UI capabilities (e.g., display, speaker, microphone, etc.) to the user terminal emulation application 110, which based on the UI capabilities, can enable data streaming to and/or from the IOD A.
- the user may have defined policies to the EAP server/user terminal emulation application 110 regarding what operations can be enabled automatically (e.g., streaming video to a display device for the communication service) and what operations requires explicit user consent before performing (e.g., to enable microphone use for the communication service)
- the EAP server/user terminal emulation application 110 may operate to interact 513 with the IODH 212 regarding other lODs in the vicinity of the user which have UI capabilities that can be used to provide the communication service to the user. These operations can provide the EAP server/user terminal emulation application 110 information about what UI and/or I/O capabilities could be available to the user for the communication service. Whether the EAP server/user terminal emulation application 110 may operate to interact 513 with the I0DH 212 regarding other IODS may depend on which service and/or application is running in the EAP server/user terminal emulation application 110.
- This communication may be performed via an entity of the IOD domain 410 acting as an EAP authenticator 400 towards the EAP server/user terminal emulation application 110, and thereby the already established secure channel could be re-used.
- the EAP authenticator 400 may generate an IODH 212 specific session key (similar to what was performed for IOD A) and provide IODH 212 with all relevant info (e.g., K_iodh, session-ID, EAP server/user terminal emulation application 110 FQDN) for securely communicating with the user terminal emulation application 110.
- the IODH 212 may itself be the EAP authenticator 400, in which case much of the “communication” between authenticator and IODH 212 is simplified, such as where the PMK is used directly by the IODH 212 to securely communicate with user terminal emulation application 110, or the already established secure session is re-used.
- the communication may include the IODH 212 telling the user terminal emulation application 110 about which IODs are available to the user, and may include the EAP server/user terminal emulation application 110 requesting certain hardware resources from the IODH 212.
- the user terminal emulation server 100 is generally configured to provide a communication service through I/O user devices 130, and includes at least one processor 800 ( Figure 8) and at least one memory 820 ( Figure 8) storing program code that is executable by the at least one processor to perform operations.
- the operations include to establish 405 ( Figure 4) and 511 ( Figure 5) a secure channel connection with a first I/O user device (130) using a session identifier and an identifier associated with the first I/O user device to determine a first I/O user device specific key generated from a master key, where the first I/O user device specific key and the session identifier being used for secure communication of messages with the first I/O user device.
- the operations by the EAP server/user terminal emulation application 110 can further include receiving 506b an EAP response through a communication channel with the EAP authenticator 400, where the EAP response contains an identifier of a user tag 101.
- the operations communicate 507 with the user tag 101 to perform an EAP exchange for authentication and establish the master key, and send 508 an EAP success message to the EAP authenticator 400, where the EAP success message contains the master key and a session identifier associated with the master key.
- the operations perform the receiving 405 of the indication of the I/O user interface capability of the first I/O user device 130 and the communicating 405 with the first I/O user device 130 to use the I/O user interface capability to provide at least part of the communication service for the user.
- the operations by the user terminal emulation server 100 can further include to store in the database 120 (Figure 1) the identifier of the user tag allowed to access the communication service, a network address of the first I/O user device based on communications with the first I/O user device, the first I/O user device specific key, and the indication of the I/O user interface capability of the first I/O user device 130.
- the user terminal emulation server 100 (i.e., via the EAP server/user terminal emulation application 110) stores the session identifier and the PMK.
- the I/O user device connects to the user terminal emulation server 100
- the I/O user device provides its identifier to the user terminal emulation server 100 as part of the connection establishment procedure.
- the user terminal emulation server 100 can now generate the I/O user device specific key.
- the user terminal emulation server 100 can authenticate the I/O user device, and after (or as part of authentication) which can establish the secure channel between the two.
- the operations to establish 405 the secure channel connection with the first I/O user device 130 using the session identifier and the identifier associated with the first I/O user device to determine the first I/O user device specific key generated from the master key can include to receive a secure channel connection request which includes the identifier of the first I/O user device 130 and the session identifier, and initiate the determination of the first I/O user device specific key based on the master key.
- the operations store the first I/O user device specific key in the database 120 with an association to the session identifier.
- the EAP authenticator 400 can include at least one processor 930 ( Figure 9), at least one memory 940 ( Figure 9) storing program code that is executable by the at least one processor to perform operations.
- the operations include receiving 505, from the first I/O user device 130, an EAP response which contains the identifier of the user tag containing an address of a user terminal emulation application 110 hosted by a user terminal emulation server 100.
- the EAP authenticator 400 may obtain 513 an identifier of a second I/O user device 130 that is proximately located to the first I/O user device 130 and has an I/O user interface capability that satisfies a rule for being combinable with the I/O user interface capability of the first I/O user device 130 to provide a communication service, and generate 514b, based on the master key, a second I/O user device specific key.
- the EAP authenticator 400 may then send 510 to the second I/O user device 130, e.g., via the IODH 212, the second I/O user device specific key, the session identifier, and the address for the user terminal emulation application 110.
- the operations receive 510 from the authenticator 400 a message comprising a first I/O user device specific key for the first I/O user device 130, a session identifier, and the address for the user terminal emulation application 110.
- the operations establish 511 a secure channel connection with the user terminal emulation application 110 using the first I/O user device specific key and the session identifier received from the authenticator 400.
- the operations send 512 an indication of an I/O user interface capability of the first I/O user device to the user terminal emulation application 110 through the secure channel connection.
- the operations communicate 512 with the user terminal emulation server 100 to use the I/O user interface capability of the first I/O user device 130 to provide at least part of a communication service to a user.
- the operation to establish 511 the secure channel connection with the user terminal emulation application 110 using the first I/O user device specific key and the session identifier received from the authenticator 400 may include sending the session identifier and an identifier of the first I/O user device to the user terminal emulation application 110 to indicate which I/O user device specific key is to be used to establish 511 the secure channel connection.
- the user tag 101 when the user tag 101 has authenticated itself towards the EAP server/user terminal emulation application 110 using EAP, the user tag 101 now also possesses the session keying material. Using this, the user tag 101 may generate authorization tokens for other IODS, e.g., IOD X, that the user tag 101 wants to add to the currently ongoing session.
- IOD X authorization tokens for other IODS
- the token may, e.g., be a signed piece of data contain things such as the session ID (so that the EAP server/user terminal emulation application 110 can map the token to the session), the newly selected I/O user devices identity (so that EAP server/user terminal emulation application 110 can verify that the correct I/O user device is using the token) and possible a lifetime of the token.
- the user tag 101 may provide such token (and a pointer to the EAP server/user terminal emulation application 110) to selected I/O user devices, which could then connect to the EAP server/user terminal emulation application 110 and present the token as proof of authorization by the user tag 101.
- the communication between user tag 101 and newly selected I/O user device may be similar to the initial registration to the IOD Domain 410 as described above starting with operation 501; the user tag 101 indicates it wants to connect the I/O user device IOD X to the EAP server/user terminal emulation application 110, but in a way that the I/O user device IOD X understands to reply with its identity.
- the I/O user device IOD X may use the help of the EAP Authenticator 400 to verify that the token is valid (e.g., the EAP Authenticator 400 possesses the same keying material and can verify the signature), or may blindly trust the token and start using it towards the EAP server/user terminal emulation application 110.
- a controller function in the IOD Domain 410 may be responsible for verifying that the connecting user terminal emulation application 110 is authorized to connect to the specified I/O user device. This could be a separate entity or a function of the IODH 212, which knows about all the I/O user devices in the domain 410. Alternatively, the controller function may be a part of the EAP Authenticator 400 or an Authentication and Key Management for Applications (AKMA) server, respectively (which in turn may be part of IODH 212, or be separate entities/functions).
- AKMA Authentication and Key Management for Applications
- the I/O user device Domain controller can operate to verify that the connecting identity (tag identity authenticated with EAP, user terminal emulation applications identity learnt during EAP while setting up secure channel to user terminal emulation applications 110, or user terminal emulation applications identity learnt during AKMA authentication) is authorized to access services in the IOD Domain 410, and the target I/O user device specifically. If there are access control policies which indicate that the user and/or user terminal emulation applications 110 is not allowed, the controller function can operate to terminate the authentication and may provide some error code indicating the user and/or service is not authorized.
- FIG. 6 illustrates a combined flowchart of operations and related data flows between a user tag, I/O user devices, a 3GPP key agreement function system (which may be an AKMA system or Generic Bootstrapping Architecture system), and a user terminal emulation server 110 in accordance with some embodiments of the present disclosure.
- Figure 10 illustrates a flowchart of operations that may be performed by the user terminal emulation server 110 in accordance with some embodiments.
- a user 601 selects a target I/O user device 602 and obtains therefrom an I/O user device identifier.
- the user may scan an I/O user device identifier, such as by scanning a QR code, reading an identifier from a sticker on the I/O user device and entering the identifier into the user's own input device, using NFC or RFID to read the identifier from the I/O user device, etc.
- the user tag and I/O user device may include circuitry configured to utilize one or more communication protocols to communicate information described herein
- the I/O user device identifier can include an identifier of the I/O user device (IOD-ID) and an IOD Domain identifier, e.g.
- the user terminal emulation application 110 generates mobile subscript on/SIM based credentials for use towards services by interacting 604 with the AKMA system 600 or a Generic Bootstrapping Architecture (GBA) function in the mobile operator.
- GBA Generic Bootstrapping Architecture
- a result of the interaction 604 is that the user terminal emulation application 110 and the mobile operator, e.g., AKMA or GBA function, has a shared master AKMA secret key or shared master GBA secret key, and has an identifier for the context or key.
- the user terminal emulation application 110 connects to the IOD Domain (e.g., AKMA Server) based on the I/O user device identifier (received in operation 603) realm part (e.g., myioddomain.com).
- the user terminal emulation application 110 derives an IOD Domain (e.g., AKMA Server 610) specific key from the AKMA or GBA master secret key.
- the user terminal emulation application 110 provides the AKMA or GBA context identifier to IOD Domain (e.g., AKMA Server 610).
- the IOD Domain (e.g., AKMA Server 610) uses the received AKMA or GBA context identifier to obtain IOD Domain specific AKMA or GBA key from the mobile operator AKMA or GBA function (e.g., AKMA Server 610), which may require pre-existing SLA/trust relationship between the IOD Domain and the mobile operator hosting the AKMA System.
- the user terminal emulation application 110 and the IOD Domain (e.g., AKMA Server 610) authenticate using the IOD Domain specific AKMA or GBA key, which may use a created secure session for further communication.
- the user terminal emulation application 110 tells 605 the IOD Domain which I/O user device (e.g., screenl23) it wants to interact with, e.g., based on the I/O user device identifier received in operation 603.
- the user terminal emulation application 110 also provides a pointer to itself, e.g., IP address, FQDN, etc.
- the IOD Domain 620 (e.g., AKMA Server 610) generates 606 an IOD specific AKMA or GBA session key from the IOD Domain specific AKMA or GBA key and the I/O user device identity, which may be similar to how in the EAP example above an IOD specific key is derived from EAP PMK and provides the IOD specific key to the I/O user device.
- the IOD Domain 620 (e.g., AKMA Server 610) may also provide a pointer to the user terminal emulation application 110, and an AKMA or GBA context identifier.
- the I/O user device connects 607 to the user terminal emulation application 110 based on received info.
- the I/O user device indicates the AKMA or GBA context identifier to the user terminal emulation application 110.
- the user terminal emulation application 110 can locate the AKMA or GBA context (e.g., AKMA or GBA master key).
- the I/O user device indicates its identity to the user terminal emulation application 110.
- the user terminal emulation application 110 can use the I/O user device identity together with the IOD Domain specific AKMA or GBA key to derive IOD specific AKMA or GBA session key.
- the user terminal emulation application 110 and the I/O user device can mutually authenticate using the I/O user device specific AKMA or GBA session key.
- the user terminal emulation application 110 and the I/O user device can use key to further establish a secure channel between themselves for streaming data for the communication service.
- the operations by the user terminal emulation server 100 may more generally include, with reference to Figure 9, to authenticate 603 an identifier of the user tag or a user, receive 603 the identifier of the first I/O user device.
- the operations generate 604 and 605 the first I/O user device specific key through communications with a 3GPP key agreement function.
- the key agreement function may include one of: an AKMA function; a GBA function; and a Battery Efficient Security for very low Throughput Machine Type Communication function.
- the operation to generate 604 and 605 the first I/O user device specific key through communication with the key agreement function may include to generate the first I/O user device specific key based on processing the master key derived through the key agreement function.
- the operations may further include communicating with the 3 GPP key agreement function, e.g., AKMA function, hosted in a mobile operator system to generate a shared secret between the user terminal emulation server 100 and the key agreement function.
- the 3 GPP key agreement function e.g., AKMA function
- Example I/O user device, user terminal emulation server, EAP authenticator or AKMA server, user tag, and IODH are now discussed below.
- FIG. 7 is a block diagram of hardware circuit components of an I/O user device 130 which are configured to operate in accordance with some embodiments.
- the I/O user device 130 can include a wired/wireless network interface circuit 702, a near field communication circuit 720, at least one processor circuit 700 (processor), and at least one memory circuit 710 (memory).
- the processor 700 is connected to communicate with the other components.
- the memory 710 stores program code (e.g., user terminal emulation application(s) 110) that is executed by the processor 700 to perform operations disclosed herein.
- the processor 700 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks.
- the processor 700 is configured to execute the program code in the memory 710, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device.
- the I/O user device 130 can include one or more UI component devices, including without limitation, a microphone 740, a speaker 750, a camera 730, a display device 760, and a user input interface 770.
- Figure 8 is a block diagram of hardware circuit components of a user terminal emulation server 100 and/or an I/O device handler (IODH) 212 which are configured to operate in accordance with some embodiments.
- the user terminal emulation server 100 and IODH 212 may reside on the same physical computing platform or may reside on different physical computing platforms which are communicatively networked together.
- the user terminal emulation server 100 and/or IODH 212 can include a wired/wireless network interface circuit 850, a database 120 (e.g., any one or more of a listing I/O user devices, UI capabilities of the I/O user devices, communication protocols used to communicate with the I/O user devices, known proximities to user identifiers, identifiers of user tags, I/O user device specific keys, session identifiers, etc.), a display device 830, a user input interface 840 (e.g., keyboard or touch sensitive display), at least one processor circuit 800 (processor), and at least one memory circuit 1220 (memory).
- the processor 800 is connected to communicate with the other components.
- the memory 1220 stores instructions that are executed by the processor 800 to perform operations disclosed herein.
- the processor 800 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks.
- the processor 800 is configured to execute computer program instructions in the memory 1220, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a user terminal emulation server and/or an IODH.
- Figure 9 illustrates a block diagram of hardware circuit components of an EAP authenticator 400 or AKMA server 610 that are configured to operate in accordance with some embodiments of the present disclosure.
- the components can include a wired/wireless network interface circuit 950, a display device 960, a user input interface 970 (e.g., keyboard or touch sensitive display), at least one processor circuit 930 (processor), and at least one memory circuit 940 (memory).
- the processor 930 is connected to communicate with the other components.
- the memory 940 stores program instructions that are executed by the processor 930 to perform operations disclosed herein.
- the processor 930 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks.
- the processor 930 is configured to execute computer program instructions in the memory 940, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device.
- Figure 10 illustrates a block diagram of hardware circuit components of a core network node 1000 that are configured to operate in accordance with some embodiments of the present disclosure.
- the components can include a wired/wireless network interface circuit 1030, at least one processor circuit 1010 (processor), and at least one memory circuit 1020 (memory).
- the processor 1010 is connected to communicate with the other components.
- the memory 1020 stores program instructions that are executed by the processor 1000 to perform operations disclosed herein.
- the processor 1010 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks.
- the processor 1010 is configured to execute computer program instructions in the memory 1020, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a core network node.
- FIG. 11 illustrates a block diagram of hardware circuit components of a radio network node 1100 that are configured to operate in accordance with some embodiments of the present disclosure.
- the radio network node 1100 may correspond to, without limitation, an eNB, gNB, other 3GPP base station, WiFi access point, etc.
- the components can include a wired network interface circuit 1140, a wireless network interface circuit 1130, at least one processor circuit 1110 (processor), and at least one memory circuit 1120 (memory).
- the processor 1110 is connected to communicate with the other components.
- the memory 1120 stores program instructions that are executed by the processor 1110 to perform operations disclosed herein.
- the processor 1110 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks.
- the processor 1110 is configured to execute computer program instructions in the memory 1120, described below as anon-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a radio network node.
- communication services can simultaneously be provided to numerous people roaming in a disaggregated device environment of different types of I/O user devices where some of which can be spaced apart great distances and some of which can be proximately located to each other. It is possible that two or more users can be simultaneously engaged in separate communication services using I/O user devices which are proximately located to one another and/or using different I/O user interface functions on the same I/O user device. A consequence is that one user may overhear or observe information communicated from/to another user(s). Another consequence is there is a risk that a microphone, camera, or other UI capability of one of the proximately located I/O user devices may be undesirably used by anon-local person to monitor such communications by other users.
- Various operational embodiments are therefore directed to monitoring and enforcing privacy of communication services which are provided through user terminal emulation involving I/O user devices.
- the first scenario is discussed regarding the flowchart of Figure 13, which illustrates operations that can be performed by the IODH 212 to monitor and control privacy of communications through I/O user devices in accordance with some embodiments of the present disclosure.
- the second scenario is discussed regarding the flowchart of Figure 14, which illustrates other operations that can be performed by the IODH 212 to monitor and control privacy of communications through I/O user devices in accordance with some other embodiments of the present disclosure.
- Session privacy requirements may be specified by users, by communication service functions 140 (e.g., application endpoints which are communicating with a user) for a type of communication service, by network entities 150, etc.
- the session privacy requirements may be determined for a communication session based on looking-up in the database 120 the session privacy requirement corresponding to a communication session for an identified type of communication service.
- the session privacy requirement for a session may be static while the session is ongoing or may be adjusted responsive to changes in the type of data being communicated through the session, e.g., video, audio, text, etc., and/or may be adjusted responsive to privacy indications transmitted with the data.
- transmission of data from a highly confidential file through the session may trigger an increase in the session privacy requirement.
- a videoconference meeting may have different privacy requirements tied to different parts of a meeting agenda, and where the privacy requirements can be indicated by signaling communicated from the videoconference service of the communication service function 140.
- a meeting participant may also define or trigger a privacy requirement to be applied.
- the IODH 212 determines that the privacy requirement satisfies a rule, e.g., above a certain threshold, the IODH 212 determines a set of privacy-affecting UI capabilities of I/O user devices which are proximately located to one or more I/O user devices which are being used or would be used to provide user terminal emulation capabilities for a communication service.
- a rule e.g., above a certain threshold
- the IODH 212 can proceed with allowing UI capabilities of one of the I/O user devices or another proximately located I/O user device to be used for the communication service.
- the IODH 212 may notify a user who is seeking access to the communication service of the non-compliant environment, and may query the user to provide authorization to proceed with establishing the communication service (via an UI output capability of one of the presently allocated or to-be allocated I/O user devices).
- the operations can be repetitively performed to dynamically identify changes in the privacy-affecting UI capabilities and responsively handle as described herein.
- the IODH 212 may check if both users are members of a relevant trust group(s) for the current content of one or both of the users’ sessions.
- a trust group may be defined as a group of users having the same access right to certain types of data, e.g., audio/ video media content.
- any non-allocated input capabilities are blocked from being allocated to users who are not present in the relevant trust group(s) for as long as the privacy conflicting I/O capability is allocated or until a timer reaches a threshold (or expires) since the privacy conflict was identified.
- Embodiments disclosed herein may be used for handling privacy affecting input capabilities, privacy affecting output capabilities, and combined privacy affecting input and output capabilities.
- the type of capabilities which are included in the set of privacyaffecting UI capabilities, and thereby affected by the privacy requirements may be determined by the IODH 212 based on the type of data communicated through a session. For example, if the data does not include audio, then microphone capabilities may not be excluded from the set of privacy-affecting UI capabilities.
- a second user is interfacing with the I/O UI capabilities of a second I/O user device to use a second communication service provided by a second communication service function through user terminal emulation via the second I/O user device.
- the IODH 212 receives a request for a communication session between a first I/O user device and a first communication service function.
- the IODH 212 determines 1302 whether UI capabilities of the first I/O user device do not satisfy a privacy-proximity rule defined for the second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O UI capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device.
- the IODH 212 controls 1304 whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy-proximity rule.
- the privacy-proximity rule is tied to the first user (first I/O user device) who can be selectively blocked from obtaining a communication service using user terminal emulation depending upon whether the second I/O user device satisfies the privacy-proximity rule associated with the first user (first I/O user device).
- the second user is interfacing with the I/O UI capabilities of the second I/O user device to use a second communication service provided by a second communication service function through user terminal emulation via the second I/O user device.
- the IODH 212 receives a request for a communication session between a first I/O user device and a first communication service function.
- the IODH 212 controls 1404 whether the communication session is setup and/or a communication capability of the communication session is allowed (i. e. , using one of the privacy-affecting I/O UI capabilities) between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule.
- the privacy requirements of one or both sessions may be modified over time based on trustworthiness of the respective users, which privacy-affecting I/O UI capabilities of the I/O user devices are being used for communication session(s), level of secrecy of the type of data being communicated through the respective session, which I/O user devices remain proximately located to the user (user tag), etc.
- Sessions may have different assigned priorities (e.g., determined from the associated privacy requirements and / or user identities associated with each session), and the priorities may be used to determine which session is blocked or terminated if the two sessions should not be simultaneously conducted as determined from their session privacy requirements.
- an open session i.e. no privacy restrictions in the session privacy requirement
- This privacy conflict may cause the IODH 212 to determine that a session through the first I/O user device should not be set up or should be terminated, or that a privacy-affecting I/O UI capability should be blocked from being setup for use or if already in-use should be disabled from being used.
- the IODH 212 will maintain a context for the authenticated UT 101, and serve any ongoing sessions. At some point if the UT 101 does not have ongoing sessions, the IODH 212 might decide to close the context. Alternatively, it might be that the IODH 212 maintains the context for as long as any of its IODS "sees/hears" the UT 101, e.g. hearing the Bluetooth beacon, which might occur until the user leaves the spatial area covered by the IOD domain.
- the I0DH 212 may determine that none of the privacy-affecting UI capabilities of IODS are allocated to an untrusted user or, in some embodiments, allocated to any other user. In a further embodiment, the IODH 212 determines if the other user is in a relevant trust group. The IODH 212 can perform further determinations based on whether any allocated UI capability of one of the IODs is in the set of privacy-affecting UI capabilities, such as determining whether that IOD is allocated to a trusted or untrusted user and/or other determinations as described further below.
- step 1207 if the result of the privacy determination (also 1302 in Fig. 13 and 1402 in Fig. 14) is not OK (NOK), e.g., does not satisfy the privacy-proximity rule, the IODH 212 may responsively not allocate the privacy-affecting UI capability (e.g., microphone, camera, etc.) to the session.
- NOK may result if a privacy-affecting UI capability is determined to be allocated to an untrusted user.
- the IODH 212 may inform the user terminal emulation server 110 and/or IOD-X that the privacy-affecting UI capability should not be added to the session.
- the IODH 212 can inform the user via an IOD and wait for receipt of a user authorization indication.
- An uncertain privacy determination may result when one of the proximate IODS is unresponsive and the allocation status therefore cannot be determined, and other scenarios resulting in uncertainty are discussed below.
- the IODH 212 can provide uncertainty information to the user either directly (e.g. IODH provides information to user directly via IOD(s) part of user’s session) or to the user terminal emulation server 110, which then provides the information to the user via current session. If user indicates authorization to proceed, then the IODH 212 may proceed according to step 1209 where the check is OK. Otherwise, if no authorization is received, the IODH 212 may proceed as NOK according to step 1207.
- These operations when performed in the scenario of Figures 13 and 14 can include the IODH 212 determining that the other I/O user device (e.g., IOD-Y and/or IOD-Z) satisfies the privacy-proximity rule responsive to receiving a message from the other I/O user device indicating that the other I/O user device received a beacon signal from a user tag of a user owning the communication session.
- the other I/O user device e.g., IOD-Y and/or IOD-Z
- the IODH 212 may determine that the other I/O user device is proximately located to the first I/O user device (IOD-X) based on identifying the other I/O user device as a member of a table (stored in at least one memory of the IODH 212, the server 110, or elsewhere) that identifies which I/O user devices of the set are proximately located to the first I/O user device (IOD-X).
- the operation by the IODH 212 to determine whether the other I/O user device (e.g., I0D-Y and/or IOD-Z) is proximately located to the first I/O user device (IOD-X) can include to obtain spatial coordinates of the I/O user devices, and determine whether the spatial coordinates of any of the other I/O user devices satisfy a rule relative to the spatial coordinates of the first I/O user device.
- the other I/O user device e.g., I0D-Y and/or IOD-Z
- IOD-X The operation by the IODH 212 to determine whether the other I/O user device (e.g., I0D-Y and/or IOD-Z) is proximately located to the first I/O user device (IOD-X) can include to obtain spatial coordinates of the I/O user devices, and determine whether the spatial coordinates of any of the other I/O user devices satisfy a rule relative to the spatial coordinates of the first I/O user device.
- the privacy determination can be performed based on individual privacy-affecting I/O user interface capabilities of the I/O user devices.
- the IODH 212 may determine the privacy-affecting I/O user interface capabilities of the other I/O user device (e.g., I0D-Y and/or IOD-Z), and determine whether any of those privacyaffecting I/O user interface capabilities violate the session privacy requirement for the communication session with the first I/O user device (IOD-X).
- the IODH 212 may operate to continuously (e.g., periodically or responsive to events) monitor to ensure that the same privacy situation that was determined in step 1204 continues. For this reason, the IODH 212 may block all UI capabilities in the set of privacy-affecting UI capabilities from being allocated by an untrusted user. In some embodiments, the IODH 212 may further start a timer at this point, to determine when a time limit for when the user is no longer allowed to block capabilities, has been reached. The timer and associated operations are discussed further below.
- the privacy requirement is determined based on the classification of data communicated (e.g., media, source of data (e.g., microphone, video, ...), the confidentiality level of data communicated, and/or another defined characteristic of the data and/or the user who is involved in the communication session.
- the IODH 212 can operate to continuously determine whether a second user of the second IOD (e.g., IOD-Y) has access rights to data having a different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140.
- the IODH 212 can respond to determining that the second user who is attempting to setup a session or is involved in an ongoing session through a proximately located second IOD (e.g., IOD-Y) does not have the access rights to access the data having the different classification, by blocking an I/O user interface capability of the second IOD (e.g., IOD-Y) from being accessed by the second user.
- a proximately located second IOD e.g., IOD-Y
- the IODH 212 can operate to continuously determine whether a second user of the second IOD (e.g., IOD-Y) has access rights to the data having the different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140.
- a second user of the second IOD e.g., IOD-Y
- the IODH 212 can respond to determining that the second user does not have the access rights to access the data having the different classification, and responsively send a message to the communication service function 140 instructing it to block sending of the data having the different classification to a first user of the first IOD (e.g., IOD-X).
- a first user of the first IOD e.g., IOD-X
- the IODH 212 may rely on priorities associated with the first and second users to determine who or what to block I/O user interface capability(ies) from being accessed by the first and/or second users. For example, if the first user has priority over the second user (premium user, first to allocate, etc.), the IODH 212 may block the second user to protect privacy of communications with the first user.
- a corresponding operation by the IODH 212 can compare priority properties associated with the first and the second users when determining whether the second user of the second IOD (e.g., IOD-Y) has access rights to data having the different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140 or, vice versa, whether the first user of the first IOD (e.g., IOD-X) has access rights to data having the different classification that is presently communicated through the communication session between the second IOD (e.g., IOD-Y) and the communication service function 140.
- the second IOD e.g., IOD-Y
- Fairness of access to the system may be ensured by the IODH 212 managing one or more timers which are used to limit how long a user can be blocked for obtaining access to a service via the user terminal emulation operations.
- the IODH 212 may start a timer responsive to sending 1207 of a message to block.
- the IODH 212 can re-evaluate the determination of whether the second user has access rights and perform one of a) send another message to the communication service function 140 instructing the cease blocking of the sending of the data having the different classification to the first user of the first IOD (e.g., IOD-X) and/or cease blocking use of privacy-affecting UI capability for a service, and notify the first user, or b) allow the block to be continued.
- the first IOD e.g., IOD-X
- the IODH 212 can re-evaluate the determination of whether the second user has access rights and perform one of a) send another message to the communication service function 140 instructing the cease blocking of the sending of the data having the different classification to the first user of the first IOD (e.g., IOD-X) and/or cease blocking use of privacy-affecting UI capability for a service, and notify the first user, or b) allow the block to be continued.
- IOD e.g., IOD-X
- step 1210 when session privacy requirements are decreased or when the user deallocates the UI capability, an earlier block is removed.
- the privacy requirements may be decreased based on the IODH 212 obtaining information that the current media session now contains less sensitive content, e.g., user closing a document with higher privacy class “company confidential, restricted access” and opening another document with lower privacy class “company internal”.
- Figures 12A and 12B are mostly directed to the scenario where a UT 101 is first connected to the user terminal emulation server 110 and then lODs are added to the session.
- the IODH 212 has learned the privacy requirements for the session and can use them when adding new lODs or privacy-affecting UI capabilities to the session, such as through the following steps.
- the UT 101 enters proximity of a new IOD, or UT 101 signals the need for a new UI capability to be added to the session.
- IODH 212 learns this based on the beacons transmitted by UT 101 and reported by lODs in proximity.
- IODH 212 verifies whether the new IOD or UI capability can be added to the session based on the known privacy requirements. If IOD or UI capability can be added without violating the privacy requirements, the IODH 212 adds the IOD or UI capability to the session.
- the privacy requirements can also be changed if the user, for example, launches a new application that has different privacy requirements than the current session.
- the user terminal emulation server 110 can operate to indicate the new privacy requirements to the IODH 212 before allowing the application to launch.
- IODH 212 verifies that the current IOD or UI capabilities allocations (to the UT 101 and also to other UTs in proximity) fulfills the new privacy requirements. If the requirements are met, IODH 212 informs the user terminal emulation server 110 that required privacy is available, after which the user terminal emulation server 110 can launch the application and/or session.
- the privacy requirements for a session may be preconfigured. Preconfiguring of the privacy requirements may be based on information provided in a meeting invitation in a calendar application that indicates the privacy requirements for the meeting. The privacy requirements may be based on the entries in a phone book application which define per entry privacy requirements. Alternatively, the privacy requirements may be dynamically selected by the user for an ongoing session, such as by using the UI capabilities of one or more of the lODs connected to the media session to select the current privacy requirements. The user terminal emulation server 110 can then use these requirements for the ongoing and future sessions as described earlier.
- a user may utilize multiple IODS in a session where at least one of the IODS is owned by a different IODH than the others, e.g., where there exist multiple domains for IODs and where the IODs are controlled by different IODHS.
- the IODs may be co-owned by two or more IODHs, requiring the IODHs to exchange UI capability allocation and status information among each other.
- a further embodiment assumes that a first IODH has an API used for communicating with at least one second IODH which directly controls at least one IOD (e.g., IOD-Y) in vicinity of an IOD (e.g., IOD-X) that is directly controlled by the first IODH.
- the IOD (e.g., IOD-Y) in proximity may be co-owned by the first and second IODH or owned exclusively by the second IODH.
- the first IODH may store in a database information identifying the other IODs which are in vicinity of an IOD(s) controlled directly by the first IODH, but where the other IODs (“non-owned IODs”) are not directly controlled by the first IODH. Then, as a part of step 1206 of Figure 12A, the first IODH can request from the at least one second IODH the information regarding the current allocation of those non-owned IODs and status of their respective privacy -affecting UI capabilities.
- the UI capabilities of the non-owned IODs may be a part of the set of privacy-affecting UI capabilities obtained in step 1205.
- the first IODH may await a response from the second IODH regarding the current allocation of privacy -affecting UI capabilities in the second IOD domain before evaluating the privacy. If there exists IODs (co-)owned by a second IODH in proximity of the first IOD and either the allocation in the other IODH domain is not known or if the users of that domain cannot be mapped to trust groups (see next section), the check may be determined as “uncertain”.
- the IODH 212 has knowledge of trust groups, i.e., groups comprising two or more users where the members have the same right to view certain content.
- a user can belong to many different trust groups that are relevant for different types of data communicated through a session, e.g., media content from the communication service function 140.
- the identity of the second user is checked against the trust group that is relevant for the current type of data (e.g., media content) of the session. For example, if the first user’s media session contains a document with class "company confidential, restricted access”, the relevant trust group comprises other user’s with at least the same access rights to the document as the first user. In another example, some content may be identified as authorized to be accessed by anyone employed in a company, some other content may be identified as authorized to be accessed by only someone having the user’s role, and some other content may be identified as authorized for access by a set of the user’s colleagues. In these examples, the IODH 212 may make the first user a member of three different trust groups.
- the check 1206 determines that the second user is a part of the relevant trust group, the check returns OK 1209 (assuming no other privacy-affecting condition has been found). If the user is not a part of the relevant trust group, the check 1206 returns NOK 1207.
- the IODH 212 when determining whether any of the privacy-affecting UI capabilities of the second I/O user device (e.g., IOD- Y) violate the session privacy requirement for the communication session with the first I/O user device (e.g., IOD-X), can perform operation to determine whether a first user of the first I/O user device (e.g., IOD-X) and a second user of the second I/O user device (e.g., IOD-Y) are members of a trust group relevant for the current data classification for the respective user’s session.
- a first user of the first I/O user device e.g., IOD-X
- a second user of the second I/O user device e.g., IOD-Y
- the IODH 212 may responsively determine that none of the privacy-affecting UI capabilities of the second I/O user device (e.g., IOD-Y) violate the session privacy requirement for the communication session with the first I/O user device (e.g., IOD-X).
- the IODH 212 keeps track of the mobile IODs. If a mobile IOD enters proximity of the first IOD (e.g., IOD-X) and the UI capabilities are allocated to a user who is not a member of a relevant trust group, the IODH 212 may responsively inform that first user and may await receipt of the first user’s indicated authorization before letting the media session with the first user continue.
- the first IOD e.g., IOD-X
- the IODH 212 may responsively inform that first user and may await receipt of the first user’s indicated authorization before letting the media session with the first user continue.
- the IODH 212 may further use the IODs to monitor for the user’s position-indicating device (UT 101), even in a situation where the user does not have any UI capability presently allocated.
- the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof.
- the common abbreviation “e.g.”, which derives from the Latin phrase “exempli gratia” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item.
- the common abbreviation “i.e.”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.
- Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits.
- These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer And Data Communications (AREA)
Abstract
An input and/or output user device handler, IODH, (212) obtains a session privacy requirement for a communication session. Based on a request for the communication session between a first I/O user device (130) of a set of I/O user devices served by the IODH and a communication service function of a network entity, the IODH determines whether the first I/O user device does not satisfy a privacy-proximity rule defined for a second I/O user device based on location relative to the second I/O user device and based on having a privacy- affecting I/O user interface capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device. The IODH controls whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy-proximity rule.
Description
CONTROLLING PRIVACY OF COMMUNICATION SERVICES THROUGH I/O USER DEVICES PERFORMING USER TERMINAL EMULATION AS A CLOUD COMPUTING SERVICE
TECHNICAL FIELD
[0001] The present disclosure relates to providing communication services through user terminals of a wireless communications system.
BACKGROUND
[0002] The market for user terminals has been driven by the quest to provide users with increasingly advanced communication and other operational features within the constraints of a portable handheld form factor. The development requirements for user terminals are increasingly complex as designers seek to integrate a greater variety of user interfaces and advanced operational features within a portable handheld form factor. Advancements in operational features have required more highly integrated and faster processing circuits with greater circuit densities, which becomes more difficult under constraints on costs and power consumption.
[0003] This all-inclusive feature-rich approach for user terminal development does not satisfy all of the myriad of differing desires held by consumers seeking solutions for the rapidly expanding variety of communication services. Moreover, the always-connected expectations of today's society obligate users to vigilantly keep their user terminals within reach or risk being unable to timely receive or initiate communication services.
[0004] Various solutions have been proposed for performing user terminal emulation as a cloud computing service. A user can be provided a communication service through one or more input and/or output (I/O) user devices which are proximately located to the user and connected to a cloud-based user terminal emulator. Which I/O user devices are selectable for use to provide the service to the user can depend upon their user interface (UI) capabilities being able to satisfy requirements of the service. Example types of UI capabilities of I/O user devices include display devices, cameras, microphones, keyboards, speakers, sensors, etc. Proximity of I/O user devices to the user can be determined based on which I/O user devices can receive a beacon signal transmitted by a user tag being transported by the user.
[0005] It can become problematic from a privacy perspective to provide user terminal emulation services through I/O user devices. For example, a user may desire to conduct a videoconference to discuss company proprietary information using UI capabilities of one or
more I/O user devices, but privacy may not be maintained if other users begin using the same or other proximately located I/O user devices.
SUMMARY
[0006] Some embodiments disclosed herein are directed to an input and/or output (I/O) user device handler (IODH) that includes at least one processor and at least one memory storing program code that is executable by the at least one processor to perform operations. The operations include obtaining a session privacy requirement for a communication session. Based on a request for the communication session between a first I/O user device of a set of I/O user devices served by the IODH and a communication service function of a network entity, the operations determine whether the first I/O user device does not satisfy a privacyproximity rule defined for a second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O user interface capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device. The operations control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy -proximity rule.
[0007] Some potential advantages of this and related embodiments include that a user can receive and initiate communication services without the necessity of a traditional all- inclusive feature-rich user terminal. The operations can emulate a user terminal using the networked second I/O user device that is proximately located to a user tag transported by a second user, and where the second I/O user device individually or combined with one or more other I/O user device(s) has/have user interface (UI) capabilities to provide an I/O user interface for the second user to interface with the communication service function (e.g., a user terminal emulation application of a user terminal emulation server) to provide a communication service. Operational embodiments disclosed herein enable monitoring and enforcing privacy of communication services provided through user terminal emulation are now discussed below, according to the privacy -proximity rule. The IODH can protect privacy of the second user's communications from being monitored (eavesdropped-on) by a first user who is requesting a session between the first I/O user device which is proximately located to the second I/O user device and having an I/O user interface capability (e.g., microphone, camera, etc.) that does not satisfy the session privacy requirement for the ongoing communication session of the second I/O user device.
[0008] Some related embodiments disclosed herein are directed to an IODH having at least one processor and at least one memory storing program code that is executable by the at least one processor to perform operations. The operations include obtaining a session privacy requirement for a communication session. Based on a request for the communication session between a first I/O user device of a set of I/O user devices served by the I/O user device handler and a communication service function of a network entity, the operations determine whether a second I/O user device of the set does not satisfy a privacy-proximity rule based on location relative to the first I/O user device and based on having an I/O user interface capability that does not satisfy the session privacy requirement for the communication session with the first I/O user device. The operations control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule.
[0009] Some potential advantages of this and related embodiments include that IODH can protect privacy of the first user's communications from being monitored (eavesdropped- on) by a second user who is using a second I/O user device. When the first user requests a communication session between the first I/O user device and the communication service function, the IODH can allow the communication session with the first I/O user device to be setup if the second I/O user device satisfies the privacy-proximity rule or otherwise may prevent setup of the requested communication session or allow setup but modify what privacy-affecting I/O user interface capabilities (e.g., microphone, camera, etc.) are allowed to be used through communication session by the first and/or second I/O user device.
[0010] Other IODHS and corresponding methods thereof will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such additional IODHs and methods be included within this description, be within the scope of the present inventive subject matter, and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented individually or combined in any way and/or combination.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011 ] Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying drawings. In the drawings:
[0012] Figure 1 illustrates a system with a user terminal emulation server that operationally integrates sets of I/O user devices that are proximately located to users to logically form virtualized user terminals providing communication services in accordance with some embodiments of the present disclosure;
[0013] Figure 2 illustrates a block diagram illustrating the user terminal emulation server communicating with various elements of a cellular system to provide communication services in accordance with some embodiments of the present disclosure;
[0014] Figure 3 illustrates a block diagram illustrating the user terminal emulation server communicating in a different manner with various elements of a cellular system to provide communication services in accordance with some other embodiments of the present disclosure;
[0015] Figures 4 and 5 illustrate combined flowcharts of operations and related data flows between a user tag, I/O user devices, an Extensible Authentication Protocol (EAP) authenticator, and a user terminal emulation server which may include an EAP server in accordance with some embodiments of the present disclosure;
[0016] Figure 6 illustrates a combined flowchart of operations and related data flows between a user tag, I/O user devices, a 3GPP key agreement function system, and a user terminal emulation server in accordance with some embodiments of the present disclosure; [0017] Figure 7 illustrates a block diagram of hardware circuit components of an I/O user device which are configured to operate in accordance with some embodiments;
[0018] Figure 8 illustrates a block diagram of hardware circuit components of a user terminal emulation server and/or an I/O device handler (IODH) that are configured to operate in accordance with some embodiments of the present disclosure;
[0019] Figure 9 illustrates a block diagram of hardware circuit components of an EAP authenticator or AKMA server that are configured to operate in accordance with some embodiments of the present disclosure;
[0020] Figure 10 illustrates a block diagram of hardware circuit components of a core network node that are configured to operate in accordance with some embodiments of the present disclosure;
[0021 ] Figure 11 illustrates a block diagram of hardware circuit components of a radio network node that are configured to operate in accordance with some embodiments of the present disclosure;
[0022] Figures 12A and 12B illustrate combined flowcharts of operations for monitoring and controlling privacy of communications through I/O user devices with communication service functions in accordance with some embodiments of the present disclosure;
[0023] Figure 13 illustrates a flowchart of operations that can be performed by a IODH to monitor and control privacy of communications through I/O user devices in accordance with some embodiments of the present disclosure; and
[0024] Figure 14 illustrates another flowchart of operations that can be performed by a IODH to monitor and control privacy of communications through I/O user devices in accordance with some other embodiments of the present disclosure.
DETAILED DESCRIPTION
[0025] Inventive concepts will now be described more fully hereinafter with reference to the accompanying drawings, in which examples of embodiments of inventive concepts are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of various present inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present or used in another embodiment.
[0026] Embodiments are directed to enabling a user to receive and initiate communication services without the necessity of a traditional all-inclusive feature-rich user terminal. Operations emulate a user terminal using a I/O user device that is proximately located to a user tag transported by a user, and where the I/O user device individually or combined with one or more other I/O user device(s) has/have user interface (UI) capabilities that are required or preferred to provide an I/O user interface for the user to interface with a communication service function (e.g., a videoconference service, media streaming service, etc. which interfaces with the I/O user device(s) through a user terminal emulation application of a user terminal emulation server) to provide a communication service. Further operations are directed to enabling monitoring and enforcing privacy of communication services provided through user terminal emulation, according to the privacy -proximity rule Operations can protect privacy of the user's communications from being monitored (eavesdropped-on) by another user.
[0027] Before discussing example operations for monitoring and enforcing various privacy mechanisms, example system components and operations are described for
performing user terminal emulation as a cloud computing service in accordance with some embodiments.
[0028] Dynamic allocation of I/O user device capabilities whenever and wherever the I/O user devices are in the proximity of a user enables efficient and flexible use of existing hardware, such as televisions, conference phones, laptops, surveillance cameras, connected household appliances, connected cars, etc., that is capable of providing necessary UI functionality to user during a communication service. The user thereby has reduced or no need to carry an expensive and all-inclusive user terminal, e.g., smart phone, that includes all necessary UI capabilities, display device, keyboard, speakers, etc. The user may instead carry a hardware device which operates to identify the user, referred to as a "UserTag" or "user tag", over a wireless or wired (e.g., smartcard reader) communication interface, such as a near field communication (NFC) interface, to one or more of the I/O user devices. Various embodiments disclosed herein may disrupt the traditional handset-centric mobile communication industry as the features and capabilities of what forms a user terminal are not constrained to the domain of mobile phone manufacturers. A user terminal emulation server can operate to provide a user terminal, which can also be referred to as a SoftUE or a user terminal emulation application that is run by the user terminal emulation server.
[0029] Figure 1 illustrates a system with a user terminal emulation server 100 that can use one or more I/O user devices 130 that is/are proximately located to users to logically emulate a user terminal providing a communication service in accordance with some embodiments of the present disclosure. The user terminal emulation server 100 may use the UI capability of a single I/O user device 130 or operationally integrate the UI capabilities of a set of the I/O user devices 130 to logically emulate a user terminal providing communication services in accordance with some embodiments of the present disclosure.
[0030] Referring to Figure 1, the user terminal emulation server 100 may be a cloud resource that is networked and remote from the I/O user devices 130, or may be more proximately located on a shared network with the I/O user devices 130. Alternatively, functionality of the user terminal emulation server 100 may be performed by an application hosted on one or more of the I/O user devices 130. The user terminal emulation server 100 is configured to communicate with the I/O user device(s) 130 proximately located to a user who can use the UI capabilities of the proximate I/O user device(s) 130 to be provided a communication service (e.g., voice call, videoconference, electronic mail, text messaging, extended reality meeting, etc.).
[0031] Users may carry a hardware tag, a.k.a. "UserTag", "user tag" or "UT", which is capable of transmitting a unique user identifier through a communications interface, such as a near-field communications interface (e.g., Bluetooth, BLE, NFC, RFID, etc., or combinations thereof), for receipt by one or more of the I/O user devices 130 which are proximately located to the user. One type of user tag can be a low-complexity standalone electronic device having limited operational capability for transmitting an identifier through a near-field communications interface and performing authentication operations such as described herein. Another type of user tag can have more operational capability (e.g., processing and memory hardware resources), such as a smartphone or smartwatch having cellular connectivity that transmits a cellular identity (e.g., from a SIM card, a USIM on a UICC, or an embedded SIM on an embedded UICC or on an integrated embedded UICC) and / or an application identity through a cellular interface or a near-field communications interface and is configured to perform authentication operations such as described herein. A user tag may be a device that does not require human interaction in order to interact with an I/O user device, and may lack a user interface. A user tag may be configured to interact with one or more types of Internet of things (loT) devices, such as a camera, sensor, actuator, or other electronic device having Internet or other wireless connectivity.
[0032] The user identifier may alternatively or additionally be operationally identified through biometrics operations performed by, e.g., one or more of the I/O user devices 130. The biometrics operations may include, without limitation, voice recognition, image/face recognition, eye recognition, fingerprint recognition, gait recognition, gesture recognition, or a combination thereof. The user identity may be determined based on credentials provided by the user when, e.g., logging into an application or account. The user identity may be provided by a cell phone using information from the subscription SIM and proximity of the cell phone to one or more of the I/O user devices 130 can be determined using the phone’s NFC capability.
[0033] A user identifier, a user tag identifier, and a user terminal emulation application 110 can be logically associated with each other in a database 120 during a user registration process or as part of another setup process. For example, during a user registration process a user may obtain an account login identifier (serving as the user identifier) that is registered in the database 120 as being associated with a user tag identifier for a physical user tag that has been provided to (e.g., purchased by) the user and being associated with a user terminal application 110 that emulates a user terminal having defined capabilities (e.g., a cell phone providing cellular and over-the-type voice-over-IP communication services).
[0034] The user terminal emulation server 100 may maintain in the database 120 network addresses of I/O user devices 130 and UI capabilities of the I/O user devices 130. Although the database 120 is illustrated as residing in the example server 100, in some other embodiments information described below as residing in the database 120 may alternatively or additionally be stored within an I/O user device handler (IODH) 212 in Figure 2 and/or the user terminal emulation applications 110. The capabilities of the I/O user devices 130 may be logically arranged in the database 120 based on the type of UI capability provided, e.g., display device, microphone, speaker, physical/virtual keyboard, and may be further arranged based on a quality of service provided by the UI capability.
[0035] The user terminal emulation server 100 may register a network address of one of the user terminal emulation applications 110 and an identity of a user with a network entity 150 providing communication services. The network entity 150 provides a communication service function 140 which may, for example, correspond to an over-the-top Voice Over Internet Protocol (VoIP) service, streaming media service (e.g., Netflix), social media service (e.g., Meta Platforms, Inc.), electronic mail service (e.g., Microsoft Outlook), online meeting service (e.g., Microsoft Teams), messaging service (e.g., Google Messenger), Internet browser service, a cellular communication service, extended reality meeting, etc. The user terminal emulation application 110 is executed by the user terminal emulation server 100. A user terminal emulation application 110 may run one or more applications that are normally run by a smart phone, such as a Netflix application, Meta (Facebook) application, Microsoft Teams application, Internet browser application, etc.
[0036] As illustrated in Figure 1, a different instantiation of the user terminal emulation application 110 may be hosted by the server 100 for each user who is to be provided communication services (i.e., illustrated user terminal emulation applications #1-#N corresponding to users 1-N). The user terminal emulation application 110 may perform registration of the user with the network entity 150 and setup of a communication service with a user responsive to communication requests.
[0037] When the communication service function 140 of the network entity 150 is a VoIP service, the operation to register the network address of the user terminal emulation application and the identity of the user with the network entity can include registering the network address of the user terminal emulation application 110 and the identity of the user with a network server of a VoIP communication service provider.
[0038] When the communication service function 140 of the network entity 150 is a cellular communication service, the operation to register the network address of the user
terminal emulation application and the identity of the user with the network entity can include registering the network address of the user terminal emulation application 110 and the identity of the user with a Home Subscriber Server (HSS) 211, Unified Data Management (UDM), or other network node of a core network operated by a cellular communication service provider.
[0039] The user terminal emulation server 100 may receive the registration messages from the I/O user devices using the Session Initiation Protocol (SlP)ZSession Description Protocol (SDP), where each of the registration messages identifies the network address and the UI capability of one of the I/O user devices. The communication request may be received from the network entity 150 using the SIP/SDP, and the operation to provide communication sessions between the user terminal emulation application 110 and each of the I/O user devices in the set, and between the user terminal emulation application 110 and the requesting user terminal may be performing using the SIP/SDP.
[0040] A registration message from an I/O user device can include, for example, an IP address and port number, MAC address, fully qualified domain name (FQDN), and/or another network address, and can further include information identifying the UI capability of the I/O user device. The I/O user device may power-up and responsively communicate the registration message to the user terminal emulation server 100.
[0041] The user terminal emulation server 100 receives a communication request from the network entity 150 for establishing a communication service between the user and a requesting user terminal, e.g., a cellular phone, computer with Microsoft Teams application, etc. Responsive to the communication request, the user terminal emulation server 100 identifies one or more of the I/O user devices 130, which may be registered in the database, that are proximately located to a location of the user and are determined, based on the UI capabilities identified by the database 120 for the set of I/O user devices and based on content of the communication request, to satisfy a capability rule for being individually usable or combinable to provide an I/O user interface for the user to interface with the user terminal emulation application 110 to provide the communication service. Although various operations are described above and elsewhere as being performed by the user terminal emulation server 100, it is to be understood that these and other operations are performed by the user terminal emulation server 100 executing one or more of the user terminal emulation applications 110 instantiated for a user.
[0042] The user terminal emulation server 100 provides one or more communication sessions between the user terminal emulation application 110 and the one or more I/O user
devices 130 and between the user terminal emulation application 110 and the requesting user terminal via the network entity 150. The communication request that is received by the user terminal emulation application 110 may contain an indication of a minimum UI capability that must be provided to the user during the communication service, such as: speaker only; combination of speaker and microphone; display only; combination of display device, speaker, and microphone; etc. A UI capability rule which can be used by the server 100 to determine whether a communication service can be provided and by which set of I/O user devices, may thereby be defined based on the minimum UI capability that is indicated by the communication request.
[0043] The user terminal emulation server 100 then routes communication traffic between at least one of the I/O user devices in the set and the requesting user terminal via the network entity 150. In some embodiments, for example, for each datatype (e.g., audio data, video data, text, etc.) that is received as communication traffic from the requesting user terminal, the user terminal emulation server 100 selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the database 120 for the one of the I/O user devices, and then routes the data of the data type toward the network address of the selected one of the I/O user devices.
[0044] As will be explained in further detail below, the server 100 may also combine data streams that are received from the I/O user devices in the set, and route the combined data streams towards the requesting user terminal, e.g., via the network entity 150.
[0045] The user terminal emulation server 100 (e.g., via the IODH 212 described below) may be responsible for tracking which I/O user devices are proximately located to a present location of the user. The server 100 can receive presence reports from individual ones of the I/O user devices containing their network address and an identifier of a user tag which is determined by the I/O user device to be proximately located thereto. For example, an I/O user device may read a user tag through an NFC communication interface and/or may perform other operations to detect presence of a user and to identify a user tag transported by the user. Responsive to the presence reports, the server 100 updates the database 120 to indicate which user tag identifiers are proximately located to which of the I/O user devices.
[0046] With further reference to the example system of Figure 1, a set of I/O user devices 130 has been determined by the IODH 212 (Fig. 2) to be proximately located to a location of a first user carrying UserTag#!, and further determined to have UI capabilities that are combinable to satisfy the UI capability rule for providing a combined I/O user interface for
the first user to use during a requested communication service. IODH 212 (Fig. 2) responsively uses that set of I/O user devices 130 to provide a combined I/O user interface for use by the first user during a communication service via Application #1 and network entity 150 between the first user and another user terminal.
[0047] Similarly, another set of I/O user devices 130 has been determined by the IODH 212 (Fig. 2) to be proximately located to a location of a second user carrying UserTag#2, and to further have UI capabilities that are combinable to satisfy the UI capability rule for providing a combined I/O user interface for the second user to use during a requested communication service. IODH 212 responsively uses that set of I/O user devices 130 to provide a combined I/O user interface for use by the second user during a communication service via Application #2 and network entity 150 between the second user and yet another user terminal.
[0048] Figure 1 also illustrates that another set of I/O user devices 130 is not proximately located to either UserTag#! or UserTag#2.
[0049] As explained above, the communication request which is requesting the establishment of communication service session ("communication session" or "session") with an identified user may be initiated by the network entity 150 using the network address of the user terminal emulation application and identity of the user which were earlier registered with the network entity 150. However, the communication request may additionally or alternatively be generated by one of the I/O user devices 130 responsive to a command received from a proximately located user. For example, a user tag may operate automatically or through action of the user to initiate communications with one of the I/O user devices 130. Alternatively, a user may operate a user interface provided by one of the I/O user devices 130 to initiate, e.g., a combined audio and video call with another user. An identity of the user tag may be sent with the communication request, or the identity of the user tag may be logically bound to the communication session between the I/O user device and the user terminal emulation server 100. The application 110 performs the identifying, providing, routing, selecting, and combining operations described above to set up and operate a communication service between the user and the other user via the network entity 150.
[0050] Further example systems and related operations will now be described to further illustrate how I/O user devices having different UI capabilities can be operationally used or combined to provide a combined UI that can be used by user to satisfy the communication requirements of a communication service.
[0051] Further illustrative operations are described regarding an example embodiment in which a speaker device is one of the I/O user devices 130 in the set capable of playing a received audio stream and a microphone device is one of the I/O user devices 130 in the set capable of sensing audio to provide a microphone stream. Operations by the user terminal emulation application include updating the database 120 based on content of registration messages from the speaker device and the microphone device to identify network addresses of the speaker device and the microphone device, and to identify UI capabilities of the speaker device as having a speaker capability and the microphone device as having a microphone capability. The speaker UI capabilities may identify a number of speakers provided, sound loudness capability, and/or other operational characteristics. The microphone UI capabilities may identify a number of microphones provided, sensitivity of the microphones, and/or other operational characteristics. The speaker device and the microphone device are each identified as belonging to the set of I/O user devices that are determined to be proximately located to the location of the user (e.g., UserTag#!) and are further determined, based on the UI capabilities identified by the database 120, to satisfy the UI capability rule for used individually or combined to provide a combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service. Based on determining that the speaker device and the microphone device satisfy the UI capability rule, further operations are performed to route a microphone stream received from the microphone device toward the requesting user terminal (e.g., via network entity 150). When an audio stream is received as communication traffic from the requesting user terminal the operations select the speaker device based on matching an audio characteristic of the audio stream to the speaker capability identified by the database 120 for the speaker device, and then route the audio stream toward the network address of the speaker device.
[0052] In some embodiments, the UI capabilities of devices may correspond to different UI functions and/or circuits provided by the same electronic device. The speaker device and the microphone device may correspond to a speaker function and circuit and to a microphone function and circuit, respectively, of a same smartphone, television, or other electronic device. The streams may then be routed to an application programming interface (API) of the targeted functions and/or circuits.
[0053] The example embodiment may include, when a display device is one of the I/O user devices in the set capable of displaying a received video stream, the operations update the database 120 based on content of registration messages to identify network addresses of
the display device, and to identify UI capabilities of the display device as having a display capability. The display UI capabilities may identify a screen display size, aspect ratio, pixel resolution, video frame rates supported, whether display device supports shared user support via split screen configuration, and/or other operational characteristics. The display device is also identified as among the set of I/O user devices that determined, based on the UI capabilities identified by the database 120, to satisfy the UI capability rule for being used individually or combined to provide the combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service. In an optional further embodiment, the set of I/O user devices is further selected based on each of the I/O user devices satisfying a rule for being proximately located to the location of the user. Based on determining that the speaker device, the display device, and the microphone device satisfy the UI capability rule, further operations respond to receipt of video stream as communication traffic from the requesting user terminal by selecting the display device based on matching a video characteristic of the video stream to the display capability identified by the database 120 for the display device, and then routing the video stream toward the network address of the display device.
[0054] In the example embodiment the operations for routing the audio stream and the video stream toward the network addresses (or APIs) of the speaker device and the display device, respectively, may include when audio data and video data are received within a same stream from the requesting user terminal through a first communication session: separating the audio data from the video data; routing the audio data toward the network address (or API) of the speaker device through a second communication session; and routing the video data toward the network address (or API) of the display device through the second communication session or a third communication session.
[0055] The example embodiment may include, when a camera device is one of the I/O user devices in the set capable of providing a camera stream, the operations update the database 120 based on content of a registration message to identify a network address of the camera device and to identify a UI capability of the camera device as having a camera capability. The camera UI capabilities may identify a camera pixel count, image quality, light sensitivity, and/or other operational characteristics. The camera device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user and is further determined, based on the UI capability identified by the database 120, to satisfy the UI capability rule for being used individually or combined with the other I/O user devices in the set to provide the combined I/O UI for the
user to interface with the user terminal emulation application 110 to provide the communication service. Based on determining that the camera device satisfies the UI capability rule, further operations are performed to route the camera stream received from the camera device toward the requesting user terminal, e.g., via the network entity 150.
[0056] The operations for routing the microphone stream received from the microphone device and the camera stream received from the camera device toward the requesting user terminal, can include: receiving the microphone stream from the microphone device through a first communication session; receiving the camera stream from the camera device through the first communication session or a second communication session; combining the microphone stream and camera stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity 150.
[0057] The example embodiment may include, when a keyboard device is one of the I/O user devices in the set capable of outputting key selection data responsive to key selections by a user among keys of the keyboard device, the operations can update the database 120 based on content of a registration message to identify a network address of the keyboard device and to identify a UI capability of the keyboard device as having a keyboard capability. The keyboard device capabilities may identify a number of keys (buttons) that are user selectable, indication of whether the keyboard is a physical keyboard or a touch sensitive input device, and/or other keyboard capabilities. The keyboard device is further identified as a member of the set of I/O user devices that are determined to be proximately located to the location of the user and is further determined, based on the UI capability identified by the database 120, to satisfy the UI capability rule for being used individually or combined with the other I/O user devices in the set to provide the combined I/O UI for the user to interface with the user terminal emulation application 110 to provide the communication service. Based on determining that the keyboard device satisfies the UI capability rule, further operations are performed to identify commands formed by the key selection data received from the keyboard and to perform operations that have been predefined as being triggered based on receipt of the identified commands.
[0058] The operations for routing the key selection data received from the keyboard device and microphone stream received from the microphone device, may include: receiving the key selection data from the keyboard device through a first communication session receiving the microphone stream from the microphone device through the first communication session or a second communication session; combining the key selection data
and the microphone stream in a combined stream; and routing the combined stream toward the requesting user terminal through a third communication session, e.g., via the network entity 150.
[0059] Figure 2 is a block diagram illustrating the user terminal emulation server 100 as an element of an operator service node 202 within a cellular system 200. Referring to Figure 2, the communication service function of the network entity 150 (Fig. 1) may be provided by the operator service node 202 or may be reached through external infrastructure 240, e.g., the Internet or private network. The server 100 may, for example, be implemented in the radio access network 220 to provide edge computing with faster responsiveness or may be implemented within another node of the cellular system 200. The user terminal emulation server 100 can include the IODH 212, a control function (CF) 214, the instantiated user terminal emulation applications 110, and a service gateway (GW) 216. A user terminal emulation application 110 may perform one or more user applications which are provided by a smart phone, such as a Netflix application, Meta (Facebook) application, Microsoft Teams application, Internet browser application, etc.
[0060] The user terminal emulation server 100, or the IODH 212 which can be part of the server 100, may perform operations to manage the I/O user devices, such as to handle maintenance of the database 120, perform registration of I/O user devices to be available for use by the user terminal emulation applications 110, and manage mobility through operations for setting up and performing handover of communication services through I/O user devices. For example, the user terminal emulation server 100 may operate to identify the IP address of a user terminal emulation application, e.g., which encapsulates a Microsoft Teams application, for a subscriber with a service provider, e.g., a Microsoft Teams server. In some other embodiments, the IODH 212 may be located outside the user terminal emulation server 100 in another network node of the system. The CF 214 may be responsible for assigning an IP address to each user terminal emulation application 110. The IP address to be assigned by the CF 214 may be received from the core network 210 functionality such as a PDN-GW. The service GW 216 may interconnect the user terminal emulation server 100 to a PSTN network, packet data network gateway of a 3GPP (3rd Generation Partnership Project) system, etc. The cellular system 200 can include a Core Network 210 having a Home Subscriber Server (HSS) 211, a Policy and Charging Roles Function (PCRF), gateway (GW) and Mobility Management Entity (MME) providing control signaling related to mobile terminal mobility and security for the radio access. The HSS 211 contains subscriber-related information and provides support functionality for user authentication and user access to the
system. The PCRF enables QoS control per data flow and radio bearer, by setting QoS rules for each data flow, based on operator set policies and subscriber information. The GW can include a Serving GW (S-GW) and a Packet Data Network GW (PDN-GW), where the S- GW interconnects the core network 210 with the radio access network 220 and routes incoming and outgoing packets for the I/O user devices 232 and/or 130 and the user terminals 230. The PDN-GW interconnects the core network 210 with external infrastructure 240, such as the Internet, and allocates IP-addresses and performs policy control and charging.
[0061] Some I/O user devices 232 having cellular communication capability can communicate via, e.g., eNBs or other radio access nodes of a Radio Access Network 220 with the operator service node 202 via the core network 210. In the system of Figure 2, the user terminal emulation server 100 may handle set up of a communication service between a selected set of the I/O user devices that are proximate to a user and a remote user terminal 230 (e.g., smart phone) via the cellular system 200.
[0062] Figure 3 is a block diagram illustrating the user terminal emulation server 100 communicating in a different manner with various elements of a cellular system 200, which may operate as the network entity 140 (Fig. 1), to provide communication services in accordance with some embodiments of the present disclosure. The system of Figure 3 differs from the system of Figure 2 by the user terminal emulation server 100 being an Internet service within external infrastructure 240 outside of the cellular system 200. In the system of Figure 3, the CF 214 may determine the IP address to be assigned to different ones of the user terminal emulation applications 110 based on signaling from the Internet service within the external infrastructure 240.
[0063] The above and other example operations will now be described in further detail in the context of two different example "use cases": 1) incoming call scenario; and 2) outgoing call scenario.
[0064] Use Case 1: Incoming call Scenario operations are now discussed below. [0065] This use case involves a user, with a user tag or other way of being identified, being proximately located to I/O user devices 130 having different UI capabilities when an incoming call is received by the user terminal emulation server. Although operations are explained below in the context of identifying a user through a physical user tag transported by the user, these operations are not limited thereto and may be used with any other way of identifying a user, such as by sensing biometric information that identifies the user and involving operational communications with the user tag transported by the user.
[0066] A user terminal emulation application 110 may be instantiated or otherwise activated responsive by an incoming call (service, session) targeting the user tag. The user terminal emulation application 110 can identify subscriptions associated with the user tag (e.g., registered in a user account) and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the incoming communication session. The user terminal emulation application 110 may ask the IODH 212 to identify which I/O user devices 130 are proximately located to the user tag, and may further ask the IODH 212 to determine or may determine itself whether the identified I/O user devices 130 are usable individually or combinable to satisfy the UI capabilities specified by the incoming communication session. The user terminal emulation application 110 and/or the IODH 212 may receive an ACK or NACK back on whether a sufficient set of I/O user devices 130 can be used to provide the communication service. If ACK, then the IODH 212 also sets the state of the I/O user devices 130 in the set to in-use to avoid another user terminal emulation application 110 attempting to utilize the same I/O user devices 130 as which are presently in use.
[0067] In case of NACK, the user terminal emulation application 110 and/or the IODH 212 can take different actions to setup a reduced UI capability communication service with the user depending on user settings, e.g., only allow sound-based communications instead of a combination of sound and video responsive to when no display device is presently available for use. An example of no display device being available may occur when the only display device that is proximately located to the user is presently being used by another user to receive information from another user terminal emulation application during an ongoing communication service or when no display device is proximately located to the user.
[0068] Further operations by user tags, I/O user devices, and the user terminal emulation server are described in accordance with this example use case. A user tag enters a room and signals its presence to any proximately located and capable I/O user device in the room using a discovery beacon signal. Alternatively, one or more of the I/O user devices determines presence of the user tag by polling, such as by periodically transmitting discover beacon signals that trigger responsive signaling by the user tag. The I/O user devices that receive signaling indicated presence of the user tag report to the IODH 212 along with a network address of the I/O user device (e.g., IP address, port number, MAC address, FQDN, etc.) or an API of an application function on the I/O user device.
[0069] Alternatively, the signaling may be implicit when the IODH 212 already has presence information and/or presence is determined based on content of IP packets sent from the I/O user device to the IODH 212. The IODH 212 may be executed by the user terminal emulation server as part of the user terminal emulation application, by the I/O user devices, and/or by another computing node of the system. The user terminal emulation application corresponding to the specific user (i.e., the user tag) is updated with respect to the detected user's presence. The IODH 212 may operate to receive the notifications from the I/O user devices proximately located to the user tag. Further UI capability discovery (synchronization) communications may optionally be performed between the IODH 212 and the I/O user devices, or the IODH 212 may be per-configured with knowledge of the UI capabilities of the I/O user devices. The I/O user devices are associated to the user in the database 120, along with associated indications of combinable UI capabilities provided by the set of I/O user devices which are proximately located to the user tag. One or more of the I/O user devices may be selected for default call reception ACK/NACK. The user via the user tag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service. The user may receive or accept a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
[0070] An incoming session (e.g., video call) from a requesting user terminal which is directed to the user (user tag) arrives at the user terminal emulation server for the user carrying the user tag. The individual or combinable UI capabilities of the available I/O user devices is compared to the UI requirements of the incoming session. When the UI requirements of the incoming session are not satisfied by the UI capabilities of the I/O user devices, the user terminal emulation server may renegotiate the required UI capabilities (e.g., QoS) of the incoming session. In contrast, when the UI requirements of the incoming session are satisfied the user terminal emulation server prompts, via one or more of the available I/O user devices (e.g., a pre-selected answer device), the user carrying the user tag to provide a session request answer (ACK/NACK). The user responds through the pre-selected answer device to accept (ACK) or reject (NACK) the incoming session, to provide signaling to the user terminal emulation server. When an ACK is received, operations route an audio stream from the requesting user terminal to one of the I/O user devices in the set that has a speaker
capability via one or more sessions, and routes a video stream from the requesting user terminal to another one of the I/O user devices in the set that has a display capability via one or more sessions. A data stream that is received from one of the I/O user devices in the set through a one or more sessions is routed toward the requesting user terminal. When two or more data streams are received through one or more sessions from the I/O user devices, they can be combined into a combined data stream that is routed toward the requesting user terminal.
[0071] The user terminal emulation server or IODH 212 may perform operations to continuously monitor presence of the I/O user devices to determine when one or more of I/O user devices is no longer proximately located to the user such that it can no longer be included as part of the combined UI be provided during the ongoing communication session. The user terminal emulation server or IODH 212 may substitute the UI capability of another I/O user device to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence.
[0072] Use case 2, Outgoing call operations are now discussed below.
[0073] This use case involves a user, with a user tag, being proximately located to I/O user devices 130 having different UI capabilities when an outgoing call (communication session) is received by the user terminal emulation server. The I/O user devices 130 are associated to the identified user via the user terminal emulation server 100 which handles all communications sessions for the user while the associated I/O user devices 130 are managed by an IODH 212.
[0074] A user terminal emulation application 110 may be instantiated or otherwise activated responsive by an outgoing call being requested by a user carrying the user tag. The user may initiate an outgoing call through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
[0075] The user terminal emulation application 110 can identify subscriptions associated with the user tag and preferred methods of communication (e.g., audio not video, audio and video, etc.) that have been specified by the user, and determines the UI capabilities of the I/O user devices that will be needed to satisfy the UI capabilities which may be specified for the outgoing call. The user terminal emulation application 110 may ask the IODH 212 to identify which I/O user devices 130 are proximately located to the user tag, and may further ask the IODH 212 to determine or may determine itself whether the identified I/O user
devices 130 are individually useable or combinable to satisfy the UI capabilities specified by the outgoing call. The user terminal emulation application 110 and/or the IODH 212 may receive an ACK or NACK back on whether one or a set of I/O user devices 130 can be used to provide the communication service. If ACK, then the IODH 212 also sets the state of the one or more I/O user devices 130 in the set to in-use to avoid another user terminal emulation application 110 attempting to utilize the same I/O user device(s) 130 or one or more capabilities of that I/O user device(s) 130 which are presently in use. In case of NACK, the user terminal emulation application 110 and/or the IODH 212 can take different actions to setup a reduced UI capability communication service with the user depending on user settings, e.g. only allow sound instead of the preferred sound and video responsive to when no display device is presently available for use (e.g., when presently used by another user terminal emulation application 110 or when none is proximately located to the user tag).
[0076] Example operations for an outgoing call and related data flows between user tags, I/O user devices, and the user terminal emulation server are now described in the context of this use case. A user tag enters a room and signals its presence to any proximately located and capable I/O user device in the room using a discovery beacon signal. Alternatively, one or more of the I/O user devices determines presence of the user tag by polling, such as by periodically transmitting discover beacon signals that trigger responsive signaling by the user tag. The I/O user devices that receive signaling indicated presence of the user tag report to the IODH 212 along with a network address or other identifier of the I/O user device (e.g., IP address, port number, MAC address, FQDN, etc.) or API of an application function on the I/O user device. The IODH 212 may be executed by the user terminal emulation server as part of the user terminal emulation application, by the I/O user devices, and/or by another computing node of the system. The user terminal emulation application corresponding to the specific user (i.e., the user tag) is updated with respect to the detected user's presence.
[0077] The IODH 212 may operate to receive the notifications from the I/O user devices proximately located to the user tag. Further UI capability discovery (synchronization) communications are performed between the user terminal emulation server or the IODH and the I/O user devices. The I/O user devices are associated to the user in the database 120, along with associated indicated service subscriptions and combinable UI capabilities provided by the set of I/O user devices which are proximately located to the user tag. One or more of the I/O user devices may be selected for use in making a call or initiating another service. The user via the user tag is now known to be reachable within the system through an identified set of I/O user devices with identified UI capabilities (e.g., speakers yes/no, display
yes/no, microphone yes/no, keyboard yes/no, etc.), thereby creating a logical virtualized user terminal through which the user may be provided in a communication service. The user may initiate a communication service through a touchscreen, voice command sensed by a microphone, performing a defined gesture observable by a camera, and/or other input provided to one of the proximately located I/O user devices.
[0078] A user carrying the user tag uses the UI of one of the I/O user devices to trigger an outgoing call (e.g., video call), which triggers signaling of the outgoing call to the user terminal emulation server. The IODH 212 and/or the user terminal emulation application queries the user (e.g., displays a message, generates a sound, etc.) through one of the I/O user devices proximately located to the user to request the user to select among available types of communication methods that can be presently used for the outgoing call. One of the I/O user devices provides responsive signaling to the IODH 212 or user terminal emulation server 100 indicating the user's selected type of communication method for the outgoing call. The user terminal emulation server communicates an outgoing session stream request to the network entity 150, where the request may include an identifier of the calling user, identifier of the user terminal of the called user, and a quality of service for the communication session. The user terminal emulation server receives a communication session acceptance (ACK) or denial (NACK) from the network entity 150. When the communication session is denied, the user terminal emulation server may attempt to renegotiate the requested communication session such as at a lower quality of service.
[0079] When the communication session is accepted (ACK), for each data type that is received as communication traffic from the requested user terminal, the user terminal emulation server selects one of the I/O user devices from among the set of I/O user devices based on matching characteristics of the data type to the UI capabilities identified by the database 120 for the one of the I/O user devices, and then routes the data of the data type toward the network address of the selected one of the I/O user devices or API of a selected application function on the I/O user device. The I/O user devices transmit data streams through one or more sessions to the user terminal emulation server, which may combine the data streams into a combined data stream that is routed toward the called user terminals via the network entity 150.
[0080] The user terminal emulation server or IODH may continuously monitor presence of the I/O user devices to determine when one or more of I/O user devices is no longer proximately located to the user such that it can no longer be included as part of the combined UI be provided during the ongoing communication session. The user terminal emulation
server or IODH may substitute the UI capability of another I/O user device to the set being used by the user for the ongoing communication session responsive to a previous member of the set no longer having required presence.
[0081] Security issues and related operations are now discussed below.
[0082] It can be desirable in some systems to provide operation for a virtual instance, e.g., virtual terminal emulation application 110, in the cloud, e.g., user terminal emulation server 100, to dynamically secure use of physical resources using authentication and key establishment procedures.
[0083] Some embodiments are directed to using a trusted party (function) to enable secure access and communications between a cloud service, e.g., user terminal emulation server 100 and I/O user device(s) 130. These embodiments may extend existing authentication functions to establish a secure association among various elements of the systems described in Figures 1-3 and, in particular, between the user terminal emulation server 100, the I/O user device(s) 130 proximately located to a user, and the user tag transported by the user.
[0084] In some embodiments, the user, who has been registered and authenticated to a cloud service, carries a user tag which has been associated (e.g., by a user identity) to the user terminal emulation server 100 such in the database 120. As explained above, there can be numerous I/O user devices 130 that are associated and registered to a lookup service, e.g., IODH 212.
[0085] When a user enters the close proximity of at least one I/O user device 130, operations are performed based on Extensible Authentication Protocol (EAP), Authentication and Key Management for Applications (AKMA), or another security function to establish a secure association between the user terminal emulation server 100, the I/O user device(s) 130 proximately located to the user, and the user tag transported by the user. The secure association enables the user terminal emulation server 100 to utilize the UI capabilities of those I/O user device(s) 130 to obtain a communication service.
[0086] In some example scenarios, the user tag is transported by a user to identify the user and can be capable of short-range radio signalling, e.g., NFC, RFID, Bluetooth, etc., and/or long-range radio (LoRa) signalling, e.g., WiFi, cellular, etc. The user tag is registered and authenticated to the user terminal emulation server. At least one I/O user device 130 has a local service providing certain UI capabilities which can be registered with the user terminal emulation server 100 or IODH 212 for inclusion in a lookup service provided
through a database, e.g., database 120 and/or which may reside in the IODH 212 and/or the I/O user devices 130.
[0087] Although some embodiments are described in the context of the user terminal emulation server 100 being a cloud-based service, these and other embodiments are not limited thereto. Operations disclosed herein can be used to enable an authenticated user to securely couple physical resources, residing on separate private networks, to a cloud-based service. The cloud-based service may be a phone service, videoconference service, streaming media service, video on-demand service, audio on-demand service, web service, digital assistant service, service technician service and/or any other service which may operate to communicatively connect to physical resources in the proximity of a user.
[0088] Various embodiments are now separately explained in the context of EAP -based authentication operations and the context of AKMA-based authentication operations, although the operations may be used with any security function operations which can facilitate secure access. In some embodiments, the user terminal emulation server 100 performs operations to establish a secure channel connection with a first I/O user device 130 using a session identifier and an identifier associated with the first I/O user device to determine a first I/O user device specific key generated from a master key, the first I/O user device specific key and the session identifier being used for authentication and secure communication of messages with the first I/O user device. The operations receive an indication of an I/O user interface capability of the first I/O user device 130 through the secure channel connection with the first I/O user device 130. The operations communicate with the first I/O user device 130 to use the I/O user interface capability to provide at least part of the communication service for a user.
[0089] These and other related operations are first described in the EAP -based authentication context and then described in the AKMA-based authentication context.
[0090] EAP-Based Authentication and related operations are now discussed below. [0091] EAP is an authentication framework which supports multiple authentication methods. EAP typically runs directly over data link layers such as Point-to-Point Protocol (PPP) or IEEE 802, without requiring IP. EAP provides its own support for duplicate elimination and retransmission but is reliant on lower layer ordering guarantees. Fragmentation is not supported within EAP itself; however, individual EAP methods may support fragmentation.
[0092] EAP is a two-party protocol spoken between the EAP peer and server. Within EAP, keying material is generated by EAP authentication algorithms, known as "methods". Part of this keying material can be used by EAP methods themselves, and part of this material can be exported. In addition to the export of keying material, EAP methods can also export associated parameters such as authenticated peer and server identities and a unique EAP conversation identifier, and can import and export lower-layer parameters known as "channel binding parameters", or simply "channel bindings".
[0093] From RFC 5247: “EAP methods supporting key derivation and mutual authentication SHOULD export a method-specific EAP conversation identifier known as the Session-Id. . . ” Exporting can entail providing the exported information to the EAP authenticator.
[0094] An example, flow of an EAP exchange for access authentication (802. lx used for WLAN/LAN), can include the device attaching to an authenticator point (AP). This means an (e.g.) 802.11 association is established between device and AP. The AP requests the identity of the device with an EAP identity request message. The device replies with its identity in an EAP response message. The AP, which works as an Authenticator, forwards the identity in a RADIUS or DIAMETER access request message to the authentication server.
[0095] The authentication server replies with a challenge for the device and indicating the EAP method to be used. The AP forwards the request in an EAP request message to the device. The device responds to the EAP message with an EAP response message. The AP forwards the response in a RADIUS or DIAMETER message to the authentication server. EAP messages are exchanged between the device and the authentication server until the authentication server has authenticated the device using the chosen method. The authentication server sends a RADIUS or DIAMETER access accept message, containing a pairwise master key (PMK) to the AP. The AP keeps the PMK and forwards the success in an EAP success message to the device. The device generates the corresponding PMK. The device is authenticated and the AP and device can use the PMK to configure access security. EAP can also be run towards Authenticators other than WLAN APs, and the Authenticator can be co-located with/part of the Authentication server.
[0096] EAP-based operations between a user tag 110 and the user terminal emulation server 110, which can operate an EAP server (function), are now described with reference to Figure 4. Figure 4 illustrates a combined flowcharts of operations and related data flows between the user terminal emulation application 110 and elements of an I/O user device (IOD) domain 410, such as I/O user devices 130, a lookup service/IODH, and an Extensible
Authentication Protocol (EAP) authenticator 400 in accordance with some embodiments of the present disclosure. For brevity when discussing various example operations below, the term "I/O user device" is abbreviated as "IOD".
[0097] Referring to Figure 4, an assumption in the EAP-based approach is that the user tag is transported with the user and contains circuitry which is configured to operate as an EAP client. The user tag therefore has at least limited communication and computational capabilities. An example user tag can be any type of mobile electronic device, such as tablet computer, mobile phone, or less complex device with NFC, RFID, Bluetooth, or other short- range communication such as capacitive communication coupling. The IODS 130 are registered with their (local) IODH 212, which functions as a lookup service for IOD A ... IOD x in the local IOD domain 410, i.e., the IODH 212 has information characterizing IOD A ... IOD x which may include an identifier, authentication credentials (e.g. public key of IOD A ... IOD x, location of IOD A ... IOD x such as spatial coordinates, room numbers, floor numbers, etc., defining type information, defining UI capabilities, etc. In some embodiments, the user terminal emulation application 110 may be executed within the IOD Domain 410, such as with the EAP Authenticator 400 and/or the IODH 212. Executing the user terminal emulation application on the same computing node or on a locally networked node as the EAP authenticator and/or IODH can simplify and reduce system resources consumed to exchange information and other communications therebetween.
[0098] Which users and/or user terminal emulation applications are allowed to interact with IOD A ... IOD x in the IOD domain 410 can be determined based on access control policies which can reside in the IODH 212 and may vary depending on use case scenario, e.g., semi-public domain, such as a hotel vs. private domain such as an enterprise office complex.
[0099] As will be explained below with regard to Figures 4 and 5, operations establish 405 (Figure 4) and 511 (Figure 5) a secure channel connection with a first IOD 130 using a session identifier and an identifier associated with the first IOD to determine a first IOD specific key generated from a master key, where the first IOD specific key and the session identifier being used for secure communication of messages with the first IOD.
[00100] In the scenario of Figures 4 and 5, a user who is transporting a user tag 101, e.g., dongle, wants to utilize a proximately located IOD A. The user may trigger IOD A, by pushing a button on the user tag 101, triggering a command responsive to the user tag seeing a Service Set IDentifier (SSID) or Bluetooth (BT) address of IOD A, and/or IOD A. For example, the user may activate or initiate attention from IOD A using the user tag over NFC
or RFID (i.e., very close proximity), and/or by the user pushing a button on the IOD A to initialize a bootstrapping process 501. Alternatively the user tag can broadcast a beacon signal which can be heard by an IOD and which will trigger the authentication.
[00101] The user tag sends 502, via the IOD (target), an attach request to the system where the IOD is connected. The attach request is forwarded 503 by the IOD to the EAP Authenticator 400 in the system. The EAP authenticator 400 may be a local process in OD A or in the IODH 212 of the user terminal emulation server 100, or may reside as a separate entity in the IOD domain 410. The attach request may therefore be processed by the IOD itself, i.e., where multiple EAP Authenticators are present in the system, be processed by the IODH 212, or be processed by the EAP authenticator 400.
[00102] The EAP authenticator 400 responds 504 with an EAP identity request, which is communicated back to the user tag 101.
[00103] The user tag 101 responds 505 with an EAP response, which carries the identity of the user tag 101. The identity identifies the user tag 101 (e.g., holder of user tag, such as the user) and points to the user terminal emulation application 110 associated with the user tag 101. The identity may be <hash(user_pub_key)>@<user_terminal emulation application_address>. The hash of the public key provides a more compact identity of the public key of the user tag 101, and may be included with a network address of the user terminal emulation application 110. As explained above, the user terminal emulation application 110 may provide a communication service function corresponding to, for example, an over-the-top Voice Over Internet Protocol (VoIP) service, Netflix service, Facebook service, Microsoft Teams meeting service, Internet browser service, a cellular communication service, etc.
[00104] When the user tag 101 has been issued to the user, it may have also been associated with the corresponding user terminal emulation application 110. This means that the user tag 101, in addition to its own credentials (e.g., public key pair), can be configured to have reachability information of the user terminal emulation application 110, e.g., a Fully Qualified Domain Name (FQDN), as well as credentials for authentication of the user terminal emulation application 110 (e.g., public key of user terminal emulation application 110). Likewise, the user terminal emulation application 110 may have been configured with the credentials of the user tag 101 (e.g., public key of user tag 101). This means that the user terminal emulation application 110 knows that the user tag 101 is an entity that is authorized to request data streams and services from the user terminal emulation application 110.
[00105] The EAP authenticator 400 identifies the user terminal emulation application 110 based on the realm part of the identifier and forwards the request to the user terminal emulation application 110, which is here acting as the EAP server (referred to as "EAP server/user terminal emulation application 110" for brevity).
[00106] The EAP authenticator 400 first establishes 506a secure connection between itself and the EAP server/user terminal emulation application 110. The EAP messages are passed over that secure channel between authenticator and EAP server/user terminal emulation application 110. The secure connection may be a Transport Layer Security (TLS) session. The EAP Authenticator 400 knows it needs to talk with the EAP server/user terminal emulation application 110 based on the identity (realm part of identity) received 506b in EAP Identity Response. It also knows it needs to have a secure channel with the EAP server/user terminal emulation application 110. Thus, if there is not an existing secure session (typically TLS), the EAP Authenticator 400 creates the secure session and then forwards 506b the Identity response to the EAP server/user terminal emulation application 110. The communication between the EAP Authenticator 400 and the EAP server/user terminal emulation application 110 function of the user terminal emulation application 110, while using EAP messages, may be carried by DIAMETER or RADIUS protocol.
[00107] The user terminal emulation application/EAP server 110 and the user tag 101 will perform an EAP exchange 507 to authenticate the user tag 101 to the user terminal emulation application 110. The EAP server/user terminal emulation application 110 may also be authenticated to the user tag 101. The authentication may require multiple messages to be exchanged between the user tag 101 and the EAP server/user terminal emulation application 110 via the EAP authenticator 400.
[00108] Once the user tag 101 and possibly also the EAP server/user terminal emulation application 110 is/are successfully authenticated, the EAP server/user terminal emulation application 110 can send 508 a final EAP SUCCESS message to the EAP authenticator 400. The EAP SUCCESS message also carries the master key, e.g., pairwise master key (PMK), and the session-ID. The PMK key is the master key for the session-ID, the PMK key can be used to derive more keys for lODs, e.g., IOD X, added to this session.
[00109] In an optional operation, the user tag 101 at this stage may derive the master key, e.g., PMK key, but it will not (necessarily) be used by the user tag 101. The master key can be used if the user wants to authorize additional IOD, e.g., IOD X, assuming the user is more in control of which lODs are added to the session by having to actively (possibly even physically) add more lODs to the EAP server/user terminal emulation application 110.
[00110] The authenticator generates 509 an IOD specific key K iodA from the received keying material to be used for streaming data between user terminal emulation application 110 and the target IOD A. Generating the IOD specific key K iodA may include using the received keying material as is, or may include performing a computational operation on the received key material such as by hashing a concatenation of the keying material and the identifier of the target IOD A and/or using another key derivation function (KDF) based on PMK and IOD specific information.
[00111] The EAP authenticator 400 provides 510 the IOD specific key K iodA to the IOD A, together with the session-ID exported by the EAP server/user terminal emulation application 110 and the address (e.g. FQDN) of the user terminal emulation application 110. [00112] The IOD A can use the received data to establish 511 a secure channel (e.g., TLS) between itself and the user terminal emulation application 110. It is noted that in some embodiments, to establish the secure channel between IOD A and the EAP server/user terminal emulation application 110 there needs to be a shared secret (which is the first IOD specific key K_iodA), an identifier for who is connecting to the EAP server/user terminal emulation application 110 (IOD A) and that is used to derive IOD specific key, and an identifier (session-ID) that the EAP server/user terminal emulation application 110 can use to lookup the context from where it can derive the corresponding K iodA.
[00113] If the EAP Authenticator 400 is part of IOD A, then IOD A could re-use the TLS session established in operation 506. However, a more scalable solution is enabled by using a uniform operation for all IODS connecting to the EAP server/user terminal emulation application 110 so as to not have special cases in an implementation.
[00114] In operation 511, the IOD A can use the session-ID to indicate to the EAP server/user terminal emulation application 110 the session /keying material/authentication context to which the connection request relates.
[00115] The EAP server/user terminal emulation application 110 locates context and associate keying material based on received session-ID. As background, the IOD A is to connect with the EAP server/user terminal emulation application 110 using a secure channel so the EAP server/user terminal emulation application 110 can stream or receive data from the IOD A, such that the IOD A needs to have keying material which it can use to establish a secure connection and the user terminal emulation application 110 needs to verify that the IOD A is authorized to send data. So, the session ID that was sent in operation 508 is what the IOD A can send to the EAP server/user terminal emulation application 110 so it knows
the request relates to the authentication session that was setup for the session ID, and then locates its copy of the PMK key and any other keys negotiated during the authentication. [00116] The IOD uses its own identifier (e.g. IOD A) as a kind of username. The IOD A thereby tells the EAP server/user terminal emulation application 110 who it is. The EAP authenticator 400 has derived an IOD A specific key based on the PMK key, e.g., by concatenating the IOD A ID and the PMK key and then hashing the concatenated string, and may truncate the hash value to a defined length. The EAP server/user terminal emulation application 110 needs to know the ID for the IOD A so it can derive the same IOD A specific key provided to IOD A in, e.g., step 510.
[00117] The EAP server/user terminal emulation application 110 uses the IOD identifier to derive IOD A specific key (K iodA). The IOD A uses the received key K iodA as the password/authentication credential to authenticate to the EAP server/user terminal emulation application 110. In this operation the IOD A and the EAP server/user terminal emulation application 110 share the same IOD A specific key which is used as a shared secret that the EAP server/user terminal emulation application 110 and IOD A use to authenticate each other and establish a secure channel therebetween. The EAP server/user terminal emulation application 110 verifies, via PSK-based authentication, that the IOD indeed possesses a valid session key (K_iodA) and is thus authorized to connect to the EAP server/user terminal emulation application 110 and exchange data with it. A secure channel is established between the IOD A and the EAP server/user terminal emulation application 110.
[00118] The IOD A, after successful authentication, can indicate 512 its UI capabilities (e.g., display, speaker, microphone, etc.) to the user terminal emulation application 110, which based on the UI capabilities, can enable data streaming to and/or from the IOD A. [00119] The user may have defined policies to the EAP server/user terminal emulation application 110 regarding what operations can be enabled automatically (e.g., streaming video to a display device for the communication service) and what operations requires explicit user consent before performing (e.g., to enable microphone use for the communication service)
[00120] In a parallel optional operation, the EAP server/user terminal emulation application 110 may operate to interact 513 with the IODH 212 regarding other lODs in the vicinity of the user which have UI capabilities that can be used to provide the communication service to the user. These operations can provide the EAP server/user terminal emulation application 110 information about what UI and/or I/O capabilities could be available to the user for the communication service. Whether the EAP server/user terminal emulation
application 110 may operate to interact 513 with the I0DH 212 regarding other IODS may depend on which service and/or application is running in the EAP server/user terminal emulation application 110.
[00121] This communication may be performed via an entity of the IOD domain 410 acting as an EAP authenticator 400 towards the EAP server/user terminal emulation application 110, and thereby the already established secure channel could be re-used. Alternatively, the EAP authenticator 400 may generate an IODH 212 specific session key (similar to what was performed for IOD A) and provide IODH 212 with all relevant info (e.g., K_iodh, session-ID, EAP server/user terminal emulation application 110 FQDN) for securely communicating with the user terminal emulation application 110.
[00122] The IODH 212 may itself be the EAP authenticator 400, in which case much of the “communication” between authenticator and IODH 212 is simplified, such as where the PMK is used directly by the IODH 212 to securely communicate with user terminal emulation application 110, or the already established secure session is re-used.
[00123] The communication may include the IODH 212 telling the user terminal emulation application 110 about which IODs are available to the user, and may include the EAP server/user terminal emulation application 110 requesting certain hardware resources from the IODH 212.
[00124] When the IODH 212 concludes, or the EAP server/user terminal emulation application 110 requests, that a certain other IOD (IOD X) should join the session established between the user and the IOD domain 410, the IODH 212 can request 514a the EAP authenticator 400 to generate credentials for the other IOD X, (e.g., K iodx, session-ID, EAP server/user terminal emulation application 110 FQDN) and provide those credentials to the other IOD X.
[00125] The IODH 212 may send 514a a request based on the user instructing the EAP server/user terminal emulation application 110 (e.g., over connection with some already connected IOD) or based on local policy and/or configuration. The decision that a further IOD is required may be dependent upon: which application and/or service the user activates; ongoing application and/or service; available devices and their respective UI capabilities; and/or a defined configuration by the user to always try to find an IOD which has certain UI capability or capabilities.
[00126] If the other IOD X receives 514c a trigger (e.g., where the trigger is the credentials etc. needed to connect to user terminal emulation application 110) from the IODH 212 or
EAP authenticator 400, the other IOD X performs similar operations to the IOD A as described above in operations 511 to 512.
[00127] More general operations are now described with respect to Figures 4 and 5. [00128] Embodiments are not limited to the particular operations illustrated in Figure 5 and discussed above. For example, the user tag 101 can more generally include circuitry that is configured to send 502 to a first I/O user device 130 (which may correspond to IOD A) an attach request, and receive 504 from the first I/O user device 130 an identity request by an authenticator 400. The user tag 101 then sends 505 to the I/O user device a response which contains an identifier of the user tag and an address of a user terminal emulation application 110 hosted by a user terminal emulation server 100. The user tag 101 then communicates 507 with the user terminal emulation application (110) to perform an exchange for mutual authentication and establish a master key used to generate one or more I/O user device specific keys.
[00129] The circuitry of the user tag 101 may be powered by NFC reader of the first I/O user device 130 to send 502 the attach request, to receive 504 the identity request, to send 505 the response, and to communicate 507 with the user terminal emulation application 110 to perform the exchange. The circuitry may be further configured to generate 505 the identifier of the user tag based on hashing a public key of the user tag.
[00130] The user terminal emulation server 100 is generally configured to provide a communication service through I/O user devices 130, and includes at least one processor 800 (Figure 8) and at least one memory 820 (Figure 8) storing program code that is executable by the at least one processor to perform operations. The operations include to establish 405 (Figure 4) and 511 (Figure 5) a secure channel connection with a first I/O user device (130) using a session identifier and an identifier associated with the first I/O user device to determine a first I/O user device specific key generated from a master key, where the first I/O user device specific key and the session identifier being used for secure communication of messages with the first I/O user device. The first I/O user device specific key may be determined based on the I/O user device identifier, such as an I/O user device key (e.g., KDF(PMK, I/O user device ID), where KDF is a key derivation function). The operations receive 405 and 512 an indication of an I/O user interface capability of the first I/O user device 130 through the secure channel connection with the first I/O user device 130. The operations communicate 405 and 512 with the first I/O user device 130 to use the I/O user interface capability to provide at least part of the communication service for a user.
[00131] The operations by the EAP server/user terminal emulation application 110 can further include receiving 506b an EAP response through a communication channel with the EAP authenticator 400, where the EAP response contains an identifier of a user tag 101. The operations communicate 507 with the user tag 101 to perform an EAP exchange for authentication and establish the master key, and send 508 an EAP success message to the EAP authenticator 400, where the EAP success message contains the master key and a session identifier associated with the master key. Following the sending 508 of the EAP success message to the EAP authenticator 400, the operations perform the receiving 405 of the indication of the I/O user interface capability of the first I/O user device 130 and the communicating 405 with the first I/O user device 130 to use the I/O user interface capability to provide at least part of the communication service for the user.
[00132] The operations by the user terminal emulation server 100 can further include to store in the database 120 (Figure 1) the identifier of the user tag allowed to access the communication service, a network address of the first I/O user device based on communications with the first I/O user device, the first I/O user device specific key, and the indication of the I/O user interface capability of the first I/O user device 130.
[00133] It is noted that initially the user terminal emulation server 100 (i.e., via the EAP server/user terminal emulation application 110) stores the session identifier and the PMK. When an I/O user device connects to the user terminal emulation server 100, the I/O user device provides its identifier to the user terminal emulation server 100 as part of the connection establishment procedure. The user terminal emulation server 100 can now generate the I/O user device specific key. Using the I/O user device specific key, the user terminal emulation server 100 can authenticate the I/O user device, and after (or as part of authentication) which can establish the secure channel between the two. The I/O user device has obtained the corresponding key from the EAP Authenticator 400 or the IODH 212 (if the EAP authenticator 400 sends the data via the IODH 212 to the I/O user device) in IOD domain 410. The I/O user device can likewise optionally authenticate the user terminal emulation application 110 using the key (i.e., verify that the user terminal emulation application 110 belongs in the session as it possesses a key associated with it). The I/O user device specific key can be generated by the user terminal emulation application 110 only after the user terminal emulation application 110 has learned the ID of the I/O user device specific key. In some embodiments, the EAP server/user terminal emulation application 110 learns the I/O user device identifier when the I/O user device tries to connect to the EAP server/user terminal emulation application 110.
[00134] In some embodiments, the order of events includes the I/O user device connects to the EAP server/user terminal emulation application 110 (with own ID, session ID and the I/O user device specific key). The EAP server/user terminal emulation application 110 identifies session context based on session ID. The EAP server/user terminal emulation application 110 derives the I/O user device specific key (and maybe stores in in the database 120). The EAP server/user terminal emulation application 110 and the I/O user device authenticate based on the I/O user device specific key. A secure channel can then be set up based on the I/O user device specific key.
[00135] The operations to establish 405 the secure channel connection with the first I/O user device 130 using the session identifier and the identifier associated with the first I/O user device to determine the first I/O user device specific key generated from the master key, can include to receive a secure channel connection request which includes the identifier of the first I/O user device 130 and the session identifier, and initiate the determination of the first I/O user device specific key based on the master key. The operations store the first I/O user device specific key in the database 120 with an association to the session identifier. The operations obtain the first I/O user device specific key from the database 120 using the session identifier, authenticate the first I/O user device based on the first I/O user device specific key, and setup the secure channel connection with the first I/O user device 130 responsive to authentication of the first I/O user device.
[00136] Corresponding operations by the EAP authenticator 400 are now described. The EAP authenticator 400 can include at least one processor 930 (Figure 9), at least one memory 940 (Figure 9) storing program code that is executable by the at least one processor to perform operations. The operations include receiving 505, from the first I/O user device 130, an EAP response which contains the identifier of the user tag containing an address of a user terminal emulation application 110 hosted by a user terminal emulation server 100. The operations establish 506a a communication channel with the user terminal emulation application 110 based on the address in the user tag of the user terminal emulation application 110, and send 506b at least one EAP message based on the EAP response through the communication channel with the user terminal emulation application 110. The EAP authenticator 400 also receives EAP messages from the user terminal emulation application 110. In general, the EAP Authenticator 400 passes EAP messages between the user tag 101 and the user terminal emulation application 110. The operations receive 508 an EAP success message from the user terminal emulation application 110, where the EAP success message contains a master key and a session identifier, and generate 509, based on the master key, a
first I/O user device specific key. The operations then send 510 to the first I/O user device 130, e.g., via the I0DH 212, the first I/O user device specific key, the session identifier, and the address for the user terminal emulation application 110.
[00137] The at least one EAP message may be sent 506b through the secure channel connection to the user terminal emulation application 110 using DIAMETER protocol or RADIUS protocol. The EAP authenticator 400 exchanges EAP messages between the user tag 101 and the user terminal emulation application 110.
[00138] The EAP authenticator 400 may generate 509 the first I/O user device specific key based on a key derivation function performed on the master key and an identifier of the first I/O user device 130.
[00139] The EAP authenticator 400 may obtain 513 an identifier of a second I/O user device 130 that is proximately located to the first I/O user device 130 and has an I/O user interface capability that satisfies a rule for being combinable with the I/O user interface capability of the first I/O user device 130 to provide a communication service, and generate 514b, based on the master key, a second I/O user device specific key. The EAP authenticator 400 may then send 510 to the second I/O user device 130, e.g., via the IODH 212, the second I/O user device specific key, the session identifier, and the address for the user terminal emulation application 110.
[00140] Corresponding operations by the first I/O user device 130 are now described. The first I/O user device 130 can include at least one processor 700 (Figure 7), and at least one memory 710 (Figure 7) storing program code that is executable by the at least one processor to perform operations. The operations include receiving 502 from a user tag an attach request, and forward 503 the attach request to an authenticator 400. The operations forward 504 to the user tag an identity request received from the authenticator 400, and forward 505 to the authenticator 400 a response received from the user tag, the response which contains an identifier of the user tag containing an address of a user terminal emulation application 110 hosted by a user terminal emulation server 100. The operations receive 510 from the authenticator 400 a message comprising a first I/O user device specific key for the first I/O user device 130, a session identifier, and the address for the user terminal emulation application 110. The operations establish 511 a secure channel connection with the user terminal emulation application 110 using the first I/O user device specific key and the session identifier received from the authenticator 400. The operations send 512 an indication of an I/O user interface capability of the first I/O user device to the user terminal emulation application 110 through the secure channel connection. The operations communicate 512
with the user terminal emulation server 100 to use the I/O user interface capability of the first I/O user device 130 to provide at least part of a communication service to a user.
[00141] The operation to establish 511 the secure channel connection with the user terminal emulation application 110 using the first I/O user device specific key and the session identifier received from the authenticator 400, may include sending the session identifier and an identifier of the first I/O user device to the user terminal emulation application 110 to indicate which I/O user device specific key is to be used to establish 511 the secure channel connection.
[00142]
[00143] Utilizing authorization tokens and related operations are now discussed below.
[00144] In operation 508 above, when the user tag 101 has authenticated itself towards the EAP server/user terminal emulation application 110 using EAP, the user tag 101 now also possesses the session keying material. Using this, the user tag 101 may generate authorization tokens for other IODS, e.g., IOD X, that the user tag 101 wants to add to the currently ongoing session.
[00145] These operations may be an alternative to having the EAP Authenticator 400 or the IODH 212 delegate session keys to the I/O user devices. The token may, e.g., be a signed piece of data contain things such as the session ID (so that the EAP server/user terminal emulation application 110 can map the token to the session), the newly selected I/O user devices identity (so that EAP server/user terminal emulation application 110 can verify that the correct I/O user device is using the token) and possible a lifetime of the token.
[00146] The user tag 101 may provide such token (and a pointer to the EAP server/user terminal emulation application 110) to selected I/O user devices, which could then connect to the EAP server/user terminal emulation application 110 and present the token as proof of authorization by the user tag 101. The communication between user tag 101 and newly selected I/O user device may be similar to the initial registration to the IOD Domain 410 as described above starting with operation 501; the user tag 101 indicates it wants to connect the I/O user device IOD X to the EAP server/user terminal emulation application 110, but in a way that the I/O user device IOD X understands to reply with its identity.
[00147] As an example, when providing the user tag's identity in the EAP identity reply message, the user tag 101 may use the session ID as an identifier (or part of the identifier). The I/O user device IOD X itself, or the EAP Authenticator 400 may determine from the identifier a relation to an already ongoing session and which would result in providing the user tag 101 with the identity of the new I/O user device.
[00148] The identity of the I/O user device may be interpreted as an authentication challenge in an EAP method. The reply to the challenge may be the token generated by the user tag 101. The I/O user device IOD X may use the help of the EAP Authenticator 400 to verify that the token is valid (e.g., the EAP Authenticator 400 possesses the same keying material and can verify the signature), or may blindly trust the token and start using it towards the EAP server/user terminal emulation application 110.
[00149] These operations provide the user tag 101 and/or the user more control with respect to which I/O user device(s) are connected to the session as the user needs to select the used I/O user device(s). For proper trust in the tag by the EAP server/user terminal emulation application 110, the user tag 101 would also have a signature generated by the permanent credential of the user tag 101 towards the EAP server/user terminal emulation application 110 (or non-exported keying material of the EAP exchange, i.e. keying material not shared with the EAP Authenticator 400), since the EAP generated session key is also known to the EAP Authenticator 400, which therefor could generate a valid looking token even without the knowledge of the user.
[00150] Public vs. private IOD domains and related operations are now discussed below.
[00151] The operations described above may be used in various scenarios such as in public locations (e.g., vacation resort/hotel) or private locations (e.g., enterprise office/complex). For these different types of scenarios there are differences with respect to access control requirements, such as for a public location (typically) anyone should be able to connect their cloud service (e.g., user terminal emulation application 110) to a public I/O user device, while in a private setting only authorized services/users would typically be allowed. This could be, e.g., the employees of a company being allowed to connect their user terminal emulation application 110 to I/O user devices in the office building. Alternatively, a mixed model may be provided where certain I/O user devices are accessible to anyone, while some other I/O user devices are only accessible to a subset of users and/or user terminal emulation applications 110.
[00152]
[00153] Controller function and related operations are now discussed below.
[00154] A controller function in the IOD Domain 410 may be responsible for verifying that the connecting user terminal emulation application 110 is authorized to connect to the specified I/O user device. This could be a separate entity or a function of the IODH 212, which knows about all the I/O user devices in the domain 410. Alternatively, the controller function may be a part of the EAP Authenticator 400 or an Authentication and Key
Management for Applications (AKMA) server, respectively (which in turn may be part of IODH 212, or be separate entities/functions).
[00155] When a user terminal emulation applications 110 or user is being authenticated to an AKMA server or the EAP Authenticator 400, the I/O user device Domain controller (IDC) can operate to verify that the connecting identity (tag identity authenticated with EAP, user terminal emulation applications identity learnt during EAP while setting up secure channel to user terminal emulation applications 110, or user terminal emulation applications identity learnt during AKMA authentication) is authorized to access services in the IOD Domain 410, and the target I/O user device specifically. If there are access control policies which indicate that the user and/or user terminal emulation applications 110 is not allowed, the controller function can operate to terminate the authentication and may provide some error code indicating the user and/or service is not authorized.
[00156]
[00157] 3GPP Key agreement method and related operations are now discussed below.
[00158] A 3 GPP key agreement function for providing a communication service through a user terminal emulation application to I/O user devices in an I/O user device domain 620 is now described. Various operations are described in the context of AKMA and GBA, but are not limited thereto. Figure 6 illustrates a combined flowchart of operations and related data flows between a user tag, I/O user devices, a 3GPP key agreement function system (which may be an AKMA system or Generic Bootstrapping Architecture system), and a user terminal emulation server 110 in accordance with some embodiments of the present disclosure. Figure 10 illustrates a flowchart of operations that may be performed by the user terminal emulation server 110 in accordance with some embodiments.
[00159] Referring to Figure 6, a user 601 selects a target I/O user device 602 and obtains therefrom an I/O user device identifier. For example, the user may scan an I/O user device identifier, such as by scanning a QR code, reading an identifier from a sticker on the I/O user device and entering the identifier into the user's own input device, using NFC or RFID to read the identifier from the I/O user device, etc. The user tag and I/O user device may include circuitry configured to utilize one or more communication protocols to communicate information described herein The I/O user device identifier can include an identifier of the I/O user device (IOD-ID) and an IOD Domain identifier, e.g. screenl23@myioddomain.com. [00160] The user 601 connects 603 to and authenticates to user terminal emulation application 110 (own cloud-based service). The I/O user device, or IOD Domain, may provide communication connectivity, e.g., over Bluetooth, Bluetooth Low Energy (BLE),
WiFi, NFC, RFID, etc. or the user device and/or user tag may be configured with its own communication connectivity. Once the user terminal emulation application 110 has authenticated user/tag, the user 601 provides the obtained IOD ID to user terminal emulation application 110.
[00161] The user terminal emulation application 110 generates mobile subscript on/SIM based credentials for use towards services by interacting 604 with the AKMA system 600 or a Generic Bootstrapping Architecture (GBA) function in the mobile operator. A result of the interaction 604 is that the user terminal emulation application 110 and the mobile operator, e.g., AKMA or GBA function, has a shared master AKMA secret key or shared master GBA secret key, and has an identifier for the context or key.
[00162] The user terminal emulation application 110 connects to the IOD Domain (e.g., AKMA Server) based on the I/O user device identifier (received in operation 603) realm part (e.g., myioddomain.com). The user terminal emulation application 110 derives an IOD Domain (e.g., AKMA Server 610) specific key from the AKMA or GBA master secret key. The user terminal emulation application 110 provides the AKMA or GBA context identifier to IOD Domain (e.g., AKMA Server 610). The IOD Domain (e.g., AKMA Server 610) uses the received AKMA or GBA context identifier to obtain IOD Domain specific AKMA or GBA key from the mobile operator AKMA or GBA function (e.g., AKMA Server 610), which may require pre-existing SLA/trust relationship between the IOD Domain and the mobile operator hosting the AKMA System. The user terminal emulation application 110 and the IOD Domain (e.g., AKMA Server 610) authenticate using the IOD Domain specific AKMA or GBA key, which may use a created secure session for further communication.
[00163] The user terminal emulation application 110 tells 605 the IOD Domain which I/O user device (e.g., screenl23) it wants to interact with, e.g., based on the I/O user device identifier received in operation 603. The user terminal emulation application 110 also provides a pointer to itself, e.g., IP address, FQDN, etc.
[00164] The IOD Domain 620 (e.g., AKMA Server 610) generates 606 an IOD specific AKMA or GBA session key from the IOD Domain specific AKMA or GBA key and the I/O user device identity, which may be similar to how in the EAP example above an IOD specific key is derived from EAP PMK and provides the IOD specific key to the I/O user device. The IOD Domain 620 (e.g., AKMA Server 610) may also provide a pointer to the user terminal emulation application 110, and an AKMA or GBA context identifier.
[00165] The I/O user device connects 607 to the user terminal emulation application 110 based on received info. The I/O user device indicates the AKMA or GBA context identifier
to the user terminal emulation application 110. The user terminal emulation application 110 can locate the AKMA or GBA context (e.g., AKMA or GBA master key). The I/O user device indicates its identity to the user terminal emulation application 110. The user terminal emulation application 110 can use the I/O user device identity together with the IOD Domain specific AKMA or GBA key to derive IOD specific AKMA or GBA session key. The user terminal emulation application 110 and the I/O user device can mutually authenticate using the I/O user device specific AKMA or GBA session key. The user terminal emulation application 110 and the I/O user device can use key to further establish a secure channel between themselves for streaming data for the communication service.
[00166] The operations by the user terminal emulation server 100 may more generally include, with reference to Figure 9, to authenticate 603 an identifier of the user tag or a user, receive 603 the identifier of the first I/O user device. The operations generate 604 and 605 the first I/O user device specific key through communications with a 3GPP key agreement function.
[00167] The key agreement function may include one of: an AKMA function; a GBA function; and a Battery Efficient Security for very low Throughput Machine Type Communication function.
[00168] The operation to generate 604 and 605 the first I/O user device specific key through communication with the key agreement function, may include to generate the first I/O user device specific key based on processing the master key derived through the key agreement function.
[00169] The operations may further include communicating with the 3 GPP key agreement function, e.g., AKMA function, hosted in a mobile operator system to generate a shared secret between the user terminal emulation server 100 and the key agreement function.
[00170]
[00171 ] Example I/O user device, user terminal emulation server, EAP authenticator or AKMA server, user tag, and IODH are now discussed below.
[00172] Figure 7 is a block diagram of hardware circuit components of an I/O user device 130 which are configured to operate in accordance with some embodiments. The I/O user device 130 can include a wired/wireless network interface circuit 702, a near field communication circuit 720, at least one processor circuit 700 (processor), and at least one memory circuit 710 (memory). The processor 700 is connected to communicate with the other components. The memory 710 stores program code (e.g., user terminal emulation application(s) 110) that is executed by the processor 700 to perform operations disclosed
herein. The processor 700 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor 700 is configured to execute the program code in the memory 710, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device. The I/O user device 130 can include one or more UI component devices, including without limitation, a microphone 740, a speaker 750, a camera 730, a display device 760, and a user input interface 770.
[00173] Figure 8 is a block diagram of hardware circuit components of a user terminal emulation server 100 and/or an I/O device handler (IODH) 212 which are configured to operate in accordance with some embodiments. The user terminal emulation server 100 and IODH 212 may reside on the same physical computing platform or may reside on different physical computing platforms which are communicatively networked together. The user terminal emulation server 100 and/or IODH 212 can include a wired/wireless network interface circuit 850, a database 120 (e.g., any one or more of a listing I/O user devices, UI capabilities of the I/O user devices, communication protocols used to communicate with the I/O user devices, known proximities to user identifiers, identifiers of user tags, I/O user device specific keys, session identifiers, etc.), a display device 830, a user input interface 840 (e.g., keyboard or touch sensitive display), at least one processor circuit 800 (processor), and at least one memory circuit 1220 (memory). The processor 800 is connected to communicate with the other components. The memory 1220 stores instructions that are executed by the processor 800 to perform operations disclosed herein. The processor 800 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor 800 is configured to execute computer program instructions in the memory 1220, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a user terminal emulation server and/or an IODH.
[00174] Figure 9 illustrates a block diagram of hardware circuit components of an EAP authenticator 400 or AKMA server 610 that are configured to operate in accordance with some embodiments of the present disclosure. The components can include a wired/wireless network interface circuit 950, a display device 960, a user input interface 970 (e.g., keyboard or touch sensitive display), at least one processor circuit 930 (processor), and at least one memory circuit 940 (memory). The processor 930 is connected to communicate with the
other components. The memory 940 stores program instructions that are executed by the processor 930 to perform operations disclosed herein. The processor 930 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor 930 is configured to execute computer program instructions in the memory 940, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a mobile electronic device. [00175] Figure 10 illustrates a block diagram of hardware circuit components of a core network node 1000 that are configured to operate in accordance with some embodiments of the present disclosure. The components can include a wired/wireless network interface circuit 1030, at least one processor circuit 1010 (processor), and at least one memory circuit 1020 (memory). The processor 1010 is connected to communicate with the other components. The memory 1020 stores program instructions that are executed by the processor 1000 to perform operations disclosed herein. The processor 1010 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor 1010 is configured to execute computer program instructions in the memory 1020, described below as a non-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a core network node.
[00176] Figure 11 illustrates a block diagram of hardware circuit components of a radio network node 1100 that are configured to operate in accordance with some embodiments of the present disclosure. The radio network node 1100 may correspond to, without limitation, an eNB, gNB, other 3GPP base station, WiFi access point, etc. The components can include a wired network interface circuit 1140, a wireless network interface circuit 1130, at least one processor circuit 1110 (processor), and at least one memory circuit 1120 (memory). The processor 1110 is connected to communicate with the other components. The memory 1120 stores program instructions that are executed by the processor 1110 to perform operations disclosed herein. The processor 1110 may include one or more data processing circuits (e.g., microprocessor and/or digital signal processor), which may be collocated or distributed across one or more data networks. The processor 1110 is configured to execute computer program instructions in the memory 1120, described below as anon-transitory computer readable medium, to perform some or all of the operations and methods for one or more of the embodiments disclosed herein for a radio network node.
[00177] Systems and operations for monitoring and enforcing privacy of communication services provided through user terminal emulation are now discussed below.
[00178] Through the user terminal emulation operations discussed above, communication services can simultaneously be provided to numerous people roaming in a disaggregated device environment of different types of I/O user devices where some of which can be spaced apart great distances and some of which can be proximately located to each other. It is possible that two or more users can be simultaneously engaged in separate communication services using I/O user devices which are proximately located to one another and/or using different I/O user interface functions on the same I/O user device. A consequence is that one user may overhear or observe information communicated from/to another user(s). Another consequence is there is a risk that a microphone, camera, or other UI capability of one of the proximately located I/O user devices may be undesirably used by anon-local person to monitor such communications by other users.
[00179] Various operational embodiments are therefore directed to monitoring and enforcing privacy of communication services which are provided through user terminal emulation involving I/O user devices.
[00180] Two different scenarios are now explained which have different corresponding operations by the IODH 212 to control session privacy.
[00181] The first scenario is discussed regarding the flowchart of Figure 13, which illustrates operations that can be performed by the IODH 212 to monitor and control privacy of communications through I/O user devices in accordance with some embodiments of the present disclosure. The second scenario is discussed regarding the flowchart of Figure 14, which illustrates other operations that can be performed by the IODH 212 to monitor and control privacy of communications through I/O user devices in accordance with some other embodiments of the present disclosure.
[00182] The IODH 212 in both Figures 13 and 14 obtains 1300 and 1400 a session privacy requirement for a communication session. Session privacy requirements may be specified by users, by communication service functions 140 (e.g., application endpoints which are communicating with a user) for a type of communication service, by network entities 150, etc. For example, the session privacy requirements may be determined for a communication session based on looking-up in the database 120 the session privacy requirement corresponding to a communication session for an identified type of communication service. [00183] The session privacy requirement for a session may be static while the session is ongoing or may be adjusted responsive to changes in the type of data being communicated
through the session, e.g., video, audio, text, etc., and/or may be adjusted responsive to privacy indications transmitted with the data. Thus, for example, transmission of data from a highly confidential file through the session may trigger an increase in the session privacy requirement. In another example, a videoconference meeting may have different privacy requirements tied to different parts of a meeting agenda, and where the privacy requirements can be indicated by signaling communicated from the videoconference service of the communication service function 140. A meeting participant may also define or trigger a privacy requirement to be applied.
[00184] In some embodiments, when the IODH 212 determines that the privacy requirement satisfies a rule, e.g., above a certain threshold, the IODH 212 determines a set of privacy-affecting UI capabilities of I/O user devices which are proximately located to one or more I/O user devices which are being used or would be used to provide user terminal emulation capabilities for a communication service.
[00185] When the privacy-affecting UI capabilities of the I/O user devices are determined to satisfy the privacy requirement (e.g., not presently allocated for use), the IODH 212 can proceed with allowing UI capabilities of one of the I/O user devices or another proximately located I/O user device to be used for the communication service. In contrast, when the privacy-affecting UI capabilities of the I/O user devices are determined to not satisfy the privacy requirement (e.g., are presently allocated for use), the IODH 212 may notify a user who is seeking access to the communication service of the non-compliant environment, and may query the user to provide authorization to proceed with establishing the communication service (via an UI output capability of one of the presently allocated or to-be allocated I/O user devices). In scenarios where the requesting user is moving and/or the I/O user devices are moving during a session, the operations can be repetitively performed to dynamically identify changes in the privacy-affecting UI capabilities and responsively handle as described herein.
[00186] Optionally, if any UI capability among the privacy-affecting UI capabilities is allocated to another user, the IODH 212 may check if both users are members of a relevant trust group(s) for the current content of one or both of the users’ sessions. A trust group may be defined as a group of users having the same access right to certain types of data, e.g., audio/ video media content.
[00187] Optionally, any non-allocated input capabilities are blocked from being allocated to users who are not present in the relevant trust group(s) for as long as the privacy conflicting I/O capability is allocated or until a timer reaches a threshold (or expires) since
the privacy conflict was identified. Embodiments disclosed herein may be used for handling privacy affecting input capabilities, privacy affecting output capabilities, and combined privacy affecting input and output capabilities.
[00188] Optionally, the type of capabilities which are included in the set of privacyaffecting UI capabilities, and thereby affected by the privacy requirements, may be determined by the IODH 212 based on the type of data communicated through a session. For example, if the data does not include audio, then microphone capabilities may not be excluded from the set of privacy-affecting UI capabilities.
[00189] In the first scenario of Figure 13, a second user is interfacing with the I/O UI capabilities of a second I/O user device to use a second communication service provided by a second communication service function through user terminal emulation via the second I/O user device. The IODH 212 receives a request for a communication session between a first I/O user device and a first communication service function. Based on the request, the IODH 212 determines 1302 whether UI capabilities of the first I/O user device do not satisfy a privacy-proximity rule defined for the second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O UI capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device.
[00190] The IODH 212 controls 1304 whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy-proximity rule.
[00191] The operation to determine whether the first I/O user device does not satisfy the privacy-proximity rule may include to determine 1302 privacy-affecting I/O user interface capabilities of the first I/O user device, and then determine whether any of the privacyaffecting I/O user interface capabilities of the first I/O user device which would be used for the communication session with the first I/O user device violate the session privacy requirement for the ongoing communication session of the second I/O user device.
[00192] In the second scenario of Figure 14, the privacy-proximity rule is tied to the first user (first I/O user device) who can be selectively blocked from obtaining a communication service using user terminal emulation depending upon whether the second I/O user device satisfies the privacy-proximity rule associated with the first user (first I/O user device). The second user is interfacing with the I/O UI capabilities of the second I/O user device to use a second communication service provided by a second communication service function through
user terminal emulation via the second I/O user device. The IODH 212 receives a request for a communication session between a first I/O user device and a first communication service function. Based on the request, the IODH 212 determines 1402 whether the second I/O user device does not satisfy a privacy-proximity rule defined for the first I/O user device based on proximate location relative to the first I/O user device and based on having a privacyaffecting I/O UI capability that does not satisfy the session privacy requirement for the ongoing communication session of the first I/O user device.
[00193] The IODH 212 then controls 1404 whether the communication session is setup and/or a communication capability of the communication session is allowed (i. e. , using one of the privacy-affecting I/O UI capabilities) between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule.
[00194] Other scenarios are also handled by the IODH 212. For example, assuming both sessions through the first and second I/O user devices are valid sessions and not malicious, the IODH 212 can operate to evaluate the session privacy requirements of both sessions when setup and while each session is ongoing to determine if one session should be paused or stopped, whether certain data type(s) should be prevented from being communicated through one of the sessions (e.g., prevent microphone data from a privacy-affecting microphone being transmitted from the first I/O user device, prevent video from a privacy-affecting camera from being transmitted from the first I/O user device, prevent video from being provided to the first I/O user device for display if risk of being seen by a privacy -affecting camera of another I/O user device, etc.). The privacy requirements of one or both sessions may be modified over time based on trustworthiness of the respective users, which privacy-affecting I/O UI capabilities of the I/O user devices are being used for communication session(s), level of secrecy of the type of data being communicated through the respective session, which I/O user devices remain proximately located to the user (user tag), etc. Sessions may have different assigned priorities (e.g., determined from the associated privacy requirements and / or user identities associated with each session), and the priorities may be used to determine which session is blocked or terminated if the two sessions should not be simultaneously conducted as determined from their session privacy requirements.
[00195] For example, an open session, i.e. no privacy restrictions in the session privacy requirement, may be ongoing through the second I/O user device, when a privacy restricted session is to begin through the first I/O user device. This privacy conflict may cause the IODH 212 to determine that a session through the first I/O user device should not be set up or
should be terminated, or that a privacy-affecting I/O UI capability should be blocked from being setup for use or if already in-use should be disabled from being used. Alternatively, if the first I/O user device has a higher priority (e.g., the first user has a higher priority designation than the second user), the IODH 212 may pause or terminate the session through the second I/O user device or disable a selected one of the privacy-affecting UI capabilities of the second I/O user device which prevent satisfaction of the privacy-proximity rule tied to the session privacy requirement of the first I/O user device.
[00196]
[00197] Monitoring and Controlling Privacy of Communications:
[00198] Operations for monitoring and controlling privacy of communications through I/O user devices with communication service functions are now described with regard to Figures 12A and 12B in accordance with some embodiments of the present disclosure.
[00199] Referring to Figures 12A and 12B, the illustrated scenario can correspond to a user who will be provided a media service (e.g., video conferencing) using UI capabilities (e.g., speaker, microphone, display device) of a first I/O user device, and where the session privacy requirement restricts use of any UI capabilities (e.g., microphone and camera) of other proximately located I/O user devices which affect privacy of the user, e.g., do not satisfy a privacy-proximity rule based on the session privacy requirement. In other scenarios, the restrictions may cover just selective input UI capabilities of one or more I/O user devices, both input and output UI capabilities, certain output UI capabilities or any combination thereof.
[00200] In step 1200, the first user’s position-indicating device (UT) 101 moves in proximity of UI capabilities of the first I/O user device (IOD-X) and performs authentication 1201 via the EAP authenticator 400 and is connected to the media server 140 (Fig. 1) via operations of the user terminal emulation server 110 and the IODH 212. It is noted that in at least some embodiments this operation would only be done if the UT 101 is not already authenticated to the IOD domain, i.e. if the UT 101 has not already ongoing session, or a context in IODH that has timed out. When UT 101 interacts with a first IOD in an IOD domain it will perform authentication. After that the IODH 212 will maintain a context for the authenticated UT 101, and serve any ongoing sessions. At some point if the UT 101 does not have ongoing sessions, the IODH 212 might decide to close the context. Alternatively, it might be that the IODH 212 maintains the context for as long as any of its IODS "sees/hears" the UT 101, e.g. hearing the Bluetooth beacon, which might occur until the user leaves the spatial area covered by the IOD domain.
[00201] Later, the first user triggers 1202 (explicitly by launching an application or implicitly by having I0D-X connect to the user terminal emulation server 110) a new media session with the user terminal emulation server 110 by indicating a capability of IOD-X that is requested to be added to the session (e.g., speaker, microphone, display device). This can be an initial step of connecting any IOD with the user terminal emulation server 110 and includes mutual authentication 1201 of the IOD and the user terminal emulation server 110, such as described above. When connecting to the user terminal emulation server 110 a media session is created. In addition, when launching an application in the user terminal emulation server 110, a separate or new media session is established. This session may overtake the original media session or there may be multiple media sessions open simultaneously.
[00202] In step 1203, before the IOD-X capability is connected to the session, the IODH 212 obtains session privacy requirements. The privacy requirements may be obtained from the media service 140 (Fig. 1, also “media server”), either continuously, responsive to session modification (e.g., launching new application), from user input via an IOD that is part of the session, or responsive to a request by the IODH 212. In some embodiments, the IODH 212 may also obtain trust groups for the first user, where members of a trust group can have the same right to view certain types of data (e.g., media content), such as described further below.
[00203] Accordingly, in the context of Figures 13 and 14, the operation to obtain 1300 and 1400 the session privacy requirement for the communication session, can be based on the request for the communication session identifying data content provided by the communication service function 140 (e.g., videoconference service), to send a message toward the communication service function requesting the session privacy requirement for the communication session. Session privacy requirements may alternatively be stored at the IODH 212.
[00204] In step 1204 the UT 101 periodically transmits beacon signals that are received by each of IOD-X, IOD-Y, and IOD-Z proximately located to UT 101. Content of the beacon signals can be verified by the EAP authenticator 400 and reported to the IODH 212, possibly via the EAP authenticator 400, or verified by the IODH 212. The IODH 212 thereby knows based on the beacon reports which IODS are proximately located to the UT 101 and therefore possibly available to provide UI capabilities for use by the first user to obtain the videoconference service from the communication service function 140.
[00205] In step 1205, the IODH 212 checks the obtained session privacy requirements. In one embodiment, if the session privacy requirements are above a threshold level, the session
is deemed as “privacy sensitive”. For example, the videoconference meeting request may identify the scheduled meeting as having a high privacy requirement. This triggers the IODH 212 to determine a set of potentially privacy conflicting capabilities in the vicinity of the IOD 101, this set is also referred to as “set of privacy-affecting UI capabilities”. The set of privacy-affecting UI capabilities may include, for example, cameras that could be used to watch meeting participants and/or their displays, and microphones which could be used to listen to audio communications among meeting participants.
[00206] The determination of which IODS are proximately located to the UT may be based at least one of:
1. a static table stored in the IODH 212 indicating which IODs are proximately located to each other;
2. reports received from the IODs indication receipt of the UT beacon of the user owning the session;
3. reports intermittently received from the IODs indication which other IOD(s) it observes (e.g., receiving beacon signals from other IOD(s), detecting machine readable identifiers from other IOD(s), etc.; and
4. positioning information reported by the IODs.
[00207] In one embodiment, the set of privacy-affecting UI capabilities and corresponding session privacy requirement is determined based on what type of data (e.g., video only, audio only, video and audio, sensor data, secrecy designation assigned to data, etc.) is communicated or will be communicated through the communication session with the first I/O user device.
[00208] In another embodiment, the operation to obtain 1203 (also 1300 in Fig. 13 and 1400 in Fig. 14) the session privacy requirement for the communication session includes to determine the session privacy requirement based on one of a privacy setting of a meeting invitation for an online meeting handled through the communication session with the first I/O user device, a privacy setting of a calendar booking for an online meeting handled through the communication session with the first I/O user device, or an interaction request for communication through the communication session.
[00209] In step 1206, the IODH 212 uses the session privacy requirement to determine whether the current allocation of privacy-affecting UI capabilities of IOD-X, IOD-Y, and IOD-Z are acceptable. This determination may include if some of privacy-affecting UI capabilities of the active IODs (e.g., communication endpoints of one or more other communication sessions) are allocated to some other UT, whether some privacy conflicting
of UI capabilities are allocated to trusted versus untrusted users, and/or determined through other operations.
[00210] For example, depending upon the session privacy requirement the I0DH 212 may determine that none of the privacy-affecting UI capabilities of IODS are allocated to an untrusted user or, in some embodiments, allocated to any other user. In a further embodiment, the IODH 212 determines if the other user is in a relevant trust group. The IODH 212 can perform further determinations based on whether any allocated UI capability of one of the IODs is in the set of privacy-affecting UI capabilities, such as determining whether that IOD is allocated to a trusted or untrusted user and/or other determinations as described further below.
[00211] In step 1207, if the result of the privacy determination (also 1302 in Fig. 13 and 1402 in Fig. 14) is not OK (NOK), e.g., does not satisfy the privacy-proximity rule, the IODH 212 may responsively not allocate the privacy-affecting UI capability (e.g., microphone, camera, etc.) to the session. As noted above, NOK may result if a privacy-affecting UI capability is determined to be allocated to an untrusted user. The IODH 212 may inform the user terminal emulation server 110 and/or IOD-X that the privacy-affecting UI capability should not be added to the session.
[00212] In an illustrative embodiment, the operation to control whether the communication session is setup between the first I/O user device (e.g., IOD-X) and the communication service function 140 based on whether the second I/O user device (e.g., IOD-Y) satisfies the privacy-proximity rule, can include the IODH 212 responding to a determination that UI capabilities of the second I/O user device (e.g., IOD-Y) satisfy the privacy-proximity rule, by performing operations to setup the communication session between the first I/O user device (e.g., IOD-X) and the user terminal emulation server 110, through which a communication service is setup with the communication service function 140, or perform operations to allow a communication capability of the communication session between the first I/O user device (e.g., IOD-X) and the user terminal emulation server 110. In contrast, the IODH 212 can respond to determining that the privacy -affecting UI capabilities of the second I/O user device (e.g., IOD-Y) do not satisfy the privacy-proximity rule, by operationally preventing setup of the communication session between the first I/O user device (e.g., IOD-X) and the user terminal emulation server 110 or blocking further use of one or more of the privacyaffecting UI capabilities of the second I/O user device (e.g., IOD-Y) that are conflicting with the privacy requirement of the first user’s session.
[00213] If the IODH 212 determines that the privacy-affecting UI capabilities of the second I/O user device (e.g., I0D-Y) partially-satisfies but does not entirely satisfy the privacy-proximity rule, the IODH 212 may responsively communicate a message toward the first I/O user device (e.g., IOD-X) containing a privacy warning notification and requesting authorization to proceed with setup of the communication session between the first I/O user device (e.g., IOD-X) and the user terminal emulation server 110. The IODH 212 may then control whether the communication session is setup between the first I/O user device (e.g., IOD-X) and the user terminal emulation server 110 responsive to whether a response message is received containing an indication of user authorization to proceed.
[00214] If the result of the privacy determination is uncertain, the IODH 212 can inform the user via an IOD and wait for receipt of a user authorization indication. An uncertain privacy determination may result when one of the proximate IODS is unresponsive and the allocation status therefore cannot be determined, and other scenarios resulting in uncertainty are discussed below. The IODH 212 can provide uncertainty information to the user either directly (e.g. IODH provides information to user directly via IOD(s) part of user’s session) or to the user terminal emulation server 110, which then provides the information to the user via current session. If user indicates authorization to proceed, then the IODH 212 may proceed according to step 1209 where the check is OK. Otherwise, if no authorization is received, the IODH 212 may proceed as NOK according to step 1207.
[00215] In step 1209, the result of the privacy determination is OK, and the IODH 212 informs 1210 the user terminal emulation server 110 and/or IOD-X to proceed with adding the privacy-affecting UI capability to the session.
[00216] In step 1211, the IODH 212 can monitor the privacy situation and block users with insufficient trust from being allocated privacy conflicting UI capabilities, and may update statuses of sessions based on the newly obtained changes to the privacy situation and/or changes to the privacy requirements (e.g., as the classification of data being communicated changes, etc.). In optional step 1212, the IODH 212 can repeat operations any or all of the operations, such as steps 1205 to 1211, to track and responsively handle changes in UT location, IODH and UI capability availability, etc.
[00217] These operations when performed in the scenario of Figures 13 and 14 can include the IODH 212 determining that the other I/O user device (e.g., IOD-Y and/or IOD-Z) satisfies the privacy-proximity rule responsive to receiving a message from the other I/O user device indicating that the other I/O user device received a beacon signal from a user tag of a user owning the communication session. Alternatively, the IODH 212 may determine that
the other I/O user device is proximately located to the first I/O user device (IOD-X) based on identifying the other I/O user device as a member of a table (stored in at least one memory of the IODH 212, the server 110, or elsewhere) that identifies which I/O user devices of the set are proximately located to the first I/O user device (IOD-X).
[00218] The operation by the IODH 212 to determine whether the other I/O user device (e.g., I0D-Y and/or IOD-Z) is proximately located to the first I/O user device (IOD-X) can include to obtain spatial coordinates of the I/O user devices, and determine whether the spatial coordinates of any of the other I/O user devices satisfy a rule relative to the spatial coordinates of the first I/O user device.
[00219] As explained, the privacy determination can be performed based on individual privacy-affecting I/O user interface capabilities of the I/O user devices. For example, the IODH 212 may determine the privacy-affecting I/O user interface capabilities of the other I/O user device (e.g., I0D-Y and/or IOD-Z), and determine whether any of those privacyaffecting I/O user interface capabilities violate the session privacy requirement for the communication session with the first I/O user device (IOD-X).
[00220] In step 1209, the IODH 212 may operate to continuously (e.g., periodically or responsive to events) monitor to ensure that the same privacy situation that was determined in step 1204 continues. For this reason, the IODH 212 may block all UI capabilities in the set of privacy-affecting UI capabilities from being allocated by an untrusted user. In some embodiments, the IODH 212 may further start a timer at this point, to determine when a time limit for when the user is no longer allowed to block capabilities, has been reached. The timer and associated operations are discussed further below.
[00221] In some embodiments, the privacy requirement is determined based on the classification of data communicated (e.g., media, source of data (e.g., microphone, video, ...), the confidentiality level of data communicated, and/or another defined characteristic of the data and/or the user who is involved in the communication session. For example, while a communication session remains setup between a first IOD (e.g., IOD-X) and the communication service function 140, the IODH 212 can operate to continuously determine whether a second user of the second IOD (e.g., IOD-Y) has access rights to data having a different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140. The IODH 212 can respond to determining that the second user who is attempting to setup a session or is involved in an ongoing session through a proximately located second IOD (e.g., IOD-Y) does not have the access rights to access the data having the different classification, by blocking an
I/O user interface capability of the second IOD (e.g., IOD-Y) from being accessed by the second user.
[00222] In some other embodiments, while a communication session remains setup between a first IOD (e.g., IOD-X) and the communication service function 140, the IODH 212 can operate to continuously determine whether a second user of the second IOD (e.g., IOD-Y) has access rights to the data having the different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140. The IODH 212 can respond to determining that the second user does not have the access rights to access the data having the different classification, and responsively send a message to the communication service function 140 instructing it to block sending of the data having the different classification to a first user of the first IOD (e.g., IOD-X).
[00223] In a further embodiment, the IODH 212 may rely on priorities associated with the first and second users to determine who or what to block I/O user interface capability(ies) from being accessed by the first and/or second users. For example, if the first user has priority over the second user (premium user, first to allocate, etc.), the IODH 212 may block the second user to protect privacy of communications with the first user. In one example, a corresponding operation by the IODH 212 can compare priority properties associated with the first and the second users when determining whether the second user of the second IOD (e.g., IOD-Y) has access rights to data having the different classification that is presently communicated through the communication session between the first IOD (e.g., IOD-X) and the communication service function 140 or, vice versa, whether the first user of the first IOD (e.g., IOD-X) has access rights to data having the different classification that is presently communicated through the communication session between the second IOD (e.g., IOD-Y) and the communication service function 140.
[00224] Fairness of access to the system may be ensured by the IODH 212 managing one or more timers which are used to limit how long a user can be blocked for obtaining access to a service via the user terminal emulation operations. For example, the IODH 212 may start a timer responsive to sending 1207 of a message to block. Thereafter, responsive to the timer expiring, the IODH 212 can re-evaluate the determination of whether the second user has access rights and perform one of a) send another message to the communication service function 140 instructing the cease blocking of the sending of the data having the different classification to the first user of the first IOD (e.g., IOD-X) and/or cease blocking use of
privacy-affecting UI capability for a service, and notify the first user, or b) allow the block to be continued.
[00225] In step 1210, when session privacy requirements are decreased or when the user deallocates the UI capability, an earlier block is removed. The privacy requirements may be decreased based on the IODH 212 obtaining information that the current media session now contains less sensitive content, e.g., user closing a document with higher privacy class “company confidential, restricted access” and opening another document with lower privacy class “company internal”.
[00226] Figures 12A and 12B are mostly directed to the scenario where a UT 101 is first connected to the user terminal emulation server 110 and then lODs are added to the session. Once the session has been established, the IODH 212 has learned the privacy requirements for the session and can use them when adding new lODs or privacy-affecting UI capabilities to the session, such as through the following steps. In one step the UT 101 enters proximity of a new IOD, or UT 101 signals the need for a new UI capability to be added to the session. IODH 212 learns this based on the beacons transmitted by UT 101 and reported by lODs in proximity. IODH 212 verifies whether the new IOD or UI capability can be added to the session based on the known privacy requirements. If IOD or UI capability can be added without violating the privacy requirements, the IODH 212 adds the IOD or UI capability to the session.
[00227] The privacy requirements can also be changed if the user, for example, launches a new application that has different privacy requirements than the current session. In this scenario, the user terminal emulation server 110 can operate to indicate the new privacy requirements to the IODH 212 before allowing the application to launch. IODH 212 verifies that the current IOD or UI capabilities allocations (to the UT 101 and also to other UTs in proximity) fulfills the new privacy requirements. If the requirements are met, IODH 212 informs the user terminal emulation server 110 that required privacy is available, after which the user terminal emulation server 110 can launch the application and/or session.
[00228] The privacy requirements for a session may be preconfigured. Preconfiguring of the privacy requirements may be based on information provided in a meeting invitation in a calendar application that indicates the privacy requirements for the meeting. The privacy requirements may be based on the entries in a phone book application which define per entry privacy requirements. Alternatively, the privacy requirements may be dynamically selected by the user for an ongoing session, such as by using the UI capabilities of one or more of the lODs connected to the media session to select the current privacy requirements. The user
terminal emulation server 110 can then use these requirements for the ongoing and future sessions as described earlier.
[00229]
[00230] Operations for detecting privacy-affecting UI capabilities outside of the IODH domain are now explained.
[00231] A user may utilize multiple IODS in a session where at least one of the IODS is owned by a different IODH than the others, e.g., where there exist multiple domains for IODs and where the IODs are controlled by different IODHS. In some embodiments, the IODs may be co-owned by two or more IODHs, requiring the IODHs to exchange UI capability allocation and status information among each other.
[00232] Accordingly a further embodiment assumes that a first IODH has an API used for communicating with at least one second IODH which directly controls at least one IOD (e.g., IOD-Y) in vicinity of an IOD (e.g., IOD-X) that is directly controlled by the first IODH. The IOD (e.g., IOD-Y) in proximity may be co-owned by the first and second IODH or owned exclusively by the second IODH. In this embodiment, the first IODH may store in a database information identifying the other IODs which are in vicinity of an IOD(s) controlled directly by the first IODH, but where the other IODs (“non-owned IODs”) are not directly controlled by the first IODH. Then, as a part of step 1206 of Figure 12A, the first IODH can request from the at least one second IODH the information regarding the current allocation of those non-owned IODs and status of their respective privacy -affecting UI capabilities.
Alternatively, or additionally, the UI capabilities of the non-owned IODs may be a part of the set of privacy-affecting UI capabilities obtained in step 1205.
[00233] The first IODH may await a response from the second IODH regarding the current allocation of privacy -affecting UI capabilities in the second IOD domain before evaluating the privacy. If there exists IODs (co-)owned by a second IODH in proximity of the first IOD and either the allocation in the other IODH domain is not known or if the users of that domain cannot be mapped to trust groups (see next section), the check may be determined as “uncertain”.
[00234]
[00235] Operations for managing privacy using trust groups are now explained.
[00236] In some embodiments, the IODH 212 has knowledge of trust groups, i.e., groups comprising two or more users where the members have the same right to view certain content. A user can belong to many different trust groups that are relevant for different types
of data communicated through a session, e.g., media content from the communication service function 140.
[00237] In some embodiments, if the check (in step 1206) determines that at least one privacy-affecting UI capability is allocated (used) by a second user, the identity of the second user is checked against the trust group that is relevant for the current type of data (e.g., media content) of the session. For example, if the first user’s media session contains a document with class "company confidential, restricted access”, the relevant trust group comprises other user’s with at least the same access rights to the document as the first user. In another example, some content may be identified as authorized to be accessed by anyone employed in a company, some other content may be identified as authorized to be accessed by only someone having the user’s role, and some other content may be identified as authorized for access by a set of the user’s colleagues. In these examples, the IODH 212 may make the first user a member of three different trust groups.
[00238] If the check 1206 determines that the second user is a part of the relevant trust group, the check returns OK 1209 (assuming no other privacy-affecting condition has been found). If the user is not a part of the relevant trust group, the check 1206 returns NOK 1207. [00239] In a corresponding operational embodiment, the IODH 212 when determining whether any of the privacy-affecting UI capabilities of the second I/O user device (e.g., IOD- Y) violate the session privacy requirement for the communication session with the first I/O user device (e.g., IOD-X), can perform operation to determine whether a first user of the first I/O user device (e.g., IOD-X) and a second user of the second I/O user device (e.g., IOD-Y) are members of a trust group relevant for the current data classification for the respective user’s session. Responsive to determining that the first and second users are members of the trust group, the IODH 212 may responsively determine that none of the privacy-affecting UI capabilities of the second I/O user device (e.g., IOD-Y) violate the session privacy requirement for the communication session with the first I/O user device (e.g., IOD-X).
[00240]
[00241] Operations for managing privacy using attestation of the set of privacy-affecting UI capabilities are now explained.
[00242] In some embodiments, the IODH 212 may require attestation of the set of privacyaffecting UI capabilities. For example, each IOD can provide to the IODH 212 a signed message with the state of respective UI capabilities. The signed message may include firmware version as well as hardware version. The signing can origin from a verifiable hardware root of trust, for instance a Trusted Execution Environment (TEE) or a Trusted
Platform Module (TPM), and the attestation of the same can be performed according to known methods as Internet Engineering Task Force (IETF) Remote ATtestation procedures (RATS).
[00243] If the attestation indicates that the firmware or hardware version is outdated (and thereby may indicate that the IOD could be exposed to actors with malicious intent), the check may be determined as “uncertain”.
[00244] In an example operational embodiment, the IODH 212 can receive a signed message from the second I/O user device (e.g., IOD-Y), wherein the signed message identifies the second I/O user device (e.g., IOD-Y), privacy-affecting I/O user interface capabilities, and at least one of firmware version and hardware version. The IODH 212 can then determine whether the second I/O user device satisfies the privacy -proximity rule further based on whether the signed message identifies acceptable privacy-affecting UI capabilities and acceptable version of the at least one of the firmware version and the hardware version.
[00245]
[00246] Operations for managing privacy responsive to detecting mobile IODS and user tags are now explained.
[00247] In some embodiments, when the IODs intermittently report other IODs that have been sensed to be proximately located thereto, the IODH 212 keeps track of the mobile IODs. If a mobile IOD enters proximity of the first IOD (e.g., IOD-X) and the UI capabilities are allocated to a user who is not a member of a relevant trust group, the IODH 212 may responsively inform that first user and may await receipt of the first user’s indicated authorization before letting the media session with the first user continue.
[00248] In one embodiment, in addition to monitoring IOD capabilities, the IODH 212 may further use the IODs to monitor for the user’s position-indicating device (UT 101), even in a situation where the user does not have any UI capability presently allocated.
[00249]
[00250] Operations for managing privacy while countering denial of service are now explained.
[00251] The privacy management can be deployed in environments where denial of service (DoS) is considered a threat vector, such as in a public setting where the users are not pre-vetted in contrast to employees in a corporate environment. In such an embodiment, the IODH 212 can operate to enforce a time limit (start a timer) on how long a user is allowed to block privacy-affecting UI capabilities (i.e., step 1209 in Fig. 12B). When the limit is reached (timer expires), the user may be informed that it is no longer allowed to block the UI
capabilities of another user and either be required to stop the media session, allow lower privacy that will stop blocking another user, or be required to approve updated billing (i.e., pay a premium for continued block).
[00252]
Further Definitions and Embodiments:
[00253] In the above-description of various embodiments of present inventive concepts, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of present inventive concepts. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which present inventive concepts belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense expressly so defined herein.
[00254] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and/or clarity. The term "and/or" includes any and all combinations of one or more of the associated listed items.
[00255] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements/operations, these elements/operations should not be limited by these terms. These terms are only used to distinguish one element/operation from another element/operation. Thus, a first element/operation in some embodiments could be termed a second element/operation in other embodiments without departing from the teachings of present inventive concepts. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.
[00256] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and
include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id est," may be used to specify a particular item from a more general recitation.
[00257] Example embodiments are described herein with reference to block diagrams and/or flowchart illustrations of computer-implemented methods, apparatus (systems and/or devices) and/or computer program products. It is understood that a block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and/or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions/acts specified in the block diagrams and/or flowchart block or blocks, and thereby create means (functionality) and/or structure for implementing the functions/acts specified in the block diagrams and/or flowchart block(s).
[00258] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer- readable medium produce an article of manufacture including instructions which implement the functions/acts specified in the block diagrams and/or flowchart block or blocks. Accordingly, embodiments of present inventive concepts may be embodied in hardware and/or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.
[00259] It should also be noted that in some alternate implementations, the functions/acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently
or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Moreover, the functionality of a given block of the flowcharts and/or block diagrams may be separated into multiple blocks and/or the functionality of two or more blocks of the flowcharts and/or block diagrams may be at least partially integrated. Finally, other blocks may be added/inserted between the blocks that are illustrated, and/or blocks/operations may be omitted without departing from the scope of inventive concepts. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.
[00260] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present inventive concepts. All such variations and modifications are intended to be included herein within the scope of present inventive concepts. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of present inventive concepts. Thus, to the maximum extent allowed by law, the scope of present inventive concepts is to be determined by the broadest permissible interpretation of the present disclosure including the following examples of embodiments and their equivalents, and shall not be restricted or limited by the foregoing detailed description.
Claims
1. An input and/or output, I/O, user device handler (212) comprising: at least one processor (800); and at least one memory (820) storing program code that is executable by the at least one processor (800) to perform operations comprising to: obtain a session privacy requirement for communication sessions; based on a request for the communication session between a first input and/or output, I/O, user device (130) of a set of I/O user devices served by the I/O user device handler and a communication service function of a network entity, determine whether the first I/O user device does not satisfy a privacy-proximity rule defined for a second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O user interface capability that does not satisfy the session privacy requirement for an ongoing communication session of the second I/O user device; and control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the first I/O user device satisfies the privacy-proximity rule.
2. The I/O user device handler (212) of Claim 1, wherein the operation to determine whether the first I/O user device does not satisfy a privacy-proximity rule defined for the second I/O user device based on location relative to the second I/O user device and based on having a privacy-affecting I/O user interface capability that does not satisfy the session privacy requirement for the ongoing communication session of the second I/O user device, comprises to: determine privacy -affecting I/O user interface capabilities of the first I/O user device; and determine whether any of the privacy-affecting I/O user interface capabilities of the first I/O user device which would be used for the communication session with the first I/O user device violate the session privacy requirement for the ongoing communication session of the second I/O user device.
3. An input and/or output, I/O, user device handler (212) comprising: at least one processor (800); and at least one memory (820) storing program code that is executable by the at least one processor (800) to perform operations comprising to: obtain a session privacy requirement for a communication session; based on a request for the communication session between a first input and/or output, I/O, user device (130) of a set of I/O user devices served by the I/O user device handler and a communication service function of a network entity, determine whether a second I/O user device of the set does not satisfy a privacy-proximity rule based on location relative to the first I/O user device and based on having a privacy-affecting I/O user interface capability that does not satisfy the session privacy requirement for the communication session with the first I/O user device; and control whether the communication session is setup and/or a communication capability of the communication session is allowed between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule.
4. The I/O user device handler (212) of Claim 3, wherein the operation to obtain the session privacy requirement for the communication session further comprises to: based on the request for the communication session identifying media content provided by the communication service function, send a message toward the communication service function requesting the session privacy requirement for the communication session.
5. The I/O user device handler (212) of any of Claims 3 to 4, wherein the operation to obtain the session privacy requirement for the communication session further comprises to: determine the session privacy requirement based on type of data that is communicated or will be communicated through the communication session with the first I/O user device.
6. The I/O user device handler (212) of any of Claims 3 to 5, wherein the operation to obtain the session privacy requirement for the communication session further comprises to:
determine the session privacy requirement based on one of a privacy setting of a meeting invitation for an online meeting handled through the communication session with the first I/O user device, a privacy setting of a calendar booking for an online meeting handled through the communication session with the first I/O user device, or an interaction request for communication through the communication session.
7. The I/O user device handler (212) of any of Claims 3 to 6, wherein the operation to determine whether a second I/O user device of the set satisfies the privacy-proximity rule based on location relative to the first I/O user device, comprises to: determine that the second I/O user device is proximately located to the first I/O user device based on receiving a message from the second I/O user device indicating that the second I/O user device received a beacon signal from a user tag of a user owning the communication session.
8. The I/O user device handler (212) of any of Claims 3 to 7, wherein the operation to determine whether a second I/O user device of the set satisfies the privacy-proximity rule based on location relative to the first I/O user device, comprises to: determine that the second I/O user device is proximately located to the first I/O user device based on identifying the second I/O user device as a member of a table stored in the at least one memory that identifies which I/O user devices of the set are proximately located to the first I/O user device.
9. The I/O user device handler (212) of any of Claims 3 to 8, wherein the operation to determine whether a second I/O user device of the set satisfies the privacy-proximity rule based on location relative to the first I/O user device, comprises to: obtain spatial coordinates of the I/O user devices of the set; and determine whether the spatial coordinates of any of the other I/O user devices of the set satisfy a rule relative to the spatial coordinates of the first I/O user device.
10. The I/O user device handler (212) of any of Claims 3 to 9, wherein the operation to determine whether a second I/O user device of the set does not satisfy the privacy-proximity rule based on having a privacy-affecting I/O user interface capability that does not satisfy the
session privacy requirement for the communication session with the first I/O user device, comprises to: determine privacy -affecting I/O user interface capabilities of the second I/O user device; and determine whether any of the privacy-affecting I/O user interface capabilities of the second I/O user device which would be used for the communication session violate the session privacy requirement for the communication session with the first I/O user device.
11. The I/O user device handler (212) of Claim 10, wherein the operation to determine whether any of the privacy-affecting I/O user interface capabilities of the second I/O user device violate the session privacy requirement for the communication session with the first I/O user device, comprises to: determine whether a first user of the first I/O user device and a second user of the second I/O user device are members of a trust group; and responsive to determining that the first and second users are members of the trust group, determine that none of the privacy-affecting I/O user interface capabilities of the second I/O user device violate the session privacy requirement for the communication session with the first I/O user device.
12. The I/O user device handler (212) of Claim 10, wherein the operation to determine whether any of the privacy-affecting I/O user interface capabilities of the second I/O user device violate the session privacy requirement for the communication session with the first I/O user device, comprises to: determine whether a second user of the second I/O user device has access rights to a type of data that is communicated or will be communicated through the communication session between the first I/O user device and the communication service function; and responsive to determining that the second user has the access rights to the type of data that is communicated or will be communicated through the communication session between the first I/O user device and the communication service function, determine that none of the privacy-affecting I/O user interface capabilities of the second I/O user device violate the session privacy
requirement for the communication session between the first I/O user device and the communication service function.
13. The I/O user device handler (212) of Claim 12, wherein the operations further comprise to: while the communication session remains setup between the first I/O user device and the communication service function, continuously determine whether the second user of the other I/O user device has access rights to data having a different classification that is presently communicated through the communication session between the first I/O user device and the communication service function, and responsive to determining that the second user does not have the access rights to access the data having the different classification , block a privacyaffecting I/O user interface capability of the second I/O user device from being accessed by the other user.
14. The I/O user device handler (212) of Claim 12, wherein the operations further comprise to: while the communication session remains setup between the first I/O user device and the communication service function, continuously determine whether the second user of the second I/O user device has access rights to new data having a different classification that is presently communicated through the communication session between the first I/O user device and the communication service function, and responsive to determining that the second user does not have the access rights to access the data having the different classification , send a message to the communication service function instructing to block sending of the data having the different classification to a first user of the first I/O user device.
15. The I/O user device handler (212) of any of Claims 13 and 14, wherein the determination of whether the second user of the second I/O user device has access rights to the data having the different classification that is presently communicated through the communication session between the first I/O user device and the communication service
function, is performed based on comparison of priority properties associated with the first and the second users.
16. The I/O user device handler (212) of any of Claims 13 and 14, wherein the operations further comprise to: start a timer responsive to the sending of the message to block; and responsive to the timer expiring, re-evaluate the determination of whether the second user has access rights and perform one of a) send another message to the communication service function instructing the cease blocking of the sending of the data having the different classification to the first user of the first I/O user device, and notify the first user, or b) allow the block to be continued.
17. The I/O user device handler (212) of any of Claims 3 to 16, wherein the operation to control whether the communication session is setup between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule, further comprises to: responsive to determining that UI capabilities of the second I/O user device satisfy the privacy-proximity rule, perform operations to setup the communication session between the first I/O user device and the communication service function or perform operations to allow a communication capability of the communication session between the first I/O user device and the communication service function; and responsive to determining that the UI capabilities of the second I/O user device do not satisfy the privacy-proximity rule, operationally prevent setup of the communication session between the first I/O user device and the communication service function.
18. The I/O user device handler (212) of any of Claims 3 to 17, wherein the operation to control whether the communication session is setup between the first I/O user device and the communication service function based on whether the second I/O user device satisfies the privacy-proximity rule, further comprises to: responsive to determining that the UI capabilities of the second I/O user device partially-satisfies but do not entirely satisfy the privacy-proximity rule,
communicate a message toward the first I/O user device containing a privacy warning notification and requesting authorization to proceed with setup of the communication session between the first I/O user device and the communication service function; and control whether the communication session is setup between the first I/O user device and the communication service function responsive to whether a response message is received containing an indication of user authorization to proceed.
19. The I/O user device handler (212) of any of Claims 3 to 18, wherein the operations further comprise to: communicate with another I/O user device handler to identify whether a third I/O user device that is managed by the other I/O user device handler satisfies the privacyproximity rule based on location relative to the first I/O user device and based on having an I/O user interface capability that does not satisfy the session privacy requirement for the communication session with the first I/O user device.
20. The I/O user device handler (212) of any of Claims 3 to 19, wherein the operations further comprise to: receive a signed message from the second I/O user device, wherein the signed message identifies the second I/O user device, privacy-affecting I/O user interface capabilities, and at least one of firmware version and hardware version, wherein the operation to determine whether the second I/O user device satisfies the privacy-proximity rule is further performed based on whether the signed message identifies acceptable privacy-affecting I/O user interface capabilities and acceptable version of the at least one of the firmware version and the hardware version.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/EP2023/087339 WO2025131286A1 (en) | 2023-12-21 | 2023-12-21 | Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/EP2023/087339 WO2025131286A1 (en) | 2023-12-21 | 2023-12-21 | Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025131286A1 true WO2025131286A1 (en) | 2025-06-26 |
Family
ID=89542194
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/EP2023/087339 Pending WO2025131286A1 (en) | 2023-12-21 | 2023-12-21 | Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2025131286A1 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8594727B2 (en) * | 2010-09-13 | 2013-11-26 | Ricoh Co., Ltd. | Mobile device input/output interface expansion device and system having the same |
| US20200220914A1 (en) * | 2019-01-04 | 2020-07-09 | Apple Inc. | User interfaces for content streaming |
| WO2023208354A1 (en) * | 2022-04-28 | 2023-11-02 | Telefonaktiebolaget Lm Ericsson (Publ) | Authentication of user tags obtaining communication services via i/o user devices performing user terminal emulation as a cloud computing service |
-
2023
- 2023-12-21 WO PCT/EP2023/087339 patent/WO2025131286A1/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8594727B2 (en) * | 2010-09-13 | 2013-11-26 | Ricoh Co., Ltd. | Mobile device input/output interface expansion device and system having the same |
| US20200220914A1 (en) * | 2019-01-04 | 2020-07-09 | Apple Inc. | User interfaces for content streaming |
| WO2023208354A1 (en) * | 2022-04-28 | 2023-11-02 | Telefonaktiebolaget Lm Ericsson (Publ) | Authentication of user tags obtaining communication services via i/o user devices performing user terminal emulation as a cloud computing service |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12009940B2 (en) | Providing communication services using sets of I/O devices | |
| CN103259775B (en) | Device association via video handshake | |
| US20130339464A1 (en) | Contact and identity management in a heterogeneous network with disparate clients | |
| US11190438B2 (en) | Twinning service for groups of internet of things (IOT) devices | |
| US11063990B2 (en) | Originating caller verification via insertion of an attestation parameter | |
| US9344417B2 (en) | Authentication method and system | |
| US20220225095A1 (en) | External Authentication Method, Communication Apparatus, and Communication System | |
| WO2012087419A2 (en) | Session management for communication in a heterogeneous network | |
| EP4515897B1 (en) | Authentication of user tags obtaining communication services via i/o user devices performing user terminal emulation as a cloud computing service | |
| WO2016015509A1 (en) | Method and device for terminal authentication for use in mobile communication system | |
| US10425812B2 (en) | Method and apparatus for establishment of private communication between devices | |
| WO2024074193A1 (en) | User tags beacon-based mobility management for communication services via i/o user devices performing user terminal emulation as a cloud computing service | |
| WO2025131286A1 (en) | Controlling privacy of communication services through i/o user devices performing user terminal emulation as a cloud computing service | |
| Nguyen et al. | An SDN‐based connectivity control system for Wi‐Fi devices | |
| WO2024199678A1 (en) | Secure allocation of a user terminal emulator for authenticated user who is registered to another user terminal emulator | |
| WO2025216670A1 (en) | Providing user-subscriber profile for i/o user devices performing user terminal emulation as a cloud computing service | |
| WO2024132173A1 (en) | Configuring discontinuous transmission of beacon signal by user tag for providing communication services via user terminal emulation server | |
| WO2024186238A1 (en) | Radio cell load balancing for communication services via i/o user devices performing user terminal emulation as a cloud computing service | |
| WO2024156342A1 (en) | A method and devices for enabling safe data transfer between input/output device handlers | |
| WO2025209969A1 (en) | Enhanced metaverse communication | |
| CN119449338A (en) | Method and device for calling service API |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23838024 Country of ref document: EP Kind code of ref document: A1 |