WO2025071578A1

WO2025071578A1 - Leveraging inactive touchpoints for gesture recognition and user interface configuration

Info

Publication number: WO2025071578A1
Application number: PCT/US2023/034225
Authority: WO
Inventors: Aleks ROZMAN; Kristin Gray; Michael E. BENTZ II; Philip Quinn; Lu SILVERSTEIN
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2023-09-29
Filing date: 2023-09-29
Publication date: 2025-04-03
Anticipated expiration: 2026-03-29

Abstract

A method comprises detecting, by a presence-sensitive housing, one or more user inputs. The method further comprises generating, by a computing device, a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected. The method further comprises generating, by the computing device, a model of a hand by at least applying a machine learning model to the map of the one or more user inputs. The method further comprises removing, by the computing device and using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs. The method further comprises determining, by the computing device and based on the filtered set of user inputs, the gesture input.

Description

LEVERAGING INACTIVE TOUCHPOINTS FOR GESTURE RECOGNITION AND USER INTERFACE CONFIGURATION

BACKGROUND

[0001] Some computing devices may include touch or other presence sensors that detect user input. For example, a computing device may include a presence-sensitive display that both displays objects and detects user input using presence sensors. Such a presence-sensitive display may enable a user to interact with the computing device by, for example, tapping on a displayed element to initiate an action (e.g., initiating execution of an application, changing information displayed by the presence-sensitive display, etc.), moving a finger across the presence-sensitive display to drag an object and/or cause the computing device to scroll content being displayed by the presence-sensitive display, etc. However, providing input at the presence-sensitive display may visually obstruct the information being displayed, which may make it more difficult for a user to interact with the computing device.

SUMMARY

[0002] In general, techniques of this disclosure are directed to techniques for identifying a gesture performed at a presence-sensitive housing of a computing device using both active and inactive touchpoints detected by the presence-sensitive housing. A computing device may include a presence-sensitive housing that detects inputs at multiple locations outside of the area of a presence-sensitive display. When a user holds the computing device by the presence-sensitive housing, the computing device detects the presence of parts of the user’s hand or other input that mimics the user’s hand such as a user’s chin or nose. The computing device may struggle to determine which inputs detected by the presence-sensitive housing should be used to control the computing device.

[0003] In accordance with the techniques of the present disclosure, a computing device uses a machine learning model, such as a federated machine learning model, to generate a model of the user’s hand based on a map of inputs detected by the presence-sensitive housing. The computing device uses the model of the hand and the federated machine learning model to identify which of the detected inputs should be used to identify a gesture input being performed by the user. Based on the identified gesture input, the computing device performs one or more actions such as adjusting a volume setting, scrubbing video playback, and modifying a graphical user interface of the computing device (e.g., portrait vs landscape mode, placement of elements within the graphical user interface, etc.), among other actions. [0004] In this way, various aspects of the techniques of this disclosure may enable a computing device to determine which inputs are part of a gesture input and filter out various inputs determined to be false inputs generated from a user holding or adjusting their grip on the computing device. Further, various aspects of this disclosure enable the computing device to provide a more consistent user experience and avoid frustrating the user by mistakenly- recognizing unintentional inputs as gesture inputs.

[0005] In one example, a method includes detecting, by a presence-sensitive housing, one or more user inputs. The method may further include generating, by a computing device, a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected. The method may further include generating, by the computing device, a model of a hand by at least applying a machine learning model to the map of the one or more user inputs. The method may further include removing, by the computing device and using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs. The method may further include determining, by the computing device and based on the filtered set of user inputs, the gesture input.

[0006] In another example, a computing device includes a memory; and one or more programmable processors in communication with the memory, and configured to detect one or more user inputs. The one or more programmable processors may be further configured to generate a map of the one or more user inputs, wherein the map indicates respective locations at which each of the one or more user inputs was detected. The one or more programmable processors may be further configured to generate a model of a hand by at least applying a machine learning model to the map of the one or more user inputs. The one or more programmable processors may be further configured to remove, using the model of the hand, at least one user input that is not consistent with a gesture input form the one or more user inputs to generate a filtered set of user inputs. The one or more programmable processors may be further configured to determine, based on the filtered set of user inputs, the gesture input.

[0007] In another example, a non-transitory computer-readable storage medium is encoded with instructions that, when executed by the one or more processors, cause the one or more processors to detect one or more user inputs. The instructions are further configured to generate a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected. The instructions are further configured to generate a model of a hand by at least applying a machine learning model to the map of the one or more user inputs. The instructions are further configured to remove, using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs. The instructions are further configured to determine, based on the filtered set of user inputs, the user input.

[0008] The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

[0009] FIG. 1 is a conceptual diagram illustrating an example computing device including a presence-sensitive housing, in accordance with one or more aspects of the present disclosure. [0010] FIG. 2 is a block diagram illustrating further details of a computing device that performs gesture detection, in accordance with one or more techniques of this disclosure.

[0011] FIG. 3 is a conceptual diagram illustrating example self-capacitance data generated by a presence-sensitive housing of a computing device, in accordance with one or more techniques of this disclosure.

[0012] FIG. 4 is a conceptual diagram illustrating example mutual-capacitance data generated by a presence-sensitive housing of a computing device, in accordance with one or more techniques of this disclosure.

[0013] FIG. 5 is a cropped set of mutual-capacitance data, in accordance with one or more techniques of this disclosure.

[0014] FIG. 6 is a flowchart illustrating example operations of an example computing device for outputting a graphical user interface, in accordance with one or more techniques of the present disclosure.

[0015] FIG. 7 is a flowchart illustrating example operations of an example computing device for identifying user gestures, in accordance with one or more techniques of the present disclosure.

DETAILED DESCRIPTION

[0016] FIG. 1 is a conceptual diagram illustrating an example computing device 102 including a presence-sensitive housing 112. FIG. 1 illustrates only one particular example of computing device 102 and many other examples of computing device 102 may be used in other instances. In the example of FIG. 1, computing deGee 102 may be a wearable computing device, a mobile computing device, or any other computing device capable of receiving user input. Computing device 102 of FIG. 1 may include a subset of the components included in example computing device 102 or may include additional components not shown in FIG. 1 . In the example of FIG. 1 , computing device 102 is being held by hand 120 of a user of computing device 102.

[0017] In the example of FIG. 1, computing device 102 can be a mobile phone. However, computing device 102 may also be any other type of computing device such as a camera device, a tablet computer, a personal digital assistant (PDA), a smart speaker, a laptop computer, a desktop computer, a gaming system, a media player, an e-book reader, a television platform, an automobile navigation system, or a wearable computing device (e.g., a computerized watch). In addition, computing device 102 may be a computing device such as a digital whiteboard. As shown in FIG. 1, computing device 102 includes user interface components (UIC) 110, UI module 114, user application module 116, and processor(s) 122. [0018] UIC 110 may function as an input and/or output device for computing device 102. As shown in FIG. I, UIC 110 includes presence-sensitive housing 112. UIC 110 may be implemented using various tec Imo logics. For instance, UIC 110 may include input devices such a presence-sensitive housing and/or one or more presence-sensitive screens, such as capacitive touchscreens or projective capacitance touchscreens. UIC 110 may include a presence-sensitive housing such as a capacitive housing. UIC 110 may also function as output (e.g., display) devices using any one or more display devices, such as liquid crystal displays (LCD), dot matrix displays, light emitting diode (LED) displays, organic light-emitting diode (OLED) displays, e-ink, or similar monochrome or color displays capable of outputting visible information to a user of computing device 102. In the example of FIG. 1, presence- sensitive housing 112 may be a presence-sensitive display capable of receiving touch input from a user of computing device 102.

[0019] UIC 110 may detect input (e.g., touch and non-touch input) from a user of respective computing device 102. For instance, presence-sensitive housing 112 may detect indications of input by detecting one or more gestures from a user (e.g., the user touching, pointing, and/or swiping at or near one or more locations of presence-sensitive housing 112 with a finger or a stylus pen). UIC 110 may output information to a user in the form of a user interface, which may be associated with functionality provided by computing device 102. Such user interfaces may be associated with computing platforms, operating systems, applications, and/or services executing at or accessible from computing device 102 (e.g., electronic message applications, chat applications, Internet browser applications, mobile or desktop operating systems, social media applications, electronic games, menus, and other types of applications).

[0020] UI module 114 manages user interactions with UIC 110 and other components of computing device 102. In other words, UI module 114 may act as an intermediary between various components of computing device 102 to make determinations based on user input detected by UIC 110 and generate output at UIC 110 in response to the user input. UIC module 114 may receive instructions from an application, service, platform, or other module of computing device 102 to cause UIC 110 to output a user interface. UI module 114 may manage inputs received by computing device 102 as a user views and interacts with the user interface presented at UIC 110 and may update the user interface in response to receiving additional instructions from the application, sendee, platform, or other module of computing device 102 that is processing the user input.

[0021] Computing device 102 may include UI module 114 and user application module 116. UI module 114 and user application module 116 may perform operations described using software, hardware, firmware, or a mixture of hardware, software, and/or firmware residing in and/or executing at computing device 102. Computing device 102 may execute UI module 114 and user application module 116 with one or more processors. Computing device 102 may execute UI module 114 and user application module 116 as a virtual machine executing on underlying hardware. UI module 114 and user application module 116 may execute as services or components of an operating system or computing platform. UI module 114 and user application module 116 may execute as one or more executable programs at an application layer of a computing platform. UI module 114 and user application module 116 may be otherwise arranged remotely to and remotely accessible to computing device 102, for instance, as one or more network services operating at a network in a network cloud.

[0022] Processor(s) 122 may implement functionality and/or execute instructions within computing device 102. Examples of processors ) 122 include, but are not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein.

[0023] User application module 116 may execute at computing device 102 to perform any of a variety of operations. Examples of user application module 116 include, but are not limited to, music applications, photo viewing applications, mapping applications, electronic message applications, chat applications, Internet browser applications, social media applications, electronic games, menus, and/or other types of applications that may operate based on user input.

[0024] In operation, user application module 116 may cause UI module 114 to generate a graphical user interface (GUI) for display at UIC 110. UIC 110 may output a graphical user interface based on instructions from user application module 116. In one example, user application module 116 may be a photo viewing application that causes UI module 114 to generate a GUI including a picture of a mountain for display at UIC 110.

[0025] A user may desire to provide user input to user application module 116. For instance, where user application module 116 may be a photo viewing application, the user may provide gesture input at UIC 110 in the form of a swipe to swipe between photos of the photo viewing application or may provide gesture input at UIC 110 to select a photo album for viewing. Presence-sensitive housing 112 may detect the gesture input. For instance, where presence- sensitive housing 112 is a capacitive-touch panel, presence-sensitive housing 112 may detect the gesture input via use of one or more capacitive touch sensors of presence-sensitive housing 112.

[0026] Presence-sensitive housing 112 may wrap around the exterior of computing device 102 such that presence-sensitive housing may detect user input at any point of the back (opposing side of computing device 102 from the side of display 106) and sides of the exterior of computing device 102. As an example, presence-sensitive housing 112 may receive touch input from a user but may not require physical contact between the user and presence-sensitive housing 112 to detect the user input. Presence-sensitive housing 112 may detect user input within a certain distance from presence-sensitive housing 112. Presence- sensitive housing 112 may be located on the sides, rear (side of computing device 102 opposing a display of computing device 102), and front of computing device 102. Presence- sensitive housing 112 may “wrap” around computing device 102 to enable the detection of user input at any point of the exterior of computing device 102. For example, presence- sensitive housing 112 may detect user touch input at any location along the back (i. e. , the side of computing device 102 opposite that of display 106) or sides of computing device 102.

[0027] Presence-sensitive housing 112 may detect user input at multiple locations at presence-sensitive housing 112. For example, the touch sensor of presence-sensitive housing 112 may detect user input from hand 120 at a plurality of touch sensor cells, and may generate a corresponding capacitance value at each touch sensor cell that detects the user input. In the example of FIG. 1, presence-sensitive housing 112 detects user input at five locations. Presence-sensitive housing 112 may detect user input from the fingers of a user’s hand such as hand 120 as illustrated in FIG. 1. Presence-sensitive housing 112 also may detect user input from the palm of hand 120.

[0028] Computing device 102 determines input points 124 A through 124E (hereinafter “input points 124”) from corresponding heatmap of input regions 118A through 118E (hereinafter “input regions 118”). A user may typically provide a gesture input in the form of user input at presence-sensitive housing 112 by moving an input object (e.g., the user’s finger, a stylus, etc.) near presence-sensitive housing 112 or by touching presence-sensitive housing 112 using the input object. Because the size of an input object used to provide the touch input may be larger than the size of a single touch point of presence-sensitive housing 112, the capacitive touch sensor of presence-sensitive housing 112 may, when the user provides user input at presence-sensitive housing 112, sense the user input at a plurality of touch locations. Computing device 102 determines input points 124 from each of input regions 118 that may encompass a region of presence-sensitive housing 112. For example, computing device 102 determines each of input points 124 based on locations of maximal touch intensity within each of input regions 118. In another example, computing device 102 determines each of input points 124 by determining a centroid of each of input regions 118.

[0029] Computing device 102 may generate a model of the hand based on the heatmap, with each input point of input points 124 assigned to a finger of the model of the hand (e.g., each of input points 124 assigned to a finger of a hand) or the palm of the model of the hand. Computing device 102 uses the federated machine learning model to generate the model of the hand based on the heatmap (e.g., uses the federated machine learning model to identify which input points from the heatmap should be assigned to a respective finger of a hand). Computing device 102 may use the federated machine learning model to determine input points from the heatmap that are consistent with the palm of hand 120.

[0030] Computing device 102 may use the model of the hand to filter false input points (e.g., from unintentional user input such as a user adjusting their grip on computing device 102) to identify gesture inputs. In an example, computing device 102 uses the model of the hand to determine that a user is swiping their pointer finger (e.g., the finger associated with input point 124B) and thus determine that the user is providing a gesture input using their pointer finger. Computing device 102 may improve the user experience of interacting with computing device 102 via presence-sensitive housing 112 and reduce the incidence of incorrectly recognizing false (e.g., unintentional) input as user input. [0031] Computing device 102 may use the model of the hand to generate a graphical user interface that is output by display 106. Computing device 102 may use the model of the hand to generate a graphical user interface with the locations of various visual elements within the graphical user interface ad justed based on the model of the hand (e.g., placing regularly selected visual elements within reach of a finger, adjusting the visual locations of the visual elements based on the size of the model of the hand, etc.). Computing device 102 causes display 106 to output the graphical user interface generated by computing device 102.

[0032] The techniques of this disclosure include one or more advantages. For example, computing device 102 may leverage a federated machine learning model to improve identification of gesture inputs and remove unintentional inputs. Computing device 102 may use the improved identification of gesture inputs to provide a better user experience of providing input via presence-sensitive housing 112 and avoid frustrating the user by mistakenly identifying unintentional inputs as gesture inputs. In another example, computing device 102 may improve the user experience of interacting with computing device 102 by enabling a user to use a variety of gesture inputs to interact with computing device 102 via presence-sensitive housing 112.

[0033] FIG. 2 is a block diagram illustrating further details of a computing device 202 that performs gesture detection, in accordance with one or more techniques of this disclosure. Computing device 202 of FIG. 2 is described below as an example of computing device 102 illustrated in FIG. 1. FIG. 2 illustrates only one particular example of computing device 202, and many other examples of computing device 202 may be used in other instances and may include a subset of the components included in example computing device 202 or may include additional components not shown in FIG. 2. In addition, for the purpose of clarify, computing device 202 is described below with the side of computing device 202 including display 232 referred to as the “front” of computing device 202, and the location of the other components of computing device 202 are referred to with respect to display 232 located on the “front” side of computing device 202.

[0034] Computing device 202 may be one or more types of computing devices. Computing device 202 may be a mobile computing device such as a smartphone, smartwatch, tablet computer, laptop computer, etc. Computing device 202 may alternatively be a computing device such as a television with a presence-sensitive display, an all-in-one computing device, a desktop computer, a digital whiteboard, or other computing device.

[0035] As shown in FIG. 2, computing device 202 may include presence-sensitive housing 212, processor(s) 222, one or more input components 242, one or more communication units 244, one or more output components 246, one or more storage components 248, and display 232. Presence-sensitive housing 212 may include electrodes 224 and touch controller 226. As also shown in FIG. 2, one or more storage components 248 may include UI module 214, user application module 216, operating system 236, and touch mapping module 252.

[0036] Communication channels 250 may interconnect each of the components 212, 214, 222, 242, 244, 246, and 248 for inter-component communications (physically, communicatively, and/or operatively). In some examples, communication channels 250 may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.

[0037] One or more input components 242 of computing device 202 may receive input. Examples of input are tactile, audio, and video input. Input components 242 of computing device 202, in one example, includes a presence-sensitive display, touch-sensitive screen, mouse, keyboard, voice responsive system, video camera, microphone or any other type of device for detecting input from a human or machine.

[0038] One or more input components 242 include one or more sensors. Numerous examples of sensors exist and include any input component configured to obtain environmental information about the circumstances surrounding computing device 202 and/or physiological information that defines the activity state and/or physical well-being of a user of computing device 202. In some examples, a sensor may be an input component that obtains physical position, movement, and/or location information of computing device 202. For instance, sensors may include one or more location sensors (GPS components, Wi-Fi components, cellular components), one or more temperature sensors, one or more motion sensors (e.g., multi-axial accelerometers, gyros), one or more pressure sensors (e.g., barometer), one or more ambient light sensors, and one or more other sensors (e.g., microphone, camera, infrared proximity sensor, hygrometer, and the like). Other sensors may include a heart rate sensor, magnetometer, glucose sensor, hygrometer sensor, olfactory sensor, compass sensor, step counter sensor, to name a few other non-limiting examples.

[0039] One or more output components 246 of computing device 202 may generate output. Examples of output are tactile, audio, and video output. Output components 246 of computing device 202, in one example, includes a presence-sensitive display, sound card, video graphics adapter card, speaker, liquid crystal display (LCD), or any other type of device for generating output to a human or machine. In another example, output components 246 may interact with display controller 230 to provide video output via display 232. [0040] One or more communication units 244 of computing device 202 may communicate with external devices via one or more wired and/or wireless networks by transmitting and/or receiving network signals on the one or more networks. Examples of communication units 244 include a network interface card (e.g. such as an Ethernet card), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device that can send and/or receive information. Other examples of communication units 244 may include short wave radios, cellular data radios, wireless network radios, as well as universal serial bus (USB) controllers.

[0041] Processor(s) 222 may implement functionality and/or execute instructions within computing device 202. For example, processor(s) 222 may receive and execute instructions that provide the functionality of UI module 214, user application module 216, operating system 236, touch mapping module 252, and federated ML module 218. These instructions executed by processor(s) 222 may cause computing device 202 to store and/or modify information within one or more storage components 248 of processor(s) 222 during program execution. Processor(s) 222 may execute instructions of UI module 214, user application module 216, operating system 236, touch mapping module 252, and federated ML module 218 to perform one or more operations. That is, UI module 214, user application module 216, operating system 236, and federated ML module 218 may be operable by processor(s) 222 to perform various functions described herein.

[0042] One or more storage components 248 within computing device 202 may store information for processing during operation of computing device 202 (e.g., computing device 202 may store data accessed by UI module 214, user application module 216, operating system 236, and touch mapping module 252 during execution at computing device 202). In some examples, storage component 248 is a temporary memory, meaning that a primary' purpose of storage component 248 is not long-term storage. Storage components 248 on computing device 202 may be configured for short-term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art.

[0043] Storage components 248, in some examples, also include one or more computer- readable storage media. Storage components 248 may be configured to store larger amounts of information than volatile memory. Storage components 248 may further be configured for long-term storage of information as non-volatile memory space and retain information after power on/off cycles. Examples of non-volatile memories include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. Storage components 248 may store program instructions and/or information (e.g., data) associated with UI module 214, user application module 216, operating system 236, touch mapping module 252, and federated ML module 218.

[0044] Operating system 236, or a component thereof, may manage interactions between applications and a user of computing device 202. As shown in the example of FIG. 2, operating system 236 may manage operations between user application module 216 and a user of computing device 202. In some examples, UI module 214 may be considered to be a component of operating system 236. In some examples, operating system 236, or a component thereof, may manage user input data received from presence-sensitive housing 212.

[0045] Presence-sensitive housing 212 of computing device 202 includes electrodes 224 and touch controller 226. Electrodes 224 may form a matrix of row and column electrodes on either side of a dielectric material. Electrodes 224 may be integrated into a layer comprising the exterior housing of the back and sides of computing device 202.

[0046] Presence-sensitive housing 212 may “wrap” around the exterior of computing device 202 such that, combined with the touch-sensitive component of display 232, computing device 202 may detect user input at any exterior location on computing device 202. As presence-sensitive housing 212 may compose much of the exterior of computing device 202, electrodes 224 may also wrap around the back and sides of computing device 202 in addition to the portion of the front of computing device 202 not occupied by display 232.

[0047] Presence-sensitive housing 212 may detect a user input using one or both of a self- capacitance scan and mutual-capacitance scan. In particular, electrodes 224 and touch controller 226 may collectively operate to generate mutual-capacitance data based on a mutual-capacitance scan and/or to generate self-capacitance data based on a self-capacitance scan. Presence-sensitive housing 212 may provide data from touch controller 226 to one or more components of computing device 202 such as processor(s) 222 and storage components 248.

[0048] Touch controller 226 may perform one or more operations to sense user input via electrodes 224. For instance, touch controller 226 may output a voltage signal across the electrodes and sense a resultant change in capacitance (e.g., as induced by the presence of a finger or other object on or near presence-sensitive housing 212). When an electrode of electrodes 224 is driven with a voltage signal, the electrode’s inherent capacitance to other objects can be measured (such as a human finger, another electrode, or ground). Changes to the surrounding environment affect changes in the inherent capacitance of the electrode. Touch controller 226 may output an indication of the sensed user input to one or more other components of computing device 202, such as touch mapping module 252.

[0049] Touch mapping module 252 may map user input data generated by a touch sensor (e.g., one or more of electrodes 224) of presence-sensitive housing 212 to an input location at the presence-sensitive housing 212, as described in further detail below. In some examples, touch mapping module 252 may be considered to be a component of operating system 236. [0050] Touch mapping module 252 may process the user input data generated by presence- sensitive housing 212 to generate a map of the user input. As presence-sensitive housing 212 may wap around the edges of computing device 202, touch mapping module 252 may process the user input to derive a two-dimensional map from a three-dimensional map of the user input.

[0051] Touch mapping module 252 may identify user input points based on the two- dimensional map of the user input. Touch mapping module 252 may identify user input points based on maximal points of input intensity. Additionally, touch mapping module 252 may identify input points using a machine learning model such as federated ML module 218. [0052] Federated ML module 218 may include one or more machine learning models on- device for identifying user input points and generating models of hands based on the identified input points. Federated ML module 218 may process user input, such as the map of user input points, and generate a hand model based on the user input points. Federated ML module 218 may generate the hand model such that it includes a model of the arcs of the fingers of the hand.

[0053] Federated ML module 218 may include one or more machine learning models that are trained using training data received from other computing devices that is compiled by another computing device or system such as a cloud computing system associated with the manufacturer of computing device 202. For example, federated ML module 218 may be trained by computing device 202 with training data that is generated by a plurality of other computing devices, compiled by a computing system, and provided to computing device 202. [0054] Computing device 202 may train federated ML module 218. Computing device 202 may obtain a plurality of prior user inputs that are each associated with a plurality of hand models obtained by computing device 202, where each map of prior user input is associated with a respective hand model. Computing device 202 may obtain the plurality of maps of prior user input from other computing devices as example maps for training federated ML module 218. Computing device 202 provides the plurality of maps of prior user input and associated hand models to federated ML module 218 for processing by federated ML module 218. Computing device 202, based on comparing the output of federated ML module 218 to the plurality of hand models, updates federated ML module 218. Computing device 202 may update the weights of federated ML module 218 based on a comparison of the output of federated ML module 218 to the plurality of hand models. Computing device 202 may modify one or more layers of federated ML module 218.

[0055] Federated ML module 218 may be or include one or more of various different types of machine-learned models. In particular, in some implementations, federated ML module 218 can perform classification, regression, clustering, anomaly detection, recommendation generation, and/or other tasks. In some implementations, federated ML module 218 can perform various types of classification based on the input data. For example, federated ML module 218 can perform binary classification or multiclass classification. In binary classification, the output data can include a classification of the input data into one of two different classes. In multiclass classification, the output data can include a classification of the input data into one (or more) of more than two classes. The classifications can be single label or multi-label. Federated ML module 218 may perform discrete categorical classification in which the input data is simply classified into one or more classes or categories.

[0056] In some implementations, federated ML module 218 can perform classification in which federated ML module 218 provides, for each of one or more classes, a numerical value descriptive of a degree to which it is believed that the input data should be classified into the corresponding class. In some instances, the numerical values provided by federated ML module 218 can be referred to as “confidence scores” that are indicative of a respective confidence associated with classification of the input into the respective class. In some implementations, the confidence scores can be compared to one or more thresholds to render a discrete categorical prediction. In some implementations, only a certain number of classes (e.g., one) with the relatively largest confidence scores can be selected to render a discrete categorical prediction.

[0057] Federated ML module 218 may output a probabilistic classification. For example, federated ML module 218 may predict, given a sample input, a probability distribution over a set of classes. Thus, rather than outputting only the most likely class to which the sample input should belong, federated ML module 218 can output, for each class, a probability that the sample input belongs to such class. In some implementations, the probability distribution over all possible classes can sum to one. In some implementations, a Softmax function, or other type of function or layer can be used to squash a set of real values respectively associated with the possible classes to a set of real values in the range (0, 1) that sum to one. [0058] In some examples, the probabilities provided by the probability distribution can be compared to one or more thresholds to render a discrete categorical prediction. In some implementations, only a certain number of classes (e.g., one) with the relatively largest predicted probability can be selected to render a discrete categorical prediction.

[0059] In cases in which federated ML module 218 performs classification, federated ML module 218 may be trained using supervised learning techniques. For example, federated ML module 218 may be trained on a training dataset that includes training examples labeled as belonging (or not belonging) to one or more classes.

[0060] In some implementations, federated ML module 218 can perform regression to provide output data in the form of a continuous numeric value. The continuous numeric value can correspond to any number of different metrics or numeric representations, including, for example, currency values, scores, or other numeric representations. As examples, federated ML module 218 can perform linear regression, polynomial regression, or nonlinear regression. As examples, federated ML module 218 can perform simple regression or multiple regression.

[0061] Federated ML module 218 may perform various types of clustering. For example, federated ML module 218 can identify one or more previously-defined clusters to which the input data most likely corresponds. Federated ML module 218 may identify one or more clusters within the input data. That is, in instances in which the input data includes multiple objects, documents, or other entities, federated ML module 218 can sort the multiple entities included in the input data into a number of clusters. In some implementations in which federated ML module 218 performs clustering, federated ML module 218 can be trained using unsupervised learning techniques.

[0062] Federated ML module 218 may perform anomaly detection or outlier detection. For example, federated ML module 218 can identify input data that does not conform to an expected pattern or other characteristic (e.g., as previously observed from previous input data). As examples, the anomaly detection can be used for fraud detection or system failure detection. [0063] Federated ML module 218 may, in some cases, act as an agent within an environment. For example, federated ML module 218 can be trained using reinforcement learning, which will be discussed in further detail below.

[0064] In some implementations, federated ML module 218 can be a parametric model while, in other implementations, federated ML module 218 can be a non-parametric model. In some implementations, federated ML module 218 can be a linear model while, in other implementations, federated ML module 218 can be a non-linear model.

[0065] As described above, federated ML module 218 can be or include one or more of various different types of machine-learned models. Examples of such different types of machine-learned models are provided below for illustration. One or more of the example models described below can be used (e.g., combined) to provide the output data in response to the input data. Additional models beyond the example models provided below can be used as well.

[0066] In some implementations, federated ML module 218 can be or include one or more classifier models such as, for example, linear classification models; quadratic classification models; etc. Federated ML module 218 may be or include one or more regression models such as, for example, simple linear regression models; multiple linear regression models; logistic regression models; stepwise regression models; multivariate adaptive regression splines; locally estimated scatterplot smoothing models; etc.

[0067] In some examples, federated ML module 218 can be or include one or more decision tree-based models such as, for example, classification and/or regression trees; iterative dichotomiser 3 decision trees; C4.5 decision trees; chi-squared automatic interaction detection decision trees; decision stumps; conditional decision trees; etc.

[0068] Federated ML module 218 may be or include one or more kernel machines. In some implementations, federated ML module 218 can be or include one or more support vector machines. Federated ML module 218 may be or include one or more instance-based learning models such as, for example, learning vector quantization models; self- organizing map models; locally weighted learning models; etc. In some implementations, federated ML module 218 can be or include one or more nearest neighbor models such as, for example, k- nearest neighbor classifications models; k- nearest neighbors regression models; etc.

Federated ML module 218 can be or include one or more Bayesian models such as, for example, naive Bayes models; Gaussian naive Bayes models; multinomial naive Bayes models; averaged one-dependence estimators; Bayesian networks; Bayesian belief networks; hidden Markov models; etc. [0069] In some implementations, federated ML module 218 can be or include one or more artificial neural networks (also referred to simply as neural networks). A neural network can include a group of connected nodes, which also can be referred to as neurons or perceptrons. A neural network can be organized into one or more layers. Neural networks that include multiple layers can be referred to as “deep” networks. A deep network can include an input layer, an output layer, and one or more hidden layers positioned between the input layer and the output layer. The nodes of the neural network can be connected or non-fully connected. [0070] Federated ML module 218 can be or include one or more feed forward neural networks. In feed forward networks, the connections between nodes do not form a cycle. For example, each connection can connect a node from an earlier layer to a node from a later layer.

[0071] In some instances, federated ML module 218 can be or include one or more recurrent neural networks. In some instances, at least some of the nodes of a recurrent neural network can form a cycle. Recurrent neural networks can be especially useful for processing input data that is sequential in nature. In particular, in some instances, a recurrent neural network can pass or retain information from a previous portion of the input data sequence to a subsequent portion of the input data sequence through the use of recurrent or directed cyclical node connections.

[0072] In some examples, sequential input data can include time-series data (e.g., sensor data versus time or imagery' captured at different times). For example, a recurrent neural network can analyze sensor data versus time to detect or predict a swipe direction, to perform handwriting recognition, etc. Sequential input data may include words in a sentence (e.g., for natural language processing, speech detection or processing, etc.); notes in a musical composition; sequential actions taken by a user (e.g., to detect or predict sequential application usage); sequential object states; etc.

[0073] Example recurrent neural networks include long short-term (LSTM) recurrent neural networks; gated recurrent units; bi-direction recurrent neural networks; continuous time recurrent neural networks; neural history compressors; echo state networks; Elman networks; Jordan networks; recursive neural networks; Hopfield networks; fully recurrent networks; sequence-to- sequence configurations; etc.

[0074] In some implementations, federated ML module 218 can be or include one or more convolutional neural networks. In some instances, a convolutional neural network can include one or more convolutional layers that perform convolutions over input data using learned filters. [0075] Filters can also be referred to as kernels. Convolutional neural networks can be especially useful for vision problems such as when the input data includes imagery such as still images or video. However, convolutional neural networks can also be applied for natural language processing.

[0076] In some examples, federated ML module 218 can be or include one or more generative networks such as, for example, generative adversarial networks. Generative networks can be used to generate new data such as new images or other content.

[0077] Federated ML module 2 i 8 may be or include an autoencoder. In some instances, the aim of an autoencoder is to learn a representation (e.g., a lower- dimensional encoding) for a set of data, typically for the purpose of dimensionality reduction. For example, in some instances, an autoencoder can seek to encode the input data and provide output data that reconstructs the input data from the encoding. Recently, the autoencoder concept has become more widely used for learning generative models of data. In some instances, the autoencoder can include additional losses beyond reconstructing the input data.

[0078] Federated ML module 218 may be or include one or more other forms of artificial neural networks such as, for example, deep Boltzmann machines; deep belief networks; stacked autoencoders; etc. Any of the neural networks described herein can be combined (e.g., stacked) to form more complex networks.

[0079] One or more neural networks can be used to provide an embedding based on the input data. For example, the embedding can be a representation of knowledge abstracted from the input data into one or more learned dimensions. In some instances, embeddings can be a useful source for identifying related entities. In some instances, embeddings can be extracted from the output of the network, while in other instances embeddings can be extracted from any hidden node or layer of the network (e.g., a close to final but not final layer of the network). Embeddings can be useful for performing auto suggest next video, product suggestion, entity or object recognition, etc. In some instances, embeddings can be useful inputs for downstream models. For example, embeddings can be useful to generalize input data (e.g., search queries) for a downstream model or processing system.

[0080] Federated ML module 218 may include one or more clustering models such as, for example, k-means clustering models; k-medians clustering models; expectation maximization models; hierarchical clustering models; etc.

[0081] In some implementations, federated ML module 218 can perform one or more dimensionality reduction techniques such as, for example, principal component analysis; kernel principal component analysis; graph-based kernel principal component analysis; principal component regression; partial least squares regression; Sammon mapping; multidimensional scaling; projection pursuit; linear discriminant analysis; mixture discriminant analysis; quadratic discriminant analysis; generalized discriminant analysis; flexible discriminant analysis; autoencoding; etc.

[0082] In some implementations, federated ML module 218 can perform or be subjected to one or more reinforcement learning techniques such as Markov decision processes; dynamic programming; Q functions or Q-leaming; value function approaches; deep Q-networks; differentiable neural computers; asynchronous advantage actor-critics; deterministic policy gradient; etc.

[0083] In some implementations, federated ML module 218 can be an autoregressive model. In some instances, an autoregressive model can specify that the output data depends linearly on its own previous values and on a stochastic term. In some instances, an autoregressive model can take the form of a stochastic difference equation. One example autoregressive model is WaveNet, which is a generative model for raw audio.

[0084] In some implementations, federated ML module 218 can include or form part of a multiple model ensemble. As one example, bootstrap aggregating can be performed, which can also be referred to as “bagging.” In bootstrap aggregating, a training dataset is split into a number of subsets (e.g., through random sampling with replacement) and a plurality of models are respectively trained on the number of subsets. At inference time, respective outputs of the plurality of models can be combined (e.g., through averaging, voting, or other techniques) and used as the output of the ensemble.

[0085] One example ensemble is a random forest, which can also be referred to as a random decision forest. Random forests are an ensemble learning method for classification, regression, and other tasks. Random forests are generated by producing a plurality of decision trees at training time. In some instances, at inference time, the class that is the mode of the classes (classification) or the mean prediction (regression) of the individual trees can be used as the output of the forest. Random decision forests can correct for decision trees' tendency to overfit their training set.

[0086] Another example ensemble technique is stacking, which can, in some instances, be referred to as stacked generalization. Stacking includes training a combiner model to blend or otherwise combine the predictions of several other machine-learned models. Thus, a plurality of machine-learned models (e.g., of same or different type) can be trained based on training data. In addition, a combiner model can be trained to take the predictions from the other machine-learned models as inputs and, in response, produce a final inference or prediction. In some instances, a single-layer logistic regression model can be used as the combiner model. [0087] Another example ensemble technique is boosting. Boosting can include incrementally building an ensemble by iteratively training weak models and then adding to a final strong model. For example, in some instances, each new model can be trained to emphasize the training examples that previous models misinterpreted (e.g., misclassified). For example, a weight associated w’ith each of such misinterpreted examples can be increased. One common implementation of boosting is AdaBoost, which can also be referred to as Adaptive Boosting. Other example boosting techniques include LPBoost, TotalBoost; BrownBoost; xgboost; MadaBoost, LogitBoost, gradient boosting: etc. Furthermore, any of the models described above (e.g., regression models and artificial neural networks) can be combined to form an ensemble. As an example, an ensemble can include a top level machine-learned model or a heuristic function to combine and/or weight the outputs of the models that form the ensemble. [0088] In some implementations, multiple machine-learned models (e.g., that form an ensemble can be linked and trained jointly (e.g., through backpropagation of errors sequentially through the model ensemble). However, in some implementations, only a subset (e.g., one) of the jointly trained models is used for inference.

[0089] In some implementations, federated ML module 218 can be used to preprocess the input data for subsequent input into another model. For example, federated ML module 218 can perform dimensionality reduction techniques and embeddings (e.g., matrix factorization, principal components analysis, singular value decomposition, word2vec/GLOVE, and/or related approaches); clustering; and even classification and regression for downstream consumption. Many of these techniques have been discussed above and will be further discussed below.

[0090] As discussed above, federated ML module 218 can be trained or otherwise configured to receive the input data and, in response, provide the output data. The input data can include different types, forms, or variations of input data. As examples, in various implementations, the input data can include features that describe the content (or portion of content) initially selected by the user, e.g., content of user-selected document or image, links pointing to the user selection, links within the user selection relating to other files available on device or cloud, metadata of user selection, etc. Additionally, with user permission, the input data includes the context of user usage, either obtained from the app itself or from other sources. Examples of usage context include breadth of share (sharing publicly, or with a large group, or privately, or a specific person), context of share, etc. When permitted by the user, additional input data can include the state of the device, e.g., the location of the device, the apps running on the device, etc.

[0091] In some implementations, federated ML module 218 can receive and use the input data in its raw form. In some implementations, the raw input data can be preprocessed. Thus, in addition or alternatively to the raw input data, federated ML module 218 can receive and use the preprocessed input data.

[0092] In some implementations, preprocessing the input data can include extracting one or more additional features from the raw input data. For example, feature extraction techniques can be applied to the input data to generate one or more new, additional features. Example feature extraction techniques include edge detection; corner detection; blob detection; ridge detection; scale-invariant feature transform; motion detection; optical flow; Hough transform; etc.

[0093] In some implementations, the extracted features can include or be derived from transformations of the input data into other domains and/or dimensions. As an example, the extracted features can include or be derived from transformations of the input data into the frequency domain. For example, wavelet transformations and/or fast Fourier transforms can be performed on the input data to generate additional features.

[0094] In some implementations, the extracted features can include statistics calculated from the input data or certain portions or dimensions of the input data. Example statistics include the mode, mean, maximum, minimum, or other metrics of the input data or portions thereof. [0095] In some implementations, as described above, the input data can be sequential in nature. In some instances, the sequential input data can be generated by sampling or otherwise segmenting a stream of input data. As one example, frames can be extracted from a video. In some implementations, sequential data can be made non-sequential through summarization.

[0096] As another example preprocessing technique, portions of the input data can be imputed. For example, additional synthetic input data can be generated through interpolation and/or extrapolation.

[0097] As another example preprocessing technique, some or all of the input data can be scaled, standardized, normalized, generalized, and/or regularized. Example regularization techniques include ridge regression; least absolute shrinkage and selection operator (LASSO); elastic net; least-angle regression; cross-validation; LI regularization; L2 regularization; etc. As one example, some or all of the input data can be normalized by subtracting the mean across a given dimension's feature values from each individual feature value and then dividing by the standard deviation or other metric.

[0098] As another example preprocessing technique, some or all or the input data can be quantized or discretized. In some cases, qualitative features or variables included in the input data can be converted to quantitative features or variables. For example, one hot encoding can be performed.

[0099] In some examples, dimensionality reduction techniques can be applied to the input data prior to input into federated ML module 218. Several examples of dimensionality reduction techniques are provided above, including, for example, principal component analysis; kernel principal component analysis; graph-based kernel principal component analysis; principal component regression; partial least squares regression; Sammon mapping; multidimensional scaling; projection pursuit; linear discriminant analysis; mixture discriminant analysis; quadratic discriminant analysis; generalized discriminant analysis; flexible discriminant analysis: autoencoding; etc.

[0100] In some implementations, during training, the input data can be intentionally deformed in any number of ways to increase model robustness, generalization, or other qualities. Example techniques to deform the input data include adding noise; changing color, shade, or hue; magnification; segmentation; amplification; etc.

[0101] In response to receipt of the input data, federated ML module 218 can provide the output data. The output data can include different types, forms, or variations of output data. As examples, in various implementations, the output data can include content, either stored locally on the user device or in the cloud, that is relevantly shareable along with the initial content selection.

[0102] Federated ML module 218 may generate output data such as a model of a hand (which may also be referred to as a “hand model”) by associating a finger with each of the identified input points. In an example, federated ML module 218 identifies five input points of presence-sensitive housing 212. Federated ML module 218 assigns a finger to each of the five input points. Based on the relative location of each of the fingers assigned to the input points, federated ML module 218 generates a model of a hand. In addition, federated ML module 218 may identify an input point associated with a palm. For example, touch mapping module 252 may receive data regarding user input generated by presence-sensitive housing 212 and identify a comparatively large area of user input located near an input point associated with a thumb. Federated ML module 218 may determine that the large comparatively input point is 1 ike ly part of the palm of the hand. [0103] Federated ML module 2 i 8 may use the model of the hand to filter touch false input points from a 10-point collection of input points, such as a user holding computing device 202 with both hands. In an example, touch mapping module 252 receives input data from presence-sensitive housing 212 consistent with a user double-tapping on the right side of computing device 202. Federated ML module 218, based on the model of the hand, filters out input received on the left side of computing device 202, as any input detected on the left side of computing device 202 is likely from the user adjusting their grip and not part of the double-tap input. In another example, display 232 displays a graphical user interface that includes a widget for a user of computing device 202 to slide. Touch mapping module 252, based on the displaying of the widget by display 232, causes federated ML module 218 to filter out user inputs consistent with a user re-gripping computing device 202 to slide the widget. Touch mapping module 252 may use federated ML module 218 to improve the accuracy of touch identification, as well as to filter out inactive user inputs (e.g., a user gripping computing device 202, minor movements of one or more touch points, etc.).

[0104] Touch mapping module 252 may use federated ML module 218 to differentiate between user input and inactive touches such as a user holding computing device 202. Federated ML module 218 may use the relative location of the fingers of the hand to determine whether a user is providing a touch gesture to computing device 202. In addition, federated ML module 218 may filter out relatively minor touch input such as involuntary finger movements or touch input consistent with a user shifting their grip on computing device 202. Federated ML module 218 may filter out touch inputs that are below a predetermined threshold of change in location over a predetermined period of time (e.g., a touch input must move 1cm in less than 2 seconds to be registered).

[0105] Touch mapping module 252 may leverage federated ML module 218 to avoid disabling presence-sensitive housing when an inactive touch is identified. For example, touch mapping module 252 may use federated ML module 218 to determine that one or more inputs to presence-sensitive housing 212 are the result of water droplets touching presence-sensitive housing 212. Touch mapping module 252, rather than disabling the zones of presence- sensitive housing 212 that detect the water droplets or providing an alert to the user, uses federated ML module 218 to remove the inactive inputs that result from the water droplets touching presence-sensitive housing.

[0106] Touch mapping module 252 may use federated ML module 218 to determine a difference between one or more user inputs and previous user inputs. Touch mapping module 252 may determine whether the difference satisfies a threshold difference. Touch mapping module 252, responsive to determining that the difference satisfies the threshold difference, removes the user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs.

[0107] Touch mapping module 252 may use federated ML module 218 to determine that a user of computing device 202 has adjusted their grip on computing device and predict an intent of the user’s input. Federated ML module 218 may predict the user intent based on a pattern of the index finger and palm of the model of the hand. Federated ML module 218, based on presence-sensitive housing 212 receiving user input consistent with a user adjusting their grip on computing device 202, determines that the user is about to provide touch input. Federated ML module 218 may predict the intent of the user by identifying specific fingers and their respective location holding computing device 202. Federated ML module 218 may identify the specific fingers based on an index pattern and palm patterns. Touch mapping module 252 may use the predicted user intent to reduce accidental recognition of unintentional input.

[0108] Touch mapping module 252 may use federated ML module 218 may determine whether computing device 202 is being held in a hand such as a hand of the user or whether computing device 202 is located in an item of clothing such as a pocket. Touch mapping module 252 may use federated ML module 218 to determine whether computing device 202 is located in an item of clothing rather than in a hand. Based on federated ML module 218 determining that computing device 202 is located in an item of clothing, touch mapping module 252 may cause computing device 202 to adjust a haptics intensity setting. For example, touch mapping module 252 may cause computing device 202 to increase the strength of the haptics intensity setting to ensure that a user of computing device 202 can still feel the haptics through the item of clothing.

[0109] Touch mapping module 252 may use federated ML module 218 to avoid requiring a user of computing device 202 to adjust their manner of interacting with computing device 202. In an example, computing device 200 includes a relatively large area where a user can provide input (e.g., a digital whiteboard). Touch mapping module 252 may determine that a user is leaning their body against the input area of computing device 200 and uses federated ML module 218 to filter the inactive inputs resulting from the user leaning their body against computing device 200 while drawing in an input area of computing device 200. Touch mapping module 252 may use federated ML module 218 to avoid requiring the user to adjust their stance and stop leaning against computing device 200 to provide input, or to treat the area the user is leaning against as a deadzone and drop all inputs resulting from that area. Touch mapping module 252 may use federated ML module 218 to avoid requiring the user to change their manner of input and to maximize the signal for detecting touch inputs.

[0110] Touch mapping module 252 may use federated ML module 218 to identify gesture inputs. Federated ML module 218 may, using the model of the hand, identify one or more gesture inputs. For example, federated ML module 218 may identify a gesture input of a user swiping their finger along the side of presence-sensitive housing 212. Touch mapping module 252, responsive to federated ML module 218 identifying a gesture input, may cause UI module 214 to generate a GUI based on the identified gesture input.

[0111] Touch mapping module 252 may use UI module 214 as an intermediary between various components of computing device 202 such as a haptic motor to generate output at the user interface component in response to the user input. As shown in FIG. 2, UI module 214 may include display driver 234. Display driver 234 may interact with display controller 230, display touch controller 260, and operating system 236 to process output for display at display panel 228, which may be altered based on user input received via electrodes 224 or display 232.

[0112] UI module 214 may generate a graphical user interface for display by output component 246. UI module 214 may generate a graphical user interface that includes one or more visual elements. UI module 214 may generate graphical user interfaces that include visual rearrangements of the one or more visual elements. UI module 214 may generate a graphical user interface based on the identification of touch input or patterns of touch input by touch mapping module 252 and/or federated ML module 218. For example, based on receiving user input consistent with the selection of a visual element corresponding to an application of computing device 202, UI module 214 may generate a graphical user interface that includes the application displayed in a full screen orientation.

[0113] UI module 214 may adjust the orientation of the graphical user interface based on receiving a determination from touch mapping module 252 that computing device 202 is being held in a changed orientation. Touch mapping module 252 may determine, based solely on touch inputs or in addition to other inputs such as from an accelerometer, that computing device 202 is being held in a changed orientation (e.g., changed from portrait to landscape, landscape to portrait, etc.). Touch mapping module 252 may determine the change in orientation based on a model of a hand and the position of the hand on computing device 202. UI module 214, based on the touch mapping module 252 determining that the orientation of computing device 202 has changed, generates a graphical user interface consistent with the changed orientation of computing device 202. [0114] UI module 214 may generate a graphical user interface that includes one or more visual elements adjusted based on the size of a model of a hand. UI module 214 may generate a graphical user interface that includes one or more visual elements adjusted from a prior graphical user interface based on touch mapping module 252 determining that a hand different from a first hand is holding computing device 202. Touch mapping module 252 determines that the second hand is different from a first hand based on a model of the second hand.

[0115] UI module 214 generates a graphical user interface based on a model of a hand. UI module 214 may generate a graphical user interface based on determining a relative location of a thumb of a model of a hand. In an example, federated ML module 218 identifies the relative location of a thumb based on a model of a hand. UI module 214, based on federated ML module 218 identifying the location of the user’s thumb, generates a second graphical user interface with one or more visual elements of the first graphical user interface visually rearranged to be within range of the thumb (e.g., rearranging the visual elements so that a user can reach them with their thumb without readjusting their hand on computing device 202).

[0116] UI module 214 may generate a second, updated graphical user interface based on receiving an identification of one or more patterns of input from touch mapping module 252 and/or federated ML module 218. Federated ML module 218 identifies patterns of input such as double-tap, gesture input, pattern of a large change in a pattern of input, a pattern of a change in a pattern of input below a predetermined threshold of input, a pattern of a change in a pattern of input above a threshold of input, a change in a position of one or more hands on computing device 202, etc. UI module 214 generates a second graphical user interface based on one or more of the identified patterns of input.

[0117] UI module 214 may generate a graphical user interface based on federated ML module 218 identifying a pattern of input that is a double-tap. Federated ML module 218 may identify a double-tap input that is associated with a visual element (e.g., a double-tap on the back of computing device 202 opposite a visual element displayed by display 232). UI module 214 may generate a graphical user interface based on federated ML module 218 identifying the double-tap input.

[0118] UI module 214, based on the identifying of a gesture input by federated ML module 218, generates a graphical user interface that includes a visual element associated with a double-tap input visually sliding in accordance with the gesture input. UI module 214 causes display 232 to output the graphical user interface with the visual element visually sliding in accordance with the gesture input.

[0119] UI module 214 may modify one or more settings of computing device 202 in response to one or more patterns of input. In an example, federated ML module 218 identifies one or more patterns of input. UI module 214, based on federated ML module 218 identifying one or more patterns of input, modifies one or more settings of computing device 202. In another example, federated ML module 218 identifies a sliding gesture. UI module 214, based on federated ML module 218 identifying a sliding gesture, modifies the brightness setting of display 232.

[0120] Display 232 may include display panel 228, display controller 230, and display touch controller 260. Display panel 228 may be a display device capable of rendering graphical user interfaces. Examples of display panel 228 include, but are not limited to, liquid crystal displays (LCD), dot matrix displays, light emitting diode (LED) displays, microLED displays organic light-emitting diode (OLED) displays, e-ink, or similar monochrome or color displays capable of outputting visible information to a user of computing device 202.

[0121] Display controller 230 may perform one or more operations to manage the operation of display panel 228. For instance, display controller 30 may receive instructions from UI module 214 that cause display controller 230 to control display panel 228 to render a particular graphical user interface.

[0122] Display controller 230 may cause display 232 may output one or more graphical user interfaces. For example, display 232 may output a graphical user interface that is configured by touch mapping module 252. In another example, display 232 may output a GUI that includes one or more widgets that visually move in response to user input.

[0123] FIG. 3 is a conceptual diagram illustrating example self-capacitance data generated by a presence-sensitive housing of a computing device, in accordance with one or more techniques of this disclosure. Self-capacitance data 300 of FIG. 3 is discussed with reference to computing device 202 of FIG. 2. However, other computing devices may generate self- capacitance data 300. To perform a self-capacitance scan (also referred to as a surface capacitance scan), touch controller 226 may drive an electrode of electrodes 224 with a signal and measure the capacitance across the entire electrode (e.g., with respect to ground). When another conductive object approaches the electrode, a capacitor is formed between them - reducing the capacitance between the electrode and ground. Touch controller 226 measures this capacitance by driving all electrodes of electrodes 224 in each direction (e.g., all of the rows and then all of the columns) and measuring their capacitance. Where electrodes 224 includes r row electrodes and c column electrodes, the self- capacitance scan produces r + c measurements, which are collectively referred to as self-capacitance data 300.

[0124] Self-capacitance data 300 of FIG. 3 may represent self-capacitance data measured by presence-sensitive housing 212, such as when the user is holding computing device 202 in their hand. As shown in FIG. 3, self-capacitance data 300 includes row capacitance values 302 and column capacitance values 304. For row capacitance values 302 and column capacitance values 304, darker locations indicate higher values. As can be seen in rowcapacitance values 302 and column capacitance values 304, by placing their fingers at positions 302A through 302F, the user may induce higher capacitance values.

[0125] FIG. 4 is a conceptual diagram illustrating example mutual-capacitance data generated by a presence-sensitive display of a computing device, in accordance with one or more techniques of this disclosure. For the purposes of clarity, FIG. 4 is discussed in reference to computing device 202 of FIG. 2. However, other computing devices may generate mutual-capacitance data 400.

[0126] To perform a mutual-capacitance scan, touch controller 226 may take advantage of the inherent capacitive coupling that exists between the row and column electrodes of electrodes 224 at the locations that they overlap (e.g., touch sensor cells). For instance, touch controller 226 may drive a single electrode of electrodes 224 (e.g., a row) and measure the capacitance on the intersecting electrodes of electrodes 224 (e.g., the columns). Touch controller 226 may repeat this process until all touch sensor cells have been sensed. Where electrodes 224 includes r row electrodes and c column electrodes, the mutual-capacitance scan produces r x c measurements, which are collectively referred to as mutual-capacitance data 400. For mutual-capacitance data 400, darker cells indicate higher values. Mutual- capacitance sensing therefore involves the sensing of each cell individually to generate a full “image” of the panel, which may allow touch controller 226 to unambiguously separate each contact (e.g., input points 402A through 402F).

[0127] Touch mapping module 252 may obtain input points 402A-402F (hereinafter “input points 402”) and cause federated ML module 218 to generate a model of a user’s hand. For example, federated ML module 218 may determine that input point 402F corresponds to a palm of a user, input point 402E corresponds to a thumb of the user, and input points 402A, 402B, 402C, and 402D correspond to other fingers of the user.

[0128] Federated ML module 218, based on assigning different parts of a user’s hand to input points 402, may generate a model of the user’s hand. Federated ML module 218 may generate a model of the user’s hand that includes a modeling of the arc of each finger of the user’s hand. Federated ML module 218 may use the model of the user’s hand to identify gesture inputs.

[0129] FIG. 5 is a cropped set of mutual-capacitance data, in accordance with one or more techniques of this disclosure. Mutual-capacitance data 500 of FIG. 5 may represent an area of mutual-capacitance data surrounding a contact (e.g., a finger contact) and, for the purposes of clarity, is discussed in reference to computing device 202 of FIG. 2. As shown in FIG. 5, by bringing an input object, such as a finger, in proximity with a presence-sensitive display, the capacitance of several cells may be altered. As discussed herein, based on the altering of these capacitances, touch controller 226 may identify touch contacts using mutual- capacitance data.

[0130] For each identified touch contact, touch controller 226 may identify the covered cells of the contact and determine, based on the covered cells, a location 502 of presence-sensitive housing 212 that corresponds to the covered cells, such as a set of coordinates that are in the resolution of display panel 228 on the opposing side of the exterior of computing device 200. In the example of FIG. 5, touch controller 226 may identify the covered cells as a contiguous set of non-white cells (e.g., a set of contiguous cells all having a capacitance value greater than a threshold value).

[0131] Touch mapping module 252 may identify location 502 as a point of user input. Touch mapping module 252 may identify one or more locations such as location 502 where input has been received by computing device 200. Touch mapping module 252 may provide location 502 to federated ML module 218.

[0132] Touch mapping module 252 may use federated ML module 218 to generate a model of a hand based on one or more points of user input such as location 502. Federated ML module 218 may generate a model of a hand that includes one or more fingers. Federated ML module 218 may assign the one or more fingers to the user input locations such as location 502. For example, federated ML module 218 may assign an index finger of the model of the hand to location 502.

[0133] Touch mapping module 252 may use federated ML module 218 to identify a gesture input received by computing device 202. Touch mapping module 252 may use federated ML module 218 to identify the gesture input and generate a GUI based on the received gesture input.

[0134] FIG. 6 is a flowchart illustrating example operations of an example computing device to output a graphical user interface, in accordance with one or more techniques of the present disclosure. The operations of the example computing device are described within the context of computing device 202 of FIG. 2.

[0135] As shown in FIG. 6, computing device 202 may detect one or more user inputs via presence-sensitive housing 212 (602). Computing device 202 may detect the user inputs via presence-sensitive housing 212 that wraps around a portion of the exterior of computing device 202 (e.g., the exterior of computing device 202 that is not part of display 232). Touch controller 226 may process the user inputs for mapping by touch mapping module 252.

[0136] Touch mapping module 252 generates a map of the one or more user inputs, wherein the map indicates respective locations of presence-sensitive housing 212 at which each of the one or more user inputs was detected (604). Touch mapping module 252 may generate a three-dimensional map of the one or more user inputs (as presence-sensitive housing 212 wraps around the exterior of computing device 202). Touch mapping module 252 may derive a two-dimensional map of the one or more user inputs from the three-dimensional map.

[0137] Touch mapping module 252 generates a model of a hand by at least applying a machine learning model such as federated ML module 218 to the map of the one or more user inputs to generate a filtered set of user inputs (606). Federated ML module 218 generates a model of a hand based on the relative locations of the one or more user inputs. Federated ML module 218 may map a finger of the model of the hand to each of the one or more user inputs.

[0138] Federated ML module 218 filters, using the model of the hand, the one or more user inputs to remove one or more of the user inputs that are not consistent with a gesture input (608). Touch mapping module 252 may filter the user inputs to determine which user inputs will be used in identifying the gesture input. For example, touch mapping module 252 may filter touch inputs that are consistent with a user holding computing device 202 rather than being consistent with a gesture input received by computing device 202.

[0139] Touch mapping module 252 determines, based on the filtered one or more user inputs, the gesture input (610). Touch mapping module 252 may determine one or more types of gesture input. Responsive to determining the gesture input, touch mapping module 252 may cause computing device 200 to output a graphical user interface based on the gesture input.

[0140] FIG. 7 is a flowchart illustrating example operations of an example computing device to identify user gesture inputs, in accordance with one or more techniques of the present disclosure. The operations of the example computing device are described within the context of computing device 202 of FIG. 2. [0141] Computing device 202 may identify the contact points as being fingertip or non- fingertip (702). Federated ML module 218 may use input shape classifications to identify contact points as fingertip or non-fingertip (704). Federated ML module 218 may classify the shape of the contact points in the input heatmap to identify contact points as fingertip or non- fingertip. Federated ML module 218 may compare the shape of the contact points against a plurality of example contact points to identify contact points as fingertip or non-fingertip. [0142] Federated ML module 218 may also use size classifications of the contact points to identify contact points as fingertip or non-fingertip (706). Federated ML module 218 may classify the contact points as fingertip or non- fingertip based on the size of the contact points (e.g., whether the size of a contact point is consistent with the size of a fingertip or whether the size of the contact point is consistent with a different part of a finger). Federated ML module 218 may use both shape classification and size classification in identifying contact points as fingertip or non-fmgertip. Federated ML module 218 may conduct size classification before conducting shape classification, and vice versa. In addition, federated ML module 218 may simultaneously conduct shape classification and size classification.

[0143] Federated ML module 218 may identify which fingertip contact) s) to use for identifying a gesture input (708). Federated ML module 218 may identify which fingertip contacts of one or more fingertip contacts are applicable to a gesture input. In an example, federated ML module 218 identifies four fingertip contacts to computing device 202. Federated ML module 218 identifies two of the four fingertip contacts that are applicable to a gesture input.

[0144] Federated ML module 218 may compare fingertip contacts to common grip patterns to identify which fingertip contacts to use for gesture inputs (710). Federated ML module 218 may compare the fingertip contacts to common grip patterns to filter out fingertip contacts that are consistent with a user holding computing device 202. In an example, federated ML module 218 identifies three fingertip contacts and compares the fingertip contacts with a plurality of common grip patterns stored in storage components 248. Federated ML module 218 determines that two of the three fingertip contacts are consistent with a user holding computing device 202 and filters out the two fingertip contacts before identifying a gesture input.

[0145] Federated ML module 218 may apply one or more machine learning models to identify fingertip contacts to use for a gesture input (712). Federated ML module 218 may apply one or more machine learning models trained by a plurality of other computing devices to identify which fingertips should be used to identify the gesture input. Federated ML module 218 may additionally use the one or more machine learning models to determine whether the fingertip contact points are consistent with one or more gesture inputs. Federated ML module 218 may simultaneously apply the one or more machine learning models while comparing the fingertip contacts with common grip patterns, before comparing the fingertip contacts with common grip patterns, or after comparing the fingertip contacts with common grip patterns.

[0146] Federated ML module 218 may filter out and remove non-gesture fingertip contacts (714). Federated ML module 218 may filter out the non-gesture fingertip contacts before identifying a gesture. Federated ML module 218 may filter out the non-gesture fingertip contacts to improve the accuracy of identifying gesture inputs.

[0147] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer- readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media, which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0148] By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to non-transient, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0149] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0150] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.

[0151] Various examples of the disclosure have been described. Any combination of the described systems, operations, or functions is contemplated. These and other examples are within the scope of the following claims.

[0152] Example 1 : A method includes detecting, by a presence-sensitive housing, one or more user inputs; generating, by a computing device, a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected; generating, by the computing device, a model of a hand by at least applying a machine learning model to the map of the one or more user inputs; removing, by the computing device and using the model of the hand, at least one user input that is not consistent -with a gesture input from the one or more user inputs to generate a filtered set of user inputs; and determining, by the computing device and based on the filtered set of user inputs, the gesture input. [0153] Example 2: The method of example 1, wherein determining the gesture input further comprises: identifying, by the computing device and using the machine learning model, one or more patterns of input, wherein the one or more patterns of input include one of: a double- tap, a pattern of change in the one or more patterns of input, a pattern of change in the one or more patterns of input , or a change in a position of the hand or a second hand on the computing device.

[0154] Example 3: The method of any of examples 1 and 2, wherein the machine learning model is a federated machine learning model, the method further includes training the federated machine learning model by at least: obtaining a plurality of maps of user inputs and a plurality of , wherein each associated with a respective map of user inputs from the plurality of maps of user inputs; providing the plurality of maps of user inputs and the plurality of models to the federated machine learning model; and updating, based on a comparison of an output of the federated machine learning model to the plurality of hand models, the federated machine learning model.

[0155] Example 4: The method of any of examples 1-3, further includes determining, by the computing device and using the machine learning model, a difference between the one or more user inputs and previous one or more user inputs; determining, by the computing device, whether the difference satisfies a threshold difference; and responsive to determining that the difference satisfies the threshold difference, removing the at least one user input that is not consistent with a gesture input from the one or more user inputs to generate filtered set of user inputs..

[0156] Example 5: The method of any of examples 1-4, further includes generating, by the computing device and based on the gesture input, a graphical user interface; and outputting, by the computing device and for display, the graphical user interface.

[0157] Example 6: The method of any of examples 1-5, wherein the graphical user interface is a first graphical user interface, wherein the first graphical user interface includes one or more visual elements, and further includes determining, by the computing device and based on the model of the hand, a size of the hand; generating, by the computing device and based on the size of the hand, a second graphical user interface, wherein the second graphical user interface includes the one or more visual elements of the first graphical user interface adjusted according to the size of the hand; and outputting, by the computing device and for display, the second graphical user interface.

[0158] Example 7: The method of any of examples 1-6, wherein the hand is a first hand, and further includes determining, by the computing device, that a second hand is holding the computing device; generating, by the computing device, a model of the second hand, wherein generating the model of the second hand includes determining a hand span and arc of fingers of the second hand; generating, by the computing device and based on the model of the second hand, a third graphical user interface that includes one or more elements, wherein the one or more elements of the third graphical user interface are sized and are visually located within the third graphical user interface according to the model of the second hand; and outputting, by the computing device and for display, the third graphical user interface.

[0159] Example 8: A computing device includes a memory; and one or more programmable processors in communication with the memory, and configured to: detect one or more user inputs; generate a map of the one or more user inputs, wherein the map indicates respective locations at which each of the one or more user inputs was detected; generate a model of a hand by at least applying a machine learning model to the map of the one or more user inputs; remove, using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs; and determine, based on the filtered set of user inputs, the gesture input.

[0160] Example 9: The computing device of example 8, wherein determine the gesture input further comprises: identify, using the machine learning model, one or more patterns of input, wherein the one or more patterns of input include one of: a double-tap, a pattern of a change in the one or more patterns of input above a predetermined threshold of input, a pattern of a change in the one or more patterns of input below a predetermined threshold of input, or a change in a position of the hand or a second hand on the computing device.

[0161] Example 10: The computing device of any of examples 8 and 9, wherein the machine learning model is a federated machine learning model, and wherein the one or more programmable processors are further configured to train the federated machine learning model, wherein train the federated machine learning model at least comprises: obtain a plurality of maps of prior user inputs and a plurality of hand models, wherein each hand model from the plurality of hand models is associated with a respective map of prior user inputs from the plurality of maps of prior user inputs; provide the plurality of maps of prior user inputs and the plurality of hand models to the federated machine learning model; and update, based on a comparison of an output of the federated machine learning model to the plurality of hand models, the federated machine learning model.

[0162] Example 11: The computing device of any of examples 8-10, wherein the one or more programmable processors are further configured to: determine, using the machine learning model, a difference between the one or more user inputs and previous one or more user inputs; determine whether the difference satisfies a threshold difference, and responsive to determining that the difference satisfies the threshold difference, remove the at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs.

[0163] Example 12: The computing device of any of examples 8-11, wherein the one or more programmable processors are further configured to: generate, based on the gesture input, a graphical user interface; and output, for display, the graphical user interface.

[0164] Example 13: The computing device of any of examples 8-12, wherein the graphical user interface is a first graphical user interface, wherein the first graphical user interface includes one or more visual elements, and wherein the one or more programmable processors are further configured to: determine based on the model of the hand, a size of the hand; generate, based on the size of the hand, a second graphical user interface, wherein the second graphical user interface includes the one or more visual elements of the first graphical user interface adjusted according to the size of the hand; and outputting, by the computing device and for display, the second graphical user interface.

[0165] Example 14: The computing device of any of examples 8-13, wherein the hand is a first hand, and wherein the one or more processors are further configured to: determine, by the computing device, that a second hand is holding the computing device; generate, by the computing device, a model of the second hand, wherein to generate the model of the second hand includes determining a hand span and arc of fingers of the second hand; generate, by the computing device and based on the model of the second hand, a third graphical user interface that includes one or more elements, wherein the one or more elements of the third graphical user interface are sized and are visually located within the third graphical user interface according to the model of the second hand; and output, by the computing device and for display, the third graphical user interface.

[0166] Example 15: A non-transitory computer-readable medium encoded with instructions that cause one or more processors of a computing system to detect one or more user inputs; generate a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected; generate a model of a hand by at least applying a machine learning model to the map of the one or more user inputs; remove, using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs; and determine, based on the filtered set of user inputs, the gesture input. [0167] Example 16: The non-transitory computer-readable storage medium of example 15, wherein the instructions are further configured to cause the one or more processors to identify, using the machine learning model, one or more patterns of input, wherein the one or more patterns of input include one of: a double-tap, a pattern of a change in the one or more patterns of input above a predetermined threshold of input, a pattern of a change in the one or more patterns of input below a predetermined threshold of input, or a change in a position of the hand or a second hand on the computing device.

[0168] Example 17: The non-transitory computer-readable storage medium of any of examples 15 and 16, wherein the machine learning model is a federated machine learning model, and wherein the instructions are further configured to cause the one or more processors to train the federated machine learning model, wherein train the federated machine learning model at least comprises: obtain a plurality of maps of prior user inputs and a plurality of hand models, wherein each hand model from the plurality of hand models is associated with a respective map of prior user inputs from the plurality of maps of prior user inputs; provide the plurality of maps of prior user inputs and the plurality of hand models to the federated machine learning model; and update, based on a comparison of an output of the federated machine learning model to the plurality of hand models, the federated machine learning model.

[0169] Example 18: The non-transitory computer-readable storage medium of any of examples 15-17, wherein the instructions are further configured to cause the one or more processors to: determine, using the machine learning model, a difference between the one or more user inputs and previous one or more user inputs; and determine, whether the difference satisfies a threshold difference, wherein removing the at least one user input is in response to determining that the difference satisfies the threshold difference.

[0170] Example 19: The non-transitory computer-readable storage medium of any of examples 15-18, wherein the instructions are further configured to cause the one or more processors to: generate, by the computing device and based on the gesture input, a graphical user interface; and output, by the computing device and for display, the graphical user interface.

[0171] Example 20: The non-transitory computer-readable storage medium of any of examples 15-19, wherein the graphical user interface is a first graphical user interface, wherein the first graphical user interface includes one or more visual elements, and wherein the instructions are further configured to cause the one or more processors to: determine, based on the model of the hand, a size of the hand; generate, based on the size of the hand, a second graphical user interface, wherein the second graphical user interface includes the one or more visual elements of the first graphical user interface adjusted according to the size of the hand; and output, by the computing device and for display, the second graphical user interface.

[0172] Example 21: The computer-readable storage medium of any of examples 15-20, wherein the hand is a first hand, and wherein the instractions are further configured to cause the one or more processors to: determine, that a second hand is holding the computing device; generate a model of the second hand, wherein generating the model of the second hand includes determining a hand span and arc of fingers of the second hand; generate, based on the model of the second hand, a third graphical user interface that includes one or more elements, wherein the one or more elements of the third graphical user interface are sized and are visually located within the third graphical user interface according to the model of the second hand; and output, for display, the third graphical user interface.

[0173] Example 22: A device comprising means for performing any combination of the methods of examples 1-12.

[0174] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: detecting, by a presence-sensitive housing, one or more user inputs; generating, by a computing device, a map of the one or more user inputs, wherein the map indicates respective locations of the presence-sensitive housing at which each of the one or more user inputs was detected; generating, by the computing device, a model of a hand by at least applying a machine learning model to the map of the one or more user inputs; removing, by the computing device and using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs; and determining, by the computing device and based on the filtered set of user inputs, the gesture input.

2. The method of claim 1, wherein determining the gesture input further comprises: identifying, by the computing device and using the machine learning model, one or more patterns of input, wherein the one or more patterns of input include one of: a double-tap, a pattern of a change in the one or more patterns of input above a predetermined threshold of input, a pattern of a change in the one or more patterns of input below a predetermined threshold of input, or a change in a position of the hand or a second hand on the computing device.

3. The method of any of claims 1 and 2, wherein the machine learning model is a federated machine learning model, the method further comprising: training the federated machine learning model by at least: obtaining a plurality of maps of prior user inputs and a plurality of hand models, wherein each hand model from the plurality of hand models is associated with a respective map of prior user inputs from the plurality of maps of prior user inputs: providing the plurality of maps of prior user inputs and the plurality of hand models to the federated machine learning model; and updating, based on a comparison of an output of the federated machine learning model to the plurality of hand models, the federated machine learning model.

4. The method of any of claims 1-3, further comprising: determining, by the computing device and using the machine learning model, a difference between the one or more user inputs and previous one or more user inputs; determining, by the computing device, whether the difference satisfies a threshold difference; and responsive to determining that the difference satisfies the threshold difference, removing the at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs.

5. The method of any of claims 1-4, further comprising: generating, by the computing device and based on the gesture input, a graphical user interface; and outputting, by the computing device and for display, the graphical user interface.

6. The method of any of claims 1-5, wherein the graphical user interface is a first graphical user interface, wherein the first graphical user interface includes one or more visual elements, and further comprising: determining, by the computing device and based on the model of the hand, a size of the hand; generating, by the computing device and based on the size of the hand, a second graphical user interface, wherein the second graphical user interface includes the one or more visual elements of the first graphical user interface adjusted according to the size of the hand; and outputting, by the computing device and for display, the second graphical user interface.

7. The method of any of claims 1-6, wherein the hand is a first hand, and further comprising: determining, by the computing device, that a second hand is holding the computing device; generating, by the computing device, a model of the second hand, wherein generating the model of the second hand includes determining a hand span and arc of fingers of the second hand; generating, by the computing device and based on the model of the second hand, a third graphical user interface that includes one or more elements, wherein the one or more elements of the third graphical user interface are sized and are visually located within the third graphical user interface according to the model of the second hand; and outputting, by the computing device and for display, the third graphical user interface.

8. A computing device, comprising: a memory; and one or more programmable processors in communication with the memory, and configured to: detect one or more user inputs; generate a map of the one or more user inputs, wherein the map indicates respective locations at which each of the one or more user inputs was detected; generate a model of a hand by at least applying a machine learning model to the map of the one or more user inputs ; remove, using the model of the hand, at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs; and determine, based on the filtered set of user inputs, the gesture input.

9. The computing device of claim 8, wherein determine the gesture input further comprises: identify, using the machine learning model, one or more patterns of input, wherein the one or more patterns of input include one of: a double-tap, a pattern of a change in the one or more patterns of input above a predetermined threshold of input, a pattern of a change in the one or more patterns of input below a predetermined threshold of input, or a change in a position of the hand or a second hand on the computing device.

10. The computing device of any of claims 8 and 9, wherein the machine learning model is a federated machine learning model, and wherein the one or more programmable processors are further configured to train the federated machine learning model, wherein train the federated machine learning model at least comprises: obtain a plurality of maps of prior user inputs and a plurality of hand models, wherein each hand model from the plurality of hand models is associated with a respective map of prior user inputs from the plurality of maps of prior user inputs; provide the plurality of maps of prior user inputs and the plurality of hand models to the federated machine learning model; and update, based on a comparison of an output of the federated machine learning model to the plurality of hand models, the federated machine learning model.

11. The computing device of any of claims 8-10, wherein the one or more programmable processors are further configured to: determine, using the machine learning model, a difference between the one or more user inputs and previous one or more user inputs; determine whether the difference satisfies a threshold difference; and responsive to determining that the difference satisfies the threshold difference, remove the at least one user input that is not consistent with a gesture input from the one or more user inputs to generate a filtered set of user inputs.

12. The computing device of any of claims 8-11, wherein the one or more programmable processors are further configured to: generate, based on the gesture input, a graphical user interface; and output, for display, the graphical user interface.

13. The computing device of any of claims 8-12, wherein the graphical user interface is a first graphical user interface, wherein the first graphical user interface includes one or more visual elements, and wherein the one or more programmable processors are further configured to: determine based on the model of the hand, a size of the hand; generate, based on the size of the hand, a second graphical user interface, wherein the second graphical user interface includes the one or more casual elements of the first graphical user interface adjusted according to the size of the hand; and outputting, by the computing device and for display, the second graphical user interface.

14. The computing device of any of claims 8-13, wherein the hand is a first hand, and wherein the one or more programmable processors are further configured to: determine that a second hand is holding the computing device; generate a model of the second hand, wherein generate the model of the second hand includes determine a hand span and arc of fingers of the second hand; generate, based on the model of the second hand, a third graphical user interface that includes one or more elements, wherein the one or more elements of the third graphical user interface are sized and are visually located within the third graphical user interface according to the model of the second hand; and output, for display, the third graphical user interface.

15. A non-transitory computer-readable medium, encoded with instructions that cause one or more processors of a computing system to perform any of the methods of claims 1-8.