Konrad et al., 2024 - Google Patents
Gazegpt: Augmenting human capabilities using gaze-contingent contextual ai for smart eyewearKonrad et al., 2024
View PDF- Document ID
- 14846677741756221086
- Author
- Konrad R
- Padmanaban N
- Buckmaster J
- Boyle K
- Wetzstein G
- Publication year
- Publication venue
- arXiv preprint arXiv:2401.17217
External Links
Snippet
Multimodal large language models (LMMs) excel in world knowledge and problem-solving abilities. Through the use of a world-facing camera and contextual AI, emerging smart accessories aim to provide a seamless interface between humans and LMMs. Yet, these …
- 230000003190 augmentative effect 0 title description 13
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
- G02B27/00—Other optical systems; Other optical apparatus
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
- G02B27/00—Other optical systems; Other optical apparatus
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
- H04N5/225—Television cameras; Cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, video cameras, camcorders, webcams, camera modules for embedding in other devices, e.g. mobile phones, computers or vehicles
- H04N5/232—Devices for controlling television cameras, e.g. remote control; Control of cameras comprising an electronic image sensor, e.g. digital cameras, video cameras, TV cameras, video cameras, camcorders, webcams, camera modules for embedding in, e.g. mobile phones, computers or vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING; COUNTING
- G06F—ELECTRICAL DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Konrad et al. | Gazegpt: Augmenting human capabilities using gaze-contingent contextual ai for smart eyewear | |
| Koulieris et al. | Near‐eye display and tracking technologies for virtual and augmented reality | |
| JP7200195B2 (en) | sensory eyewear | |
| US11217021B2 (en) | Display system having sensors | |
| US10831268B1 (en) | Systems and methods for using eye tracking to improve user interactions with objects in artificial reality | |
| RU2693329C2 (en) | Method and device for displaying with optimization of pixel redistribution | |
| CA3011257C (en) | Language element vision augmentation methods and devices | |
| US11934572B2 (en) | Dynamic content presentation for extended reality systems | |
| US9838597B2 (en) | Imaging device, imaging method, and program | |
| US10810797B2 (en) | Augmenting AR/VR displays with image projections | |
| US9165381B2 (en) | Augmented books in a mixed reality environment | |
| JP6165846B2 (en) | Selective enhancement of parts of the display based on eye tracking | |
| EP4248413A1 (en) | Multiple device sensor input based avatar | |
| CN111630478B (en) | High-speed staggered binocular tracking system | |
| CN109791433A (en) | Prediction type central fovea virtual reality system | |
| CN107105333A (en) | A kind of VR net casts exchange method and device based on Eye Tracking Technique | |
| WO2015026645A1 (en) | Automatic calibration of scene camera for optical see-through head mounted display | |
| US10983591B1 (en) | Eye rank | |
| US20230288701A1 (en) | Sensor emulation | |
| CN105843397A (en) | Virtual reality interactive system based on pupil tracking technology | |
| WO2021246134A1 (en) | Device, control method, and program | |
| CN105898598A (en) | Video focusing zoom method and apparatus | |
| US20240104862A1 (en) | Spatially aware playback for extended reality content | |
| US11823343B1 (en) | Method and device for modifying content according to various simulation characteristics | |
| US20250106506A1 (en) | Camera Focusing for Video Passthrough Systems |