[go: up one dir, main page]

WO2021052660A1 - Procédé et dispositif de traitement d'une image enregistrée par une caméra - Google Patents

Procédé et dispositif de traitement d'une image enregistrée par une caméra Download PDF

Info

Publication number
WO2021052660A1
WO2021052660A1 PCT/EP2020/071084 EP2020071084W WO2021052660A1 WO 2021052660 A1 WO2021052660 A1 WO 2021052660A1 EP 2020071084 W EP2020071084 W EP 2020071084W WO 2021052660 A1 WO2021052660 A1 WO 2021052660A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
transformation
camera
user
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2020/071084
Other languages
German (de)
English (en)
Inventor
Johannes Von Hoyningen-Huene
Andreas Mueller
Nikolaj Marchenko
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of WO2021052660A1 publication Critical patent/WO2021052660A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality

Definitions

  • the present invention relates to a method for processing an image captured by a camera.
  • the present invention also relates to a corresponding device, a corresponding computer program and a corresponding storage medium.
  • Augmented reality is generally understood to mean the computer-aided expansion of the perception of reality in real time.
  • US9269011B1 uses images from a variety of sources to capture different angles, zoom levels and views of objects and sights in different lighting conditions.
  • the images are tagged or linked to a specific position or a specific point of interest and stored in a database in order to use AR to identify objects that appear in the live view of a scene that was recorded by at least one camera of a data processing device.
  • the layout platform determines at least one zoom level for the reproduction of one or more images of one or more elements in a perspective representation, which is at least partially on at least one Push interaction or at least one pull interaction in the perspective representation is based.
  • the layout platform at least partially reproduces the representations that are based at least partially on the zoom level.
  • GB2513865A describes an AR scene with one or more objects, each object having augmented data and using one or more sensors to detect a request from a user to search at least one of the enriched data sets, possibly one or more categories such as image, video or multimedia belong.
  • a sequence of extended data to be searched is provided and searched based on a command signal sensed at one or more sensors.
  • the searched object can then be displayed based on a command signal received from the sensors, which can be touch sensors.
  • Searching the sequence of extended data can be based on rotary touch commands.
  • the one or more objects can be filtered based on a proximity to the user, wherein the filtering can be based on a pinch, spread zoom, or other multi-finger command.
  • the invention provides a method for processing an image recorded by a camera, a corresponding device, a corresponding computer program and a corresponding storage medium according to the independent claims.
  • the method according to the invention is based on the knowledge that in conventional AR applications the (augmented) content displayed on the AR terminal is based on virtual 3D models or two-dimensional surfaces on real surfaces or in space for displaying text, images, videos or programs Respectively.
  • the appearance of real objects is displayed in processed form for the user of the AR terminal.
  • the recorded camera image is transformed in a desired manner and displayed in the user's field of vision. A detailed interpretation of the camera image is not absolutely necessary, which keeps the computational effort within limits.
  • One advantage of this solution is the creation of a helpful extension of AR video glasses, which prepares the displayed image for the viewer and supports him in doing certain work more easily.
  • Various types of assistance for the user are conceivable through different image transformations.
  • a suitable pre-selection of the interesting image areas as well as a simple and quick change between AR functionality and a changed representation of the real environment can simplify many tasks in which different aids are currently used.
  • the proposed transformation mode does not require any major changes to the hardware of the AR terminals.
  • the camera, processing unit and screen are also present in conventional AR devices.
  • the additional mode may even make lower demands on the hardware of the AR glasses than their conventional AR mode.
  • a transformation of the entire image or a fixed image section sometimes causes less computational effort than the representation of virtual objects at real reference points by means of optical image recognition.
  • Embodiments of the invention could be used in the context of a wide variety of applications in which the user can be supported both by additional information and by a changed representation of the environment without having to change the optical aid, in this case the AR glasses.
  • Such applications include the construction and repair or maintenance of filigree components (enlargement), monitoring of systems (highlighting of image sections) and working in poor, changing visibility conditions (changes in brightness and contrast).
  • Modes can be changed by voice commands and head movements. This has the advantage that the user continues to have both hands available to carry out his task, since he does not need them to change the mode. This is particularly useful for tasks that have to be carried out with both hands so that no hand is free to operate the terminal.
  • Figure 1 shows the flow chart of a method according to a first embodiment.
  • FIG. 2 schematically shows AR glasses according to a second embodiment in standard mode.
  • FIG. 3 shows an extended mode with a magnifying glass function, in which the camera image is shown enlarged.
  • FIG. 1 illustrates the sequence of a method according to the invention (10).
  • the starting point is a conventional AR mode in which the user sees the environment, which may have been expanded to include AR content, but is otherwise unadulterated.
  • the AR terminal makes a preselection of the image sections and recognized objects in the user's field of vision for the image transformation and marks them on the display unit as areas that can be selected by the user.
  • the area or the recognized object is selected for image transformation by the user.
  • the image transformation mode is activated for this image area.
  • a third step (13) the entire image of the surroundings or a part thereof is subjected to the desired image transformation and is displayed on the display unit in the user's field of vision.
  • the changed display is intended to help the user, for example, to do his job in a simpler, more efficient or more ergonomic way.
  • a fourth step (14) the image transformation mode is exited again. This can be initiated, among other things, directly by the user or automatically by the AR terminal if, for example, a certain period of time expires or it is recognized that a work step has ended.
  • the detection of interesting image areas in the first step (11) takes place, for. B. using a suitable image recognition.
  • the markers can be attached directly to the viewed objects or with a known offset to them.
  • the areas that come into consideration for image transformation are offered to the user for selection by corresponding markings on the display unit.
  • the user selects the area for the image transformation in the second step (12).
  • Manual selection by means of a manually operated input device, e.g. B. keyboard, touchpad or joystick.
  • the input device can be located directly on the AR terminal. Alternatively, the entry can be made on another device and transmitted to the AR terminal.
  • gestures that are carried out in the field of vision of the camera and captured by image processing.
  • gestures could be recognized by a sensor glove and communicated to the AR terminal.
  • voice commands that are recorded via a microphone and processed according to their meaning are possible, special head movements such as nodding, turning or tilting the head, which can be detected by position, acceleration and rotation rate sensors, but also by image processing of the camera image, or the fixation of the gaze on an area.
  • head movements such as nodding, turning or tilting the head, which can be detected by position, acceleration and rotation rate sensors, but also by image processing of the camera image, or the fixation of the gaze on an area.
  • Separate cameras that capture the user's eyes can be used to determine the viewing direction.
  • the center of the camera image can also be used as a reference point for the viewing direction.
  • the final selection of the line of sight lying area can be done either automatically after a certain period of fixation or a confirmation by manual input, gesture or voice command.
  • the selection of the area for the image transformation is made by a third party.
  • the automatic detection of eligible areas and their selection by the user or according to a predetermined rule within the framework of the method (10) is optional.
  • the entire image area of the camera is selected for an image transformation or for the user to manually select part of the image without object recognition taking place in this area.
  • the user could, for example, use a suitable gesture to outline the area to be transformed in his field of vision or to mark it in some other way.
  • the entire image, the selected area or the recognized and selected object is displayed in a modified form on the screen.
  • image transformations are conceivable: Enlargement - for example, according to the operating principle of a magnifying glass - or reduction, mirroring or rotation, color matching or color replacement, brightness change or contrast enhancement, application of an image filter, edge detection and edge highlighting or replacement of the image by a recording in a different wavelength range , e.g. B. with the help of infrared or ultraviolet cameras.
  • an image transformation can take place on all other areas of the camera image outside the selected area or away from the selected object in order to e.g. B. to prevent distraction of the user.
  • Relevant measures include blackening or other covering with one or more colors or color gradients as well as blurring these areas.
  • the image transformation is not selected by the user himself, but rather by the AR terminal or a third party start the transformation (13) fluently - for example by simulating a change in the focal length ("zooming") or slowly increasing the filter function so that the user does not lose his or her orientation when changing the display mode.
  • the image transformation mode is exited and switched back to the standard mode, in which the user sees the non-transformed environment together with AR content.
  • the termination of the method (10) can - similar to the selection of the area to be transformed - be terminated by an action by the user using manual input devices or in the form of hand and finger gestures, head movements and voice commands.
  • the return to normal operation can also be initiated after a defined period of time, in that the selected objects leave the field of vision of the camera, or by external events that are transmitted to the AR terminal.
  • a termination by a third party or the recognition of a situation in which the user must see his environment unchanged - i.e. without image transformation - in order to be able to recognize and avert a danger to himself or others, for example, are conceivable.
  • FIG. 1 An application example for the practical use of AR glasses with standard and additional functions could be the computer-aided assembly of a circuit board with very small components.
  • FIG. 1 In the standard mode, the user of the AR glasses (20) sees the real environment as he would see it without glasses (20).
  • virtual objects (27) are shown on the screen of the glasses (20), which support him in his task.
  • such an object (27) in the form of a speech bubble could guide the user equipped with solder (23) and soldering iron (24) to attach a certain component (26) to a certain point (25) on the circuit board (22) to support him in this complex workflow. That way he would be in his direct field of vision pointed to the upcoming work step without having to take your eyes off the board (22) and focus on an assembly plan.
  • the second mode of the AR glasses (20) is for carrying out the work step itself, in the present case the filigree soldering of a very small component
  • the user is suggested to enlarge the image area around the solder joint or the entire image.
  • said image area or the entire image is digitally enlarged and - as shown in FIG. 3 - displayed on the screen of the AR glasses (20). It is expedient that the perception of the actual environment is completely suppressed so that the user does not see the actual environment in unchanged form and a partially transparent representation of the digitally enlarged environment at the same time.
  • the user switches the AR glasses (20) back to standard mode.
  • This method (10) can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in the AR glasses (20).

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un procédé (10) de traitement d'une image enregistrée par une caméra, caractérisé par les caractéristiques suivantes : - une image est soumise dans un mode de transformation à une transformation prédéfinie (13) et affichée à l'attention d'un utilisateur sur une unité d'affichage (21), et - le mode de transformation est laissé à un événement prédéfini (14).
PCT/EP2020/071084 2019-09-19 2020-07-27 Procédé et dispositif de traitement d'une image enregistrée par une caméra Ceased WO2021052660A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102019214283.4A DE102019214283A1 (de) 2019-09-19 2019-09-19 Verfahren und Vorrichtung zum Verarbeiten eines durch eine Kamera aufgenommenen Bildes
DE102019214283.4 2019-09-19

Publications (1)

Publication Number Publication Date
WO2021052660A1 true WO2021052660A1 (fr) 2021-03-25

Family

ID=71833341

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2020/071084 Ceased WO2021052660A1 (fr) 2019-09-19 2020-07-27 Procédé et dispositif de traitement d'une image enregistrée par une caméra

Country Status (2)

Country Link
DE (1) DE102019214283A1 (fr)
WO (1) WO2021052660A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102023202839A1 (de) 2023-03-28 2024-10-02 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zur Ermittlung einer Konfiguration einer nutzerzustandsabhängigen Ausgabe von Informationen für einen Nutzer einer AR-Vorrichtung, sowie AR-Vorrichtung
DE102023202840A1 (de) 2023-03-28 2024-10-02 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zur Ermittlung einer Konfiguration einer Ausgabe von Informationen für einen Nutzer einer AR-Vorrichtung, sowie AR-Vorrichtung

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043365A1 (en) 2012-08-10 2014-02-13 Nokia Corporation Method and apparatus for layout for augmented reality view
US20140082500A1 (en) * 2012-09-18 2014-03-20 Adobe Systems Incorporated Natural Language and User Interface Controls
US20140195921A1 (en) * 2012-09-28 2014-07-10 Interactive Memories, Inc. Methods and systems for background uploading of media files for improved user experience in production of media-based products
GB2513865A (en) 2013-05-07 2014-11-12 Present Pte Ltd A method for interacting with an augmented reality scene
US20150106755A1 (en) * 2013-10-16 2015-04-16 3M Innovative Properties Company Editing digital notes representing physical notes
US9269011B1 (en) 2013-02-11 2016-02-23 Amazon Technologies, Inc. Graphical refinement for points of interest

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140043365A1 (en) 2012-08-10 2014-02-13 Nokia Corporation Method and apparatus for layout for augmented reality view
US20140082500A1 (en) * 2012-09-18 2014-03-20 Adobe Systems Incorporated Natural Language and User Interface Controls
US20140195921A1 (en) * 2012-09-28 2014-07-10 Interactive Memories, Inc. Methods and systems for background uploading of media files for improved user experience in production of media-based products
US9269011B1 (en) 2013-02-11 2016-02-23 Amazon Technologies, Inc. Graphical refinement for points of interest
GB2513865A (en) 2013-05-07 2014-11-12 Present Pte Ltd A method for interacting with an augmented reality scene
US20150106755A1 (en) * 2013-10-16 2015-04-16 3M Innovative Properties Company Editing digital notes representing physical notes

Also Published As

Publication number Publication date
DE102019214283A1 (de) 2021-03-25

Similar Documents

Publication Publication Date Title
DE112006002954B4 (de) Virtuelles Schnittstellensystem
EP2157903B1 (fr) Procede de mesure de perception
DE60225933T2 (de) Tragbare virtuelle realität
DE60133386T2 (de) Vorrichtung und verfahren zur anzeige eines ziels mittels bildverarbeitung ohne drei dimensionales modellieren
DE102016210288A1 (de) Bedienvorrichtung mit Eyetrackereinheit und Verfahren zum Kalibrieren einer Eyetrackereinheit einer Bedienvorrichtung
EP2691934B1 (fr) Images "identikit" avec base de donnees
DE102011056714A1 (de) Systembereitschaftsschalter für ein Mensch-Maschine-Interaktionssteuerungssystem mit Augenverfolgung
DE3609469A1 (de) Anzeigesteuervorrichtung
DE202005021436U1 (de) Optisch verbessertes digitales Abbildungssystem
WO2011060952A1 (fr) Appareil de prise de vues et de représentation d'objets, en particulier jumelles numériques, appareil photographique numérique ou caméra vidéo numérique
EP3857303A1 (fr) Procédé de réglage de mise au point d'une caméra
DE102015115394A1 (de) Verfahren und Vorrichtung zum Überlagern eines Abbilds einer realen Szenerie mit einem virtuellen Bild und mobiles Gerät
WO2021052660A1 (fr) Procédé et dispositif de traitement d'une image enregistrée par une caméra
DE102017215161A1 (de) Verfahren und Vorrichtung zur Anwahl von einem Umgebungsobjekt im Umfeld eines Fahrzeugs
WO2019174862A1 (fr) Procédé et dispositif de fourniture d'informations par un appareil à réalité augmentée, procédé et dispositif de fourniture d'informations pour la commande d'un affichage d'un appareil à réalité augmentée, procédé et dispositif pour la commande d'un affichage d'un appareil à réalité augmentée, support d'enregistrement lisible par ordinateur ayant des instructions pour l'exécution du procédé
DE102014115363A1 (de) Virtuelles Zeichen in realer Umgebung
WO1998015964A1 (fr) Procede et dispositif pour commander au moins un appareil
DE102004061841B4 (de) Markerloses Tracking System für Augmented Reality Anwendungen
DE102012008986B4 (de) Kamerasystem mit angepasster ROI, Kraftfahrzeug und entsprechendes Verfahren
DE102018206675A1 (de) Verfahren zur Ansteuerung einer Maschine mittels mindestens einer Raumkoordinate als Ansteuergröße sowie Ansteuerungssystem einer Maschine
DE19812599C2 (de) Verfahren zur Video-Mikroskopie
WO2016046124A1 (fr) Dispositif d'affichage pouvant être monté sur la tête d'un utilisateur et procédé de commande de ce dispositif d'affichage
WO2019228780A1 (fr) Concept pour la commande d'un écran d'un appareil mobile à réalité augmentée
DE102019207454B4 (de) Augmented-Reality-System
DE102014115988A1 (de) Verfahren zum Aufzeichnen und Bearbeiten von zumindest einer zumindest eine Videospur und eine Tonspur umfassenden Videosequenz

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20746632

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20746632

Country of ref document: EP

Kind code of ref document: A1