[go: up one dir, main page]

WO2016108847A1 - Procédés et appareil de traitement d'images à informations de mouvement - Google Patents

Procédés et appareil de traitement d'images à informations de mouvement Download PDF

Info

Publication number
WO2016108847A1
WO2016108847A1 PCT/US2014/072764 US2014072764W WO2016108847A1 WO 2016108847 A1 WO2016108847 A1 WO 2016108847A1 US 2014072764 W US2014072764 W US 2014072764W WO 2016108847 A1 WO2016108847 A1 WO 2016108847A1
Authority
WO
WIPO (PCT)
Prior art keywords
frames
phase
video sequence
frame
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2014/072764
Other languages
English (en)
Inventor
Ekta Prashnani
Maneli NOORKAMI
Daniel Andre VAQUERO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Technologies Oy
Nokia USA Inc
Original Assignee
Nokia Technologies Oy
Nokia USA Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Technologies Oy, Nokia USA Inc filed Critical Nokia Technologies Oy
Priority to PCT/US2014/072764 priority Critical patent/WO2016108847A1/fr
Publication of WO2016108847A1 publication Critical patent/WO2016108847A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation

Definitions

  • One or more embodiments of the present invention relate generally to image creation and modification. More particularly, embodiments of the invention relate to systems and techniques for adding motion information from video sequences to still images so as to enhance the still images with motion.
  • a user of an electronic device may wish to display still images using a device, and a still image maybe more engaging, may attract more attention, or may meet other needs of a user, if motion can be added to areas of the image. For example, water may ripple, trees may sway, flames may flicker, or clouds may drift.
  • Moving images such as those taken with electronic video equipment, are a ubiquitous source of motion elements, but the images need to be analyzed both for resemblance to a still image to which motion is to be added and to recover motion elements to be added to the still image.
  • Moving images are made up of numerous elements - a moving image comprises a number of frames, each of which includes numerous pixels.
  • an apparatus comprises at least one processor and memory storing a program of instructions.
  • the memory storing the program of instructions is configured to, with the at least one processor, configure the apparatus to at least analyze successive frames of a video sequence to extract phase difference information indicating motion, successively construct frames based on a still image, wherein successively constructing the frames based on a still image comprises using the phase difference information to successively update phase information and adding the phase information to successively created still image f ames so as to introduce variations representing motion, and assemble the successively constructed frames to create a reconstructed output video sequence based on the still image.
  • an apparatus comprises at least one processor and memory storing a program of instructions.
  • the memory storing the program of instructions is configured to, with the at least one processor, configure the apparatus to at least identify two or more video sequences for similarity analysis, for each of the video sequences, analyze phase variations between frames of the video sequence to determine local motion in the video sequence, and determine similarity between two or more of the video sequences based on the analysis of the phase variations.
  • an apparatus comprises at least one processor and memory storing a program of instructions.
  • the memory storing the program of instructions is configured to, with the at least one processor, configure the apparatus to at least identify motion layers of a video sequence based on phase difference information representing variations between frames of the video sequence and identify and classify types of motion in one or more of the motion layers.
  • an apparatus comprises at least one processor and memory storing a program of instructions.
  • the memory storing the program of instructions is configured to, with the at least one processor, cause the apparatus to at least analyze successive frames of a video sequence to extract phase difference information indicating motion, and create an interpolated updated frame between a pair of successive frames using phase information representing intermediate values between information for members of the pair of successive video frames.
  • a method comprises analyzing successive frames of a video sequence to extract phase difference information indicating motion, successively construct frames based on a still image, wherein successively constructing the frames based on a still image comprises using the phase difference information to successively update phase information, adding the phase information to successively created still image frames so as to introduce variations representing motion, and assembling the successively constructed frames to create a reconstructed output video sequence based on the still image.
  • a method comprises identifying two or more video sequences for similarity analysis; for each of the video sequences, analyzing phase variations between frames of the video sequence to determine local motion in the video sequence, and determining similarity between two or more of the video sequences based on the analysis of the phase variations.
  • a method comprises identifying motion layers of a video sequence based on phase difference information representing variations between frames of the video sequence, and identifying and classifying types of motion in one or more of the motion layers.
  • a method comprises analyzing successive frames of a video sequence to extract phase difference information indicating motion, and creating an interpolated updated frame between a pair of successive frames using phase information representing intermediate values between information for members of the pair of successive video frames.
  • a computer readable medium stores a program of instructions. Execution of the program of instructions by at least one processor configures an apparatus to at least analyze successive frames of a video sequence to extract phase difference information indicating motion, successively construct frames based on a still image, wherein successively constructing the frames based on a still image comprises using the phase difference information to successively update phase information and adding the phase information to successively created still image frames so as to introduce variations representing motion, and assemble the successively constructed frames to create a reconstructed output video sequence based on the still image.
  • a computer readable medium stores a program of instructions. Execution of the program of instructions by at least one processor configures an apparatus to at least identify two or more video sequences for similarity analysis, for each of the video sequences, analyze phase variations between frames of the video sequence to determine local motion in the video sequence, and determine similarity between two or more of the video sequences based on the analysis of the phase variations.
  • a computer readable medium stores a program of instructions. Execution of the program of instructions by at least one processor configures an apparatus to at least identify motion layers of a video sequence based on phase difference information representing variations between frames of the video sequence and identify and classify types of motion in one or more of the motion layers.
  • a computer readable medium stores a program of instructions. Execution of the program of instructions by at least one processor configures an apparatus to at least analyze successive frames of a video sequence to extract phase difference information indicating motion, and create an interpolated updated frame between a pair of successive frames using phase information representing intermediate values between information for members of the pair of successive video frames.
  • Fig. 1 illustrates a system for constructing an image with motion elements according to an embodiment of the present invention
  • FIGS. 2-5 illustrate processes according to an embodiment of the present invention
  • Figs. 6A-6I illustrate exemplary processing stages undertaken in processing frames according to an embodiment of the present invention.
  • Fig. 7 illustrates elements that may suitably be used to carry out an embodiment of the present invention.
  • One or more embodiments of the present invention provide mechanisms for matching a video sequence to a still image, analyzing frames of the video sequence to isolate information representing motion, and using the information representing motion to iteratively construct successive frames of the still image, with variations representing motion.
  • one or more video images in a set of video sequences is compared to a still image to which motion elements are to be added.
  • the best matched video sequence is selected, and all of the frames of the video sequence are decomposed into complex steerable pyramids.
  • a complex steerable pyramid can be represented as an amplitude pyramid and a phase pyramid.
  • the decomposition provides a complex, multi-scale, multi-orientation decomposition of the image (still image or frame of a video sequence).
  • Temporal phase variations between frames at each pixel maybe seen as encoding the motion in the video sequence.
  • Phase differences in the complex steerable pyramids representing consecutive frames of video can be used to represent motion elements, or changes caused by motion.
  • the motion elements representing motion between a pair of video frames can be mapped to corresponding elements of the still image and, thus, used to reconstruct a new, changed frame of the still image to provide the appearance of motion.
  • an output video is constructed, comprising a succession of such new, changed frames of the still image, with the new, changed frames being created in an iterative process.
  • a correspondence map is created based on visual similarity with the still image.
  • the phase pyramid of video frames is warped using the correspondence maps. After warping, phase differences between consecutive frames are determined, to generate a phase difference pyramid.
  • the phase difference pyramid is added to the phase pyramid of the last created frame of the output video (depending on the stage of the operation, either to the phase of the initial image frame or the phase of the preceding frame calculated in the previous iteration).
  • the updated phase pyramid and the amplitude pyramid are then used to create a reconstructed image, which can be stored to a video file.
  • complexity may be reduced by reconstructing only using a limited number of frames, particularly in cases in which the motion goes back and forth within a relatively tight bound. This may be the case with motions such as flames or swaying trees, for example.
  • Fig. 1 illustrates an exemplary processing and display device 100 according to an embodiment of the present invention.
  • the device 100 comprises a processor 102, memory 104, and long term storage 106, as well as a display 108.
  • the system 100 may store a collection of moving images, suitably stored in the form of video sequences (with each sequence suitably being stored as a video file in a desired format), with the collection taking the form of a video sequence database 110, residing in storage 106.
  • the system 100 may also store a collection of still images stored as a still image database 112, also residing in storage 106.
  • the system 100 may receive still images as inputs— for example through a user interface 114 or a remote interface 116.
  • the system 100 performs processing, suitably using an image processing application 118, residing in storage 106 and transferred to memory 104 as needed for execution by the processor 102. Processing is performed to select images, to perform matching between images and image frames, and to use a still image frame and video sequence frames to add motion elements to construct an animated image with motion elements.
  • the still image frame may, for example, be elected from among images stored in the still image database 112, or received as an input through the user interface 114 or the remote interface 116.
  • Processing to add motion elements to the image comprises selecting a video sequence similar to the image and with quahties conducive to extracting motion elements and adding the motion elements to the still image.
  • a suitable video sequence is, as previously noted, generally similar to the still image. Desired characteristics of a video sequence are, for example, that it does not change drastically with time and that the amplitude of the motion displayed falls within a specified range.
  • Fig. 2 illustrates a process 200 according to an embodiment of the present invention.
  • the process 200 is performed iteratively, with an initial still image being decomposed, and successive video frames being similarly decomposed.
  • Phase information representing motion between the successive video frames is used to determine variations to be used to represent motion of the scene depicted in the still image frame, and used to reconstruct a still image frame exhibiting the variations.
  • the process continues to successively add the warped phase variation from video sequence to the previously calculated phase of the preceding frame and reconstructing a succeeding frame exhibiting variation representing motion between successive pairs of video frames.
  • a selected still image frame, and successive frames of a video sequence identified as suitably similar to the still image are decomposed into complex steerable pyramids.
  • the pyramids representing phase information for the video sequence frames are analyzed to identify phase variations between corresponding pixels in successive frames.
  • Temporal phase variations at each pixel in this complex steerable pyramid encode the motion in the video sequence, and this phase information can be used to represent motion-caused changes between successive frames.
  • Phase differences between successive frames of the video represent motion, and the motion represented by phase differences can animate the still image by introducing appropriate variations into successive duplicate frames of the still image.
  • correspo dence maps are calculated, with each map being calculated based on visual similarity between the still image and each frame of video.
  • the phases of the complex steerable pyramid of video frames are warped using the correspondence maps.
  • phase differences between successive frames are calculated and at block 210, the phase differences are added to phases of the preceding output frame calculated in the previous iteration.
  • the newly computed phase information, and amplitude information is used to construct a new frame of the still image, varying from the previous frame by changes represented by the phase information.
  • postprocessing may be performed using the magnitude of the steerable pyramid decomposition of the original still image.
  • the analysis described above may, as noted, be accomplished by decomposition of each image into a complex steerable pyramid and comparison of elements of the still image to corresponding elements of the video sequence.
  • Decomposition may be accomplished using a set of rotated bandpass filters, with the exception that the initial and final filters are highpass and lowpass, respectively.
  • the decomposition thus achieved is a complex, multi-scale, multi- orientation decomposition of the image.
  • Basis functions for the transform thus achieved may resemble Gabor wavelets, and may suitably be over-complete.
  • video sequences are analyzed to determine motion similarity between pairs of sequences.
  • Each video frame of each video is decomposed to a complex steerable pyramid.
  • the phase differences between frames of the complex steerable pyramid correspond to motion.
  • Phase differences between corresponding elements of frames are used as descriptors to calculate motion similarity between videos.
  • Such analysis can also be used for activity recognition: for example, phase differences between corresponding elements of frames can be used as motion descriptors and these motion descriptors can be compared to descriptors characteristic of motions associated with particular activities, with the activity most closely fitted to the example being identified as the activity taking place.
  • FIG. 3 illustrates a process 300 according to an embodiment of the present invention.
  • each frame of a video is decomposed to a complex steerable pyramid.
  • pairs of successive frames are analyzed and phase differences between corresponding elements of each frame are calculated.
  • Blocks 302 and 304 may be performed for a number of videos.
  • a pair of video sequences for which phase difference values have been computed are compared in terms of phase difference values.
  • a similarity metric comparing the phase difference values is computed indicating the similarity between the two videos.
  • This similarity metric may be used to compare different pairs of videos in terms of their motion similarity to choose for example the most similar pair, trio, or other grouping of videos. Similarity may suitably be in terms of frequency of motion and locality (area of the image in which the motion occurs) and amplitude of motion.
  • a complex steerable pyramid is used to decompose video frames into magnitude and phase pyramids.
  • the phase differences between subsequent frames are then used as a feature to cluster pixels together into motion layers.
  • a phase difference encodes motion, and using the phase difference as a motion feature provides for an efficient, lower complexity estimation procedure.
  • figure 4 illustrates a process 400 according to an embodiment of the present invention.
  • a video sequence is selected which is presumed to be taken from a fixed viewpoint. If this is not the case, image alignment may be performed before processing in order to give the video presented for processing the appearance of a fixed viewpoint.
  • frames of the video are decomposed into complex steerable pyramids, which are represented by amplitude and phase pyramids. Subsequent blocks discussed below perform processing to estimate and segment motion layers.
  • phase differences are computed between the pyramids of every pair of successive frames.
  • a feature vector is created by concatenating the phase differences for that pixel between every pair of successive frames of the video.
  • clustering is performed to cluster the pixels in different layers based on the phase difference feature vectors.
  • one or more motion layers may be selected and successive frames of an output video maybe constructed to reflect changes in one or more selected motion layers.
  • a complex steerable pyramid may be used to decompose frames into magnitude and phase pyramids and then interpolate motion by interpolating the phase difference between the pyramids representing frame A and frame B, with motion being encoded by the phase differences.
  • Figure 5 therefore, illustrates a process 500 according to an embodiment of the present invention.
  • successive frames A and B are selected representing initial and succeeding frames of video in which it is desired to interpolate an intermediate frame.
  • the frames A and B are decomposed into amplitude and phase pyramids. Motion is then interpolated between frames A and B.
  • the phase differences between the pyramids of frames A and B are computed.
  • the phase difference value of each pixel is multiplied by a number between zero and 1, with the multiplier being chosen by the stage between frames A and B that is being represented. For example, if the intermediate frame is midway between frames A and B, the multiplier will be 0.5.
  • Figs. 6A-6I illustrate a succession of processing stages on an exemplary input still image and video image according to one or more embodiments of the present invention.
  • a still image in the present example, consisting of a single frame
  • a video image consisting of multiple successive frames
  • the video image is chosen based in part on its similarity to the still image.
  • the video image is processed (for example, in a pixel-by-pixel analysis) to identify changes in successive frames that represent motion, and extract (in the form of phase changes) information identifying such changes.
  • the phase change information is mapped to the still image in order to tie changes caused by motion to pixels of the still image, and variations are added to successive frames of the still image.
  • Fig. 6 A presents a diagram 600 illustrating a still image 602 and an exemplary video sequence 603, suitable for processing to extract motion elements. Successive frames 604A-604D of the video sequence 603 can be seen. Processing yields an animation 605 of the still image, of which frames 606A-606D can be seen.
  • the animation 605 is an output video constructed by processing the still image 602 and the video sequence 603, which are provided as inputs.
  • a still image such as the still image 602 may often have a higher resolution than does the frame 604A of the video sequence (as is often the case with dual- function still and video cameras that have become ubiquitous with the rise of portable electronic devices), and embodiments of the present invention may provide an image with a higher resolution than a typical video sequence, but the image thus produced may be made more engaging by realistic animation taken from a video sequence of real motion.
  • Fig. 6B presents a diagram 609, illustrating decomposition of the still image 602 into a complex steerable pyramid 610, which can be represented by amplitude and phase pyramids 612 A and 612B.
  • the pyramids represent decomposition of the image along multiple scales and orientations. As previously noted, decomposition may be accomplished with initial high pass, final low pass, and intermediary rotated bandpass filters.
  • Fig. 6C presents a diagram 613, illustrating the complex steerable pyramid decomposition of successive video frames of the video sequence 603.
  • Frames 614A and 614B of the sequence 603 are each decomposed into complex steerable pyramids representing amplitude and phase, and the phase portions of the pyramids 616A and 616B are processed to obtain phase difference information representing the motion between frames 614A and 14B.
  • phase differences are used to create variations in the still image, and this process is conducted iteratively to generate a changing succession of copies of the still frame, with variations mirroring the variations between frames of the video sequence. That is, second frame of the still image is created, varying from the original according to the variations between the first and second frames of the video sequence. A third frame is created, varying from the second frame according to the variations between the second and third frames of the video sequence, and so on.
  • Fig. 6D presents a diagram 617, illustrating an operation, performed at each iteration, of warping the phases of the pair of frames to the image to create a correspondence map between each output frame of the animated still image and the corresponding frame of the video sequence. Shown in Fig. 6D are the (N-l)th frame (618) of the reconstructed image and the (N-l)th frame (620A) of the video sequence. This operation maps the phases of the video sequence frame onto equivalent points of the reconstructed output video frame, creating (in the present example) a correspondence map CN-I .
  • Fig. 6E presents a diagram 619, illustrating warping of successive frames (for example 620A and 620B) of the video sequence, using the correspondence map CN-I-
  • the frames are decomposed into complex steerable pyramids 622A and 622B, which are warped using the correspondence map CN-I.
  • the phase information is thus made suitable for further processing and application to the still image frame to which it is to be applied.
  • Fig. 6F presents a diagram 623, providing a more expanded view of the process of decomposition of a still image frame 618, warping of successive video sequence frames, determination of phase differences between the video sequence frames, and their use in processing the still image frame.
  • the scenario presented in Fig. 6E is shown as the diagram 61 and in addition, phase information 628 is obtained from the complex steerable pyramid 626.
  • the (N-l)th and Nth frames of the selected video sequence are (as discussed above) decomposed into a complex steerable pyramid, and the phase pyramids are warped using the correspondence map CN-I to produce the warped pyramids 624A and 624B.
  • the phase pyramids 624A and 624B are processed to determine a warped phase difference between the (N-l)th frame and the Nth frame, and the phase difference information is added to the phase information of the (N-l)th frame of the reconstructed output video at iteration (N-l) to calculate the phase information for the reconstructed output video frame N that is to be constructed.
  • Fig. 6G presents a diagram 631, illustrating reconstruction of the Nth output frame, using the previously calculated phase in the (N-l)th iteration and the computed phase difference between the (N-l)th and the Nth frames.
  • the scenario 623 of Fig. 6F is illustrated here, and an amplitude pyramid 632 of. the output of the (N-l)th iteration is combined with the computed phase 630 for the Nth frame, to create the Nth reconstructed output frame 634.
  • Fig. 6H presents a diagram 635, illustrating post-processing that may be performed on the reconstructed output frame 634.
  • the reconstructed frame is decomposed into its complex steerable pyramid 638 and the amplitude of the decomposition is set to match the amplitude pyramid 640 of the original image 602.
  • Fig. 61 presents a diagram 645, illustrating an overall process of generating an Nth frame from an (N-l)th frame.
  • the phase and magnitude of (N-l)th frame of the reconstructed output video from previous iteration is used with the magnitude and phase of the complex steerable decomposition of the (N-l)th and Nth frame of the input video to construct frame N of the reconstructed output video.
  • the phase pyramids for the (N-l)th and Nth video frames are warped using the C N -i correspondence map.
  • the warped phase difference is computed and added to the phase pyramid for the (N-l)th frame of the constructed output video to yield phase information for reconstruction of the Nth frame.
  • the phase information is combined with the amplitude pyramid information for the (N-l)th frame and used to generate the Nth reconstructed frame. Further processing may be performed, and the Nth frame is added to a succession of previously created frames.
  • FIG. 7 presents details of a data processing device 700, suitable for carrying out one or more embodiments of the present invention.
  • the device 700 may suitably comprise a data processor (DP) 708 and memory (MEM) 710.
  • the device 700 may employ data 712 and programs (PROGS) 714, residing in memory 710.
  • DP data processor
  • MEM memory
  • PROGS programs
  • At least one of the PROGs 714 employed by the device 700 is assumed to include a set of program instructions that, when executed by the associated DP 708, enable the device to operate in accordance with embodiments of this invention.
  • embodiments of this invention may be implemented at least in part by computer software stored on the MEM 710, which is executable by the DP 708 of the device 700, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware).
  • Electronic devices implementing these aspects of the invention need not be the entire devices as depicted at Figure 1 or Fig. 7 or may be one or more components of same such as the above described tangibly stored software, hardware, firmware and DP, or a system on a chip SOC or an application specific integrated circuit ASIC.
  • the various embodiments of the device 700 can include, but are not limited to personal portable digital devices having wireless communication capabilities, including but not limited to cellular telephones, navigation devices, laptop/palmtop/tablet computers, digital cameras and music devices, and Internet appliances, or may also include dedicated data processing systems (for example, remotely accessible systems performing processing tasks submitted by remote users).
  • personal portable digital devices having wireless communication capabilities, including but not limited to cellular telephones, navigation devices, laptop/palmtop/tablet computers, digital cameras and music devices, and Internet appliances, or may also include dedicated data processing systems (for example, remotely accessible systems performing processing tasks submitted by remote users).
  • Various embodiments of the computer readable MEM 710 include any data storage technology type which is suitable to the local technical environment, including but not limited to semiconductor based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, removable memory, disc memory, flash memory, DRAM, SRAM, EEPROM and the like.
  • Various embodiments of the DP 708 include but are not limited to general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and multi-core processors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne des systèmes et des techniques de traitement d'images pour ajouter le mouvement d'une séquence vidéo à une image fixe. Les trames successives d'une séquence vidéo sont traitées pour extraire des informations de différence de phase entre des paires de trames successives. Le traitement peut avantageusement comprendre la décomposition de chaque trame dans une paire en une pyramide orientable complexe. Les informations de phase peuvent avantageusement être déformées en une trame d'image fixe à mettre à jour avec des informations de mouvement, en se basant sur une table de correspondance. Des informations de différence de phase sont déterminées et ajoutées aux informations de phase d'une trame d'image fixe qui doit être mise à jour. Cette addition crée des informations de phase de trame de sortie, et ces informations sont combinées avec des informations d'amplitude pour créer une trame mise à jour avec des variations par rapport à une trame précédente. Ces variations représentent le mouvement entre les trames successives d'une séquence vidéo. D'autres modes de réalisation comprennent la détermination de la similitude entre des séquences vidéo, la création de couches de mouvement et la création de trames interpolées.
PCT/US2014/072764 2014-12-30 2014-12-30 Procédés et appareil de traitement d'images à informations de mouvement Ceased WO2016108847A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2014/072764 WO2016108847A1 (fr) 2014-12-30 2014-12-30 Procédés et appareil de traitement d'images à informations de mouvement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2014/072764 WO2016108847A1 (fr) 2014-12-30 2014-12-30 Procédés et appareil de traitement d'images à informations de mouvement

Publications (1)

Publication Number Publication Date
WO2016108847A1 true WO2016108847A1 (fr) 2016-07-07

Family

ID=52440827

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/072764 Ceased WO2016108847A1 (fr) 2014-12-30 2014-12-30 Procédés et appareil de traitement d'images à informations de mouvement

Country Status (1)

Country Link
WO (1) WO2016108847A1 (fr)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101870355B1 (ko) * 2017-01-31 2018-06-22 서울대학교병원 영상 처리 방법
CN110268338A (zh) * 2017-02-09 2019-09-20 谷歌有限责任公司 使用视觉输入进行代理导航
US11055859B2 (en) 2018-08-22 2021-07-06 Ford Global Technologies, Llc Eccentricity maps
US11460851B2 (en) 2019-05-24 2022-10-04 Ford Global Technologies, Llc Eccentricity image fusion
US11521494B2 (en) 2019-06-11 2022-12-06 Ford Global Technologies, Llc Vehicle eccentricity mapping
US11662741B2 (en) 2019-06-28 2023-05-30 Ford Global Technologies, Llc Vehicle visual odometry
US11783707B2 (en) 2018-10-09 2023-10-10 Ford Global Technologies, Llc Vehicle path planning
US12046047B2 (en) 2021-12-07 2024-07-23 Ford Global Technologies, Llc Object detection
CN119741631A (zh) * 2024-11-26 2025-04-01 清华大学 一种计算机视觉的结构振动识别方法及装置

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140072228A1 (en) * 2012-09-07 2014-03-13 Massachusetts Institute Of Technology Complex-Valued Eulerian Motion Modulation

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140072228A1 (en) * 2012-09-07 2014-03-13 Massachusetts Institute Of Technology Complex-Valued Eulerian Motion Modulation

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
NEAL WADHWA ET AL: "Phase-based video motion processing", ACM TRANSACTIONS ON GRAPHICS, vol. 32, no. 4, 1 July 2013 (2013-07-01), pages 1, XP055154489, ISSN: 0730-0301, DOI: 10.1145/2461912.2461966 *
PIOTR DIDYK ET AL: "Joint view expansion and filtering for automultiscopic 3D displays", ACM TRANSACTIONS ON GRAPHICS (TOG), ACM, US, vol. 32, no. 6, 1 November 2013 (2013-11-01), pages 221:1 - 221:8, XP058033872, ISSN: 0730-0301, DOI: 10.1145/2508363.2508376 *
YUNG-YU CHUANG ET AL: "Animating pictures with stochastic motion textures", ACM TRANSACTIONS ON GRAPHICS, vol. 24, no. 3, 1 July 2005 (2005-07-01), pages 853, XP055213866, ISSN: 0730-0301, DOI: 10.1145/1073204.1073273 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101870355B1 (ko) * 2017-01-31 2018-06-22 서울대학교병원 영상 처리 방법
WO2018143565A1 (fr) * 2017-01-31 2018-08-09 서울대학교병원 Procédé de traitement d'images
CN110268338A (zh) * 2017-02-09 2019-09-20 谷歌有限责任公司 使用视觉输入进行代理导航
CN110268338B (zh) * 2017-02-09 2022-07-19 谷歌有限责任公司 使用视觉输入进行代理导航
US11055859B2 (en) 2018-08-22 2021-07-06 Ford Global Technologies, Llc Eccentricity maps
US11783707B2 (en) 2018-10-09 2023-10-10 Ford Global Technologies, Llc Vehicle path planning
US11460851B2 (en) 2019-05-24 2022-10-04 Ford Global Technologies, Llc Eccentricity image fusion
US11521494B2 (en) 2019-06-11 2022-12-06 Ford Global Technologies, Llc Vehicle eccentricity mapping
US11662741B2 (en) 2019-06-28 2023-05-30 Ford Global Technologies, Llc Vehicle visual odometry
US12046047B2 (en) 2021-12-07 2024-07-23 Ford Global Technologies, Llc Object detection
CN119741631A (zh) * 2024-11-26 2025-04-01 清华大学 一种计算机视觉的结构振动识别方法及装置

Similar Documents

Publication Publication Date Title
WO2016108847A1 (fr) Procédés et appareil de traitement d'images à informations de mouvement
Luo et al. Refusion: Enabling large-size realistic image restoration with latent-space diffusion models
US11544831B2 (en) Utilizing an image exposure transformation neural network to generate a long-exposure image from a single short-exposure image
US12067659B2 (en) Generating animated digital videos utilizing a character animation neural network informed by pose and motion embeddings
Pittaluga et al. Revealing scenes by inverting structure from motion reconstructions
Meng et al. 3d interacting hand pose estimation by hand de-occlusion and removal
Luc et al. Transformation-based adversarial video prediction on large-scale data
CN105654492B (zh) 基于消费级摄像头的鲁棒实时三维重建方法
US11443481B1 (en) Reconstructing three-dimensional scenes portrayed in digital images utilizing point cloud machine-learning models
CA3197841A1 (fr) Reseau d'attention multiresolution pour la reconnaissance d'actions dans une video
JP7355851B2 (ja) ビデオを識別するための方法及び装置
Gomes et al. Spatio-temporal graph-RNN for point cloud prediction
CN107851332A (zh) 经由拓扑知晓表面跟踪的一致细分
Ashir et al. Facial expression recognition based on image pyramid and single-branch decision tree
KR20200116728A (ko) 자체 운동 정보 추정 장치 및 방법
WO2020043296A1 (fr) Dispositif et procédé de division d'image en avant-plan et en arrière-plan, par apprentissage profond
CN118922854A (zh) 针对视频超分辨率的轨迹感知变换器
Huang et al. Evaluation of the impact of lossy compression on event camera-based computer vision tasks
CN113177483B (zh) 视频目标分割方法、装置、设备以及存储介质
Van Luong et al. Sparse signal reconstruction with multiple side information using adaptive weights for multiview sources
JP2024508930A (ja) ワンショットニューラルアーキテクチャ検索による軽量でリアルタイムな顔の位置合わせ
Wang et al. Object tracking with sparse representation and annealed particle filter
Voronin et al. Inpainting for videos with dynamic objects using texture and structure reconstruction
US20240394834A1 (en) Increasing resolution of digital images using self-supervised burst super-resolution
US12174922B2 (en) Method, computer program, computer program product and system for representing visual information

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14833441

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14833441

Country of ref document: EP

Kind code of ref document: A1