[go: up one dir, main page]

WO2016159613A1 - Procédé et système de suivi d'objet pour synthèse d'image - Google Patents

Procédé et système de suivi d'objet pour synthèse d'image Download PDF

Info

Publication number
WO2016159613A1
WO2016159613A1 PCT/KR2016/003142 KR2016003142W WO2016159613A1 WO 2016159613 A1 WO2016159613 A1 WO 2016159613A1 KR 2016003142 W KR2016003142 W KR 2016003142W WO 2016159613 A1 WO2016159613 A1 WO 2016159613A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
target image
area
region
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2016/003142
Other languages
English (en)
Korean (ko)
Inventor
전수영
권지용
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
STARSHIP VENDING-MACHINE CORP
Original Assignee
STARSHIP VENDING-MACHINE CORP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by STARSHIP VENDING-MACHINE CORP filed Critical STARSHIP VENDING-MACHINE CORP
Publication of WO2016159613A1 publication Critical patent/WO2016159613A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing

Definitions

  • the present invention relates to a method and a system for tracking an object for synthesizing an image. More particularly, the present invention relates to a target image to which an object is synthesized. The present invention relates to a method and system for determining an expected area by calculating a similarity of an expected area to which an object is synthesized for each frame.
  • Image synthesis refers to a technique of generating a new image by synthesizing only a patch region, which is a region of a specific object, in an arbitrary original image to an arbitrary target image.
  • Such image synthesis is divided into global image synthesis and local image synthesis. Unlike the global image synthesis, the local image synthesis is synthesized according to the movement of a specific object.
  • a marker is attached to a part that needs to be synthesized in the step of shooting a target image, and then the marker is tracked by visual recognition technique to synthesize a special effect image and erase the marker. There is a way.
  • Korean Laid-Open Patent Publication No. 10-2009-0100741 relates to an image synthesis method, comprising: detecting a subject region from an image signal received from a camera, synthesizing a subject region and a background image, Checking whether the predetermined object regions in the background image overlap with each other; if the object region and the predetermined object region in the background region overlap, calculating the pixel area of the overlapping portion; and if the calculated pixel area exceeds the predetermined sensitivity, Interactive between the background image and the subject during the image synthesis using chroma key by generating a touch event for the subject and calculating the collision speed and the collision angle between the subject and the object by calculating the movement speed of the subject and the movement speed of the object. It is effective to realize one screen.
  • the technique disclosed in the above patent document does not solve the problem described above.
  • the background art described above is technical information that the inventors possess for the derivation of the present invention or acquired in the derivation process of the present invention, and is not necessarily a publicly known technique disclosed to the general public before the application of the present invention. .
  • One embodiment of the present invention is intended to relatively accurately track the position of an object while maintaining user convenience by the user designating an expected position of the object synthesized in the target image in advance.
  • an embodiment of the present invention by specifying the expected position of the object synthesized in the target image in advance, it is possible to relatively accurately track the position of the object while maintaining user convenience Can be.
  • any one of the problem solving means of the present invention by receiving the expected position of the object by the user by limiting the area on the frame for tracking the position of the object for each frame of the target image according to the position tracking of the object The complexity can be reduced, enabling fast and accurate object tracking.
  • FIG. 1 is a block diagram showing a configuration of an image synthesis system according to an embodiment of the present invention.
  • FIG. 2 is a flowchart illustrating an image synthesis method according to an embodiment of the present invention.
  • FIG 3 is an exemplary view for explaining an image synthesis method according to an embodiment of the present invention.
  • the "original image” is an image that is an object of image synthesis, and the original image includes a "object” such as a specific object or a region selected by the user, and the destination image where the object is synthesized is called a "target image”.
  • the 'expected area' is the area where the object is to be synthesized on each frame of the target image.
  • the image synthesis system 100 may be implemented by one electronic device, or may be implemented by various electronic devices.
  • the image synthesis system 100 may be implemented on a user terminal.
  • the image synthesis system 100 may be implemented, for example, on a user terminal on which a client is installed and on a server communicating with the client through a network.
  • the image synthesizing system 100 may determine an expected region in which the object is to be synthesized in the target image by using a guide region selected by the user to synthesize the object selected by the user to the target image.
  • the position input unit 210 may receive a region where the object is synthesized from the user for each frame of the target image as a guide region, and store the guide region input for each frame.
  • the guide region of the start frame of the target image is used as a focus value for calculating the similarity of the predicted region of each frame in the similarity calculator 220 to be described later, it is preferable to receive an accurate position from which the object is to be synthesized. Do.
  • the image synthesis system 100 includes a similarity calculator 220.
  • the similarity calculator 220 of the image synthesis system 100 may calculate the similarity of the predicted region of each frame of the target image based on the pixel value of the predicted region that is the region where the object is to be synthesized for each frame of the target image.
  • the similarity calculator 220 calculates a similarity by calculating a difference between the predicted pixel value of the frame at the time t + 1 of the target image and the predicted pixel value of the frame at the time t + 1 of the plurality of frames of the target image. Can be.
  • Equation 1 Is a value for evaluating the similarity of the prediction region between frames.
  • the difference between the predicted areas may be calculated through various methods. For example, a method of calculating the sum of the difference between the pixel values corresponding to the pixels within each prediction area and calculating the sum thereof may be used.
  • Equation 2 P means the position of each pixel in the prediction region.
  • Equation 2 is a function Within Contains unknowns, such unknowns.
  • the similarity calculator 220 may designate the guide region received by the location input unit 210 as the initial solution as the initial solution for the similarity calculation.
  • each repeated optimization step is further subdivided to solve the current iteration step. Part of the steps broken down into Add to solve for the next iteration Can be obtained.
  • Equation 2 which measures the similarity between similar regions using Equation 3
  • Equation 4 Equation 4
  • the image synthesis system 100 includes an acceleration calculator 230.
  • the acceleration calculator 230 of the image synthesis system 100 may calculate the acceleration of the predicted region of each frame of the target image based on the position of the predicted region on each frame of the target image.
  • Equation 5 Indicates the degree of movement of the predicted area in each frame. Is the second derivative of time, i.e. acceleration.
  • the image synthesis system 100 includes a space calculating unit 240.
  • the separation calculator 240 of the image synthesis system 100 may calculate a separation degree that is a difference between the position of the expected area and the guide area for each frame of the target image.
  • the image synthesis system 100 can reduce the area to be calculated in order to track the object for each frame, thereby reducing the complexity, and can accurately estimate the position of the area where the object is synthesized for each frame.
  • Equation 6 is the position of the user's guide area And the location of the estimated area that is actually calculated It can be determined whether is similar.
  • the separation calculation unit 240 is the position of the expected area so that the value of Equation 6 has a minimum value It can be limited to the position of the guide area of the user.
  • the image synthesis system 100 may include an area determiner 250.
  • the region determiner 250 of the image synthesis system 100 may determine the position of the expected region for each frame of the target image.
  • the area determiner 250 may determine a location for minimizing similarity, acceleration, and spacing for each frame of the target image as the location of the expected area.
  • the position where the calculation result of Equation 2 and Equation 3 is minimized is the position of the expected region where the object is synthesized.
  • Equation 8 may determine the predicted position more accurately and faster than the position determined by Equation 7 by determining a location where the total value is minimized by adding weights to the values of similarity, acceleration, and separation for each frame.
  • the projection synthesis method according to the embodiment shown in FIG. 2 includes the steps that are processed in time series in the image synthesis system 100 shown in FIG. Therefore, even if omitted below, the above description of the image synthesis system 100 shown in FIG. 1 may be applied to the image synthesis method according to the embodiment shown in FIG. 2.
  • the image synthesis system 100 may receive a guide region, which is an area designated by a user, for each frame of a target image including a plurality of frames (S2001).
  • the image synthesis system 100 may provide a target image to a user through a client, and receive a guide area from the user through an input method such as a touch input or a mouse cursor for each frame.
  • the image synthesis system 100 may calculate the similarity of the predicted regions of each frame of the target image based on the pixel value of the predicted region which is the region where the object is to be synthesized for each frame of the target image (S2002).
  • the image synthesis system 100 may calculate a difference between the expected area pixel value of the frame at time t of the target image and the expected area pixel value of the frame at time t + 1 of the target image.
  • the image synthesis system 100 may use the guide area input from the user as the predicted area for the frame at the time t of the target image to calculate the similarity of the predicted area for each frame of the target image.
  • the similarity of the expected area of the frame at time t can be calculated based on the guide area for the frame at.
  • the similarity calculation can solve the problem by an iterative optimization method as a nonlinear optimization problem.
  • the image synthesis system 100 has a similarity degree between the predicted region 304 of the frame 301 at the time t of the target image and the predicted region 305 of the frame 302 at the time t + 1.
  • the position of the guide region 303 of the frame 301 at the time t can be designated as the position of the expected region 304 to start the calculation of the similarity.
  • the image synthesis system 100 may predict the pixel region between the predicted region 304 of the frame 301 at time t and the predicted region 305 of the frame 302 at time t + 1 to minimize the difference in pixel values.
  • the time t at which the difference in pixel values is minimized by repeating the step of changing the position of the position to the expected area 306 of the frame 301 at time t and the expected area 307 of the frame 302 at time t + 1.
  • the predicted area of the frame 301 at can be found.
  • the image synthesis system 100 may calculate the acceleration of the predicted area of each frame of the target image based on the position of the predicted area on each frame of the target image (S2003).
  • the image synthesis system 100 generates a frame at time t-1, a frame at time t, and a frame at time t + 1 of the target image. Based on the position of the predicted area, the acceleration of the predicted area of the frame at the time t can be calculated.
  • the image synthesis system 100 may calculate a separation degree that is a difference between the position of the expected area and the position of the guide area for each frame of the target image (S2004).
  • the image synthesizing system 100 reduces the calculation amount for determining the position at which the object is synthesized and calculates the separation distance, which is the difference between the position of the predicted area and the position of the guide area, for a more precise result, from the guide area received from the user. Only an area within a certain distance can be set as an expected area.
  • the image synthesis system 100 may calculate the sum of the similarities, accelerations, and separations calculated in steps S2002 to S2004 for the predicted region of each frame of the target image, and determine whether the calculated value is the minimum. (S2005).
  • the image synthesis system 100 repeats steps S2002 to S2005 while moving the positions of the predicted regions until the sum of the similarity, acceleration, and spacing in the predicted region for each frame of the target image no longer decreases. can do.
  • the image synthesizing system 100 may repeat steps S2002 to S2005 to determine a position of an expected region in which the sum of similarity, acceleration, and separation degree is minimum for each frame of the target image as the position at which the object is synthesized (S2006).
  • the image synthesis method according to the embodiment described with reference to FIG. 2 may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer.
  • Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media.
  • computer readable media may include both computer storage media and communication media.
  • Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.
  • the image synthesis method may be implemented as a computer program (or computer program product) including instructions executable by a computer.
  • the computer program includes programmable machine instructions processed by the processor and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language.
  • the computer program may also be recorded on tangible computer readable media (eg, memory, hard disks, magnetic / optical media or solid-state drives, etc.).
  • the image synthesis method may be implemented by executing the computer program as described above by the computing device.
  • the computing device may include at least a portion of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device.
  • a processor may include at least a portion of a processor, a memory, a storage device, a high speed interface connected to the memory and a high speed expansion port, and a low speed interface connected to the low speed bus and the storage device.
  • Each of these components are connected to each other using a variety of buses and may be mounted on a common motherboard or otherwise mounted in a suitable manner.
  • the processor may process instructions within the computing device, such as to display graphical information for providing a graphical user interface (GUI) on an external input, output device, such as a display connected to a high speed interface. Instructions stored in memory or storage. In other embodiments, multiple processors and / or multiple buses may be used with appropriately multiple memories and memory types.
  • the processor may also be implemented as a chipset consisting of chips comprising a plurality of independent analog and / or digital processors.
  • the memory also stores information within the computing device.
  • the memory may consist of a volatile memory unit or a collection thereof.
  • the memory may consist of a nonvolatile memory unit or a collection thereof.
  • the memory may also be other forms of computer readable media, such as, for example, magnetic or optical disks.
  • the storage device can provide a large amount of storage space to the computing device.
  • the storage device may be a computer readable medium or a configuration including such a medium, and may include, for example, devices or other configurations within a storage area network (SAN), and may include a floppy disk device, a hard disk device, an optical disk device, Or a tape device, flash memory, or similar other semiconductor memory device or device array.
  • SAN storage area network
  • the present invention receives a guide area to which an object is synthesized to a target image to be synthesized from a user, and calculates a similarity of the predicted area to be synthesized for each frame of the target image based on the input guide area to determine the predicted area.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé et un système de suivi d'objet pour synthèse d'image. Selon un premier aspect de la présente invention, un système de synthèse d'image permettant de suivre une zone d'un objet pour la synthèse d'image dans une image cible avec lequel l'objet est synthétisé peut comprendre : une unité d'entrée de position permettant de recevoir une entrée d'une zone de guidage qui est une zone désignée par un utilisateur pour chaque trame de l'image cible contenant de multiples trames ; une unité de calcul de similitude permettant de calculer, pour chaque trame de l'image cible, la similitude d'une zone prévue de chaque trame de l'image cible, qui est une zone où l'objet doit être synthétisé, en fonction de la valeur de pixel de la zone prévue ; une unité de calcul d'accélération permettant de calculer l'accélération de la zone prévue de chaque trame de l'image cible en fonction de la position de la zone prévue sur chaque trame de l'image cible ; et une unité de détermination de zone permettant de déterminer, pour chaque trame de l'image cible, la position où l'objet est synthétisé.
PCT/KR2016/003142 2015-03-31 2016-03-28 Procédé et système de suivi d'objet pour synthèse d'image Ceased WO2016159613A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020150044825A KR101555876B1 (ko) 2015-03-31 2015-03-31 영상 합성을 위한 객체 추적 방법 및 시스템
KR10-2015-0044825 2015-03-31

Publications (1)

Publication Number Publication Date
WO2016159613A1 true WO2016159613A1 (fr) 2016-10-06

Family

ID=54249147

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2016/003142 Ceased WO2016159613A1 (fr) 2015-03-31 2016-03-28 Procédé et système de suivi d'objet pour synthèse d'image

Country Status (2)

Country Link
KR (1) KR101555876B1 (fr)
WO (1) WO2016159613A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020015539A1 (fr) * 2018-07-17 2020-01-23 中兴通讯股份有限公司 Procédé de traitement de données, dispositif de traitement de données et support de stockage lisible par ordinateur

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112150398B (zh) * 2019-06-27 2024-03-22 杭州海康威视数字技术股份有限公司 图像合成方法、装置及设备
CN112270349B (zh) * 2020-10-23 2023-02-21 福州大学 基于gcn-lstm的个体位置预测方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100331050B1 (ko) * 2000-06-01 2002-04-19 송종수 동영상 데이터상의 객체 추적 방법
JP2006042066A (ja) * 2004-07-28 2006-02-09 Toshiba Corp 動画像情報伝送装置および伝送画像生成方法
KR20090100741A (ko) * 2008-03-21 2009-09-24 (주)아이뮤직소프트 영상 합성 방법
KR101319667B1 (ko) * 2012-05-24 2013-10-17 삼성에스디에스 주식회사 증강현실 제공 시스템 및 그 방법과 이에 사용되는 관심 영역 트래킹 방법 및 단말기
KR20140103046A (ko) * 2013-02-15 2014-08-25 삼성전자주식회사 객체 추적 방법 및 이를 지원하는 전자 장치

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100331050B1 (ko) * 2000-06-01 2002-04-19 송종수 동영상 데이터상의 객체 추적 방법
JP2006042066A (ja) * 2004-07-28 2006-02-09 Toshiba Corp 動画像情報伝送装置および伝送画像生成方法
KR20090100741A (ko) * 2008-03-21 2009-09-24 (주)아이뮤직소프트 영상 합성 방법
KR101319667B1 (ko) * 2012-05-24 2013-10-17 삼성에스디에스 주식회사 증강현실 제공 시스템 및 그 방법과 이에 사용되는 관심 영역 트래킹 방법 및 단말기
KR20140103046A (ko) * 2013-02-15 2014-08-25 삼성전자주식회사 객체 추적 방법 및 이를 지원하는 전자 장치

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020015539A1 (fr) * 2018-07-17 2020-01-23 中兴通讯股份有限公司 Procédé de traitement de données, dispositif de traitement de données et support de stockage lisible par ordinateur
CN110730375A (zh) * 2018-07-17 2020-01-24 中兴通讯股份有限公司 一种数据处理方法、装置及计算机可读存储介质

Also Published As

Publication number Publication date
KR101555876B1 (ko) 2015-09-30

Similar Documents

Publication Publication Date Title
CN110866496B (zh) 基于深度图像的机器人定位与建图方法和装置
WO2015005577A1 (fr) Appareil et procédé d'estimation de pose d'appareil photo
EP3639243A1 (fr) Détermination et suivi de pose de caméra
JP7204786B2 (ja) 視覚的検索方法、装置、コンピュータ機器及び記憶媒体
WO2020071701A1 (fr) Procédé et dispositif de détection d'un objet en temps réel au moyen d'un modèle de réseau d'apprentissage profond
WO2016003191A1 (fr) Système et procédé basés sur une reconnaissance optique de caractères pour reconnaître une image de carte, support d'enregistrement et système de distribution de fichier
CN105308618B (zh) 借助于并行检测和跟踪和/或分组特征运动移位跟踪的人脸识别
WO2015178542A1 (fr) Appareil de mesure de paramètre de caméra et procédé associé
WO2016159613A1 (fr) Procédé et système de suivi d'objet pour synthèse d'image
WO2020256517A2 (fr) Procédé et système de traitement de mappage de phase automatique basés sur des informations d'image omnidirectionnelle
WO2021221334A1 (fr) Dispositif de génération de palette de couleurs formée sur la base d'informations gps et de signal lidar, et son procédé de commande
WO2021025364A1 (fr) Procédé et système utilisant un lidar et une caméra pour améliorer des informations de profondeur concernant un point caractéristique d'image
WO2016021830A1 (fr) Procédé et dispositif de correction d'informations de mouvement collectées par un dispositif nui
CN106096659A (zh) 图像匹配方法和装置
WO2021125578A1 (fr) Procédé et système de reconnaissance de position reposant sur un traitement d'informations visuelles
WO2016021829A1 (fr) Procédé et de reconnaissance de mouvement et dispositif de reconnaissance de mouvement
WO2020111844A2 (fr) Procédé et appareil pour améliorer un point caractéristique d'image dans le slam visuel à l'aide d'une étiquette d'objet
WO2013077546A1 (fr) Appareil et procédé de détection d'une modification d'une scène dans une vidéo stéréoscopique
WO2022154467A1 (fr) Procédé et système d'apprentissage pour aligner de manière temporelle des signaux avec des valeurs aberrantes intercalées
WO2017018701A1 (fr) Procédé et appareil pour prévenir la fragmentation de données de boîte noire
WO2019194561A1 (fr) Procédé et système de reconnaissance d'emplacement pour fournir une réalité augmentée dans un terminal mobile
WO2017007047A1 (fr) Procédé et dispositif de compensation de la non-uniformité de la profondeur spatiale en utilisant une comparaison avec gigue
WO2025013985A1 (fr) Procédé d'entraînement d'un détecteur d'objet pour prédire le centre de gravité d'un objet projeté sur le sol, procédé d'utilisation d'un détecteur d'objet entraîné pour identifier ledit objet dans un espace spécifique filmé par une pluralité de caméras présentant différents troncs de visualisation, et dispositif d'entraînement et dispositif d'identification d'objet qui utilisent les procédés
WO2019009579A1 (fr) Procédé et appareil de correspondance stéréo utilisant une interpolation à points de support
CN116129154A (zh) 图像目标关联方法、计算机设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16773394

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15.01.2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16773394

Country of ref document: EP

Kind code of ref document: A1