[go: up one dir, main page]

WO2011033675A1 - Appareil de traitement d'image et appareil d'affichage d'image - Google Patents

Appareil de traitement d'image et appareil d'affichage d'image Download PDF

Info

Publication number
WO2011033675A1
WO2011033675A1 PCT/JP2009/066446 JP2009066446W WO2011033675A1 WO 2011033675 A1 WO2011033675 A1 WO 2011033675A1 JP 2009066446 W JP2009066446 W JP 2009066446W WO 2011033675 A1 WO2011033675 A1 WO 2011033675A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion blur
image
filter
subject
motion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2009/066446
Other languages
English (en)
Japanese (ja)
Inventor
信幸 松本
孝 井田
賢造 五十川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Original Assignee
Toshiba Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp filed Critical Toshiba Corp
Priority to PCT/JP2009/066446 priority Critical patent/WO2011033675A1/fr
Publication of WO2011033675A1 publication Critical patent/WO2011033675A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20201—Motion blur correction

Definitions

  • the present invention relates to an image processing apparatus and an image display apparatus that suppress, for example, image blurring.
  • the above-described conventional technique has a problem in that motion blur is empirically suppressed only in accordance with the movement width of the subject at the time of shooting, and an unnatural image is generated.
  • the present invention provides an image processing apparatus and an image display apparatus capable of suppressing blur with a natural feeling.
  • An image processing apparatus calculates a shooting motion blur amount of a subject according to a motion width of the subject in an input image, and obtains a first motion blur filter that reproduces the shooting motion blur amount.
  • a calculation unit that calculates a pixel value conversion filter from a first motion blur filter and an inverse filter of the second motion blur filter, and a memory that stores a first image having a pixel value of the input image as an initial value And the first image is converted by the pixel value conversion filter to obtain a second image, and the difference between the pixel value of the input image and the pixel value of the corresponding pixel of the second image is reduced.
  • the image of the first image By correcting the value, it comprises a conversion unit for generating a motion blur suppressing image.
  • FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment. The figure explaining the imaging
  • FIG. 6 is a diagram for explaining the operation of a conversion unit 105. 6 is a flowchart showing a flow of operations performed by the image processing apparatus according to the first embodiment.
  • FIG. 5 is a block diagram of an image processing apparatus according to a second embodiment. 9 is a flowchart showing a flow of operations performed by the image processing apparatus according to the second embodiment.
  • FIG. 9 is a block diagram of an image display device according to a third embodiment.
  • FIG. 1 is a block diagram of the image processing apparatus of the present embodiment.
  • the image processing apparatus includes a shooting motion blur amount setting unit (first setting unit) 101, a display motion blur amount setting unit (second setting unit) 102, a motion blur suppression amount calculating unit (calculating unit) 103, and a storage.
  • Unit 104 and conversion unit 105 This image processing apparatus processes an input image y1 obtained by photographing a subject, and generates and outputs an image in which the blur amount of the subject in the input image y1 is suppressed.
  • the shooting motion blur amount setting unit 101 receives a signal representing the movement width of the subject in the input image (the movement width of the subject when shooting with the imaging device).
  • the shooting motion blur amount setting unit 101 calculates the blur amount of the subject (shooting motion blur amount) in the input image from the motion range indicated by the input signal, the camera opening time during shooting, and the shooting interval time during shooting. Then, a first motion blur filter that reproduces the shooting motion blur amount is set.
  • the signal representing the movement width may be received from the outside through user input, or may be read from a recording medium such as a memory. Information on the camera opening time and the shooting interval time may be given in advance or may be given from the outside.
  • the left side of FIG. 2 is a diagram showing the relationship between the movement width of the subject at the time of shooting, the shooting motion blur amount, the camera opening time, and the shooting interval time.
  • y 1 is the vector in which the pixel values of the motion blur-suppressed image of the subject at the time of shooting are arranged in raster scan order
  • x is the vector in which the pixel values of the image without motion blur are arranged in raster scan order
  • the first motion blur filter If the matrix representation of W 1 is W 1 , the relationship of y 1 , x, W 1 is
  • the filter W 1 is It becomes a circular matrix expressed as
  • the panning operation which is the main camera work, can be formulated with this expression because the movement of the entire screen is in the horizontal direction. If the movement direction is an oblique direction, the above-described movement in the horizontal direction can be formulated by rotating the entire screen so as to correct the movement direction in the horizontal direction.
  • the imaging motion blur amount setting unit 101 sends the set first motion blur filter W1 to the motion blur suppression amount calculation unit 103.
  • the shooting motion blur amount setting unit 101 calculates the shooting motion blur amount based on the movement width of the subject at the time of shooting, the opening time, and the shooting interval time, but the opening time and the shooting interval time are the same.
  • the first motion blur filter W1 may be set simply using the movement width of the subject at the time of shooting as the shooting motion blur amount.
  • a signal indicating the shooting motion blur amount is given to the shooting motion blur amount setting unit 101, and the shooting motion blur amount setting unit 101 performs a first operation based on the given signal.
  • One motion blur filter W1 may be set.
  • the display motion blur amount setting unit 102 receives an input of a signal indicating the movement width of the subject at the time of display.
  • the display motion blur amount setting unit 102 determines a display motion blur amount that is a desired blur amount at the time of display according to the motion width indicated in the signal, and a second motion blur filter that reproduces the display motion blur amount Set. More specifically, the display motion blur amount is an amount indicating how much motion blur is to be generated on each frame and still image frame of a moving image.
  • the movement width of the subject at the time of display can be used as the display movement blur amount, and therefore, the input movement width can be used as the display movement blur amount as it is.
  • the signal may be received from the outside via a user input, or may be read from a recording medium such as a memory.
  • the display motion blur amount can be a user-specified display motion blur amount input from the outside in order to reflect the motion blur amount intended by the creator. As a result, it is possible to perform processing such as converting motion blur in a still image photograph into a dynamic expression desired by the creator.
  • y 2 is a vector in which the pixel values of an image (motion blur-suppressed image) having an object movement width (display motion blur amount) at the time of display are arranged in raster scan order, and the pixel value of an image without motion blur is raster scanned.
  • the vectors ordered x (same as shooting motion blur amount setting unit), when the matrix representation of the second motion blur filter to reproduce the display motion blur and W 2, y 2, x, the relationship of W 2 is , It becomes.
  • the matrix W 2 of the second motion blur filter for example, when the movement width of the subject at the time of display is a movement of 4 pixels in the horizontal direction, It becomes a circular matrix expressed as
  • Motion blur suppression amount calculation unit 103 calculates a first motion blur filter W 1, the pixel value conversion filter and a second motion blur inverse filter of the filter W 2.
  • the pixel value conversion filter will be described.
  • a matrix W in Expression 3 represents a pixel value conversion filter according to the present embodiment.
  • the second motion blur filter W 2 is the horizontal direction of the first motion-blur filter W 1 (if to reproduce the 8 pixels of the motion blur) laterally in the case of eight pixels of the motion Pixel value conversion filter W based on Equation 3, if it is a matrix in the case of motion for minutes (when reproducing motion blur for 4 pixels) It becomes a circular matrix expressed as
  • the motion blur suppression amount calculation unit 103 sends the calculated pixel value conversion filter to the conversion unit 105.
  • a pixel value conversion filter is calculated and stored in advance for combinations of various values of the shooting motion blur amount of the subject at the time of shooting and the motion width (display motion blur amount) of the subject at the time of display. Also good.
  • the motion blur suppression amount calculation unit 103 receives data indicating the shooting motion blur amount from the shooting motion blur amount setting unit 101, and receives data indicating the display motion blur amount from the display motion blur amount setting unit 102.
  • the corresponding pixel value conversion filter is selected from the pixel value conversion filter group calculated in advance based on the received data. Thereby, the amount of processing can be reduced.
  • the opening time and the shooting interval time are equal, the movement width of the subject at the time of shooting may be used as the shooting motion blur amount.
  • the storage unit 104 is configured by a readable / writable storage medium such as a DRAM (memory) of a PC (personal computer), an HDD (hard disk), or an LSI frame memory.
  • DRAM dynamic random access memory
  • PC personal computer
  • HDD hard disk
  • LSI frame memory a readable / writable storage medium
  • FIG. 4 is a flowchart for explaining an operation in which the image processing apparatus of the present embodiment generates an output image in which motion blur is suppressed from an input image.
  • the shooting motion blur amount setting unit 101 determines the shooting motion blur amount in accordance with the movement width of the subject at the time of shooting, and sets a first motion blur filter that reproduces the shooting motion blur amount (step S201).
  • the display motion blur amount setting unit 102 determines the display motion blur amount according to the movement width of the subject at the time of display, and sets a second motion blur filter that reproduces the display motion blur amount (step S202). .
  • the motion blur suppression amount calculation unit 103 calculates a pixel value conversion filter from the first motion blur filter and the inverse filter of the second motion blur filter (step S203). Specifically, the pixel value conversion filter is calculated by multiplication of the first motion blur filter and the inverse filter of the second motion blur filter.
  • the storage unit 104 stores the input image, and generates and stores a temporary motion blur suppression image having each pixel value of the input image as an initial pixel value (step S204).
  • the conversion unit 105 performs a conversion process of the temporary motion blur suppression image stored in the storage unit 104 based on the input image stored in the storage unit 104 and the pixel value conversion filter (step S205). Specifically, first, one of the pixels of the input image is set as the target pixel. A pixel value of the provisional motion blur suppression image is converted by a pixel value conversion filter to calculate a trial calculation pixel value corresponding to the target pixel. Then, the pixel value of the temporary motion blur suppression image is converted (corrected) so that the difference amount between the pixel value of the target pixel and the trial calculation pixel value becomes small. Each other pixel of the input image is also set as a pixel of interest sequentially, and the above conversion process is repeated.
  • the conversion unit 105 determines whether the conversion process has been performed for a predetermined number of repetitions (step S206). Specifically, the conversion unit 105 determines whether the conversion process using the target pixel has been performed a predetermined number of times. If a determination result is NO, it will return to Step S205. If the determination result is YES, the provisional motion blur suppression image at that stage is output as a motion blur suppression image, and the process ends.
  • the operation of the present embodiment described so far is particularly effective when the entire screen is moving, such as camera panning.
  • the motion blur of only the subject can be suitably suppressed by performing the following operation. Two methods of operation are shown.
  • a moving area (a set of pixels) is detected from the input image, and y 1 , y 2 , W 1 , W 2 , x are defined only in that area, and conversion processing is performed.
  • the shooting blur amount setting unit sets the first motion blur filter W1 targeting only the region
  • the display blur amount setting unit sets the second motion blur filter W2 targeting only the region.
  • the conversion process is performed only on the pixels in the region of the input image and the pixels in the region of the temporary motion blur suppression image.
  • Information on the area may be notified from an external device to the imaging blur amount setting unit, the display blur amount setting unit, and the conversion unit, or a processing unit that detects the region is added, and the processing unit performs detection. May be.
  • the photographing blur amount setting unit and the display blur amount setting unit may operate in this way.
  • the motion blur suppression is performed by calculating the pixel value conversion filter from the motion width of the subject at the time of shooting, the opening time of the camera, and the motion width of the subject at the time of display (display motion blur amount).
  • display motion blur amount By performing the processing, it is possible to suppress the blur amount according to the movement width of the subject at the time of shooting, the opening time of the camera, and the movement width of the subject at the time of display, and a natural image can be generated.
  • FIG. 5 is a block diagram of the image processing apparatus of the present embodiment.
  • the image processing apparatus of this embodiment is obtained by adding a subject motion width estimation unit 401 to the image processing apparatus of the first embodiment shown in FIG.
  • the subject movement width estimation unit 401 estimates the movement width of the subject from the input image. For example, the motion width of the subject at the time of shooting is estimated by using a general motion vector calculation process such as a gradient method or a region matching method between two input images that are temporally continuous. Since the other components 101 to 105 are the same as those in the first embodiment, a duplicate description is omitted.
  • FIG. 6 is a flowchart for explaining an operation in which the image processing apparatus of the present embodiment outputs an output image in which motion blur is suppressed from the input image.
  • the subject motion width estimation unit 401 estimates the subject motion width from the input image (step S501).
  • the shooting motion blur amount setting unit (first setting unit) 101 determines the shooting motion blur amount according to the estimated movement width of the subject, and the first motion blur for reproducing the shooting motion blur amount
  • a filter is set (step S502).
  • the display motion blur amount setting unit (second setting unit) 102 determines the display motion blur amount according to the movement width of the subject at the time of display, and the second motion blur for reproducing the display motion blur amount.
  • a filter is set (step S503).
  • the motion blur suppression amount calculation unit (calculation unit) 103 calculates a pixel value conversion filter from the first motion blur filter and the inverse filter of the second motion blur filter (step S504).
  • the storage unit 104 stores the input image, and generates and stores a temporary motion blur suppression image having each pixel value of the input image as an initial value (step S505).
  • the conversion unit 105 performs pixel value conversion processing of the temporary motion blur suppression image stored in the storage unit 104 (step S506).
  • the conversion method is as described in the first embodiment.
  • the conversion unit 105 determines whether the conversion process has been performed for a predetermined number of repetitions (step S507). If the determination result is NO, the process returns to step S506, and if YES, the temporary motion blur suppression image at that stage is output as a motion blur suppression image, and the process ends.
  • the blur amount can be suppressed according to the movement width of the subject at the time of shooting, the opening time of the camera, and the movement width of the subject at the time of display, and a natural image can be generated.
  • FIG. 7 is a block diagram of the image display apparatus of the present embodiment.
  • This image display device includes an image processing device 100 and an image display unit 200.
  • the image processing apparatus 100 has the same configuration as the image processing apparatus shown in FIG.
  • the image display unit 200 displays the output image obtained by the image processing apparatus 100.
  • the image display unit 200 is connected to the image processing apparatus 100 of the first embodiment, but is connected to the image processing apparatus of the second embodiment, and displays an output image obtained by the image processing apparatus. May be.
  • the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying constituent elements without departing from the scope of the invention in the implementation stage.
  • various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment.
  • constituent elements over different embodiments may be appropriately combined.
  • the image processing apparatuses in the first and second embodiments and the image display apparatus in the third embodiment described above can also be realized by using, for example, a general-purpose computer device as basic hardware. Is possible.
  • the program to be executed has a module configuration including each function described above.
  • the program is an installable or executable file that is recorded on a computer-readable recording medium such as a CD-ROM, CD-R, DVD, etc. Also good.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

Selon l'invention, une image à flou de mouvement peut être générée avec une quantité de suppression calculée à partir d'une largeur de mouvement d'un sujet au moment de prise d'image, d'un temps d'ouverture d'appareil photo et d'une largeur de mouvement du sujet au moment d'affichage. Une unité de réglage de quantité de flou de mouvement à la prise d'image (101) règle un premier filtre de flou de mouvement à partir d'une largeur de mouvement d'un sujet survenant au moment de prise d'image. Une unité de réglage de quantité de flou de mouvement à l'affichage (102) règle un second filtre de flou de mouvement à partir d'une largeur de mouvement du sujet survenant au moment d'affichage. Une unité de calcul de quantité de suppression de flou de mouvement (103) calcule un filtre de conversion de valeur de pixel à partir du premier filtre de flou de mouvement et d'un filtre inverse du second filtre de flou de mouvement. Une unité de stockage (104) stocke une première image dont les valeurs de pixel initiales sont égales aux valeurs de pixel d'une image d'entrée. Une unité de conversion (105) corrige les valeurs de pixel de la première image de telle manière que la différence entre la valeur de pixel de chaque pixel d'une seconde image, qui est obtenue par utilisation de la première image et du filtre de conversion de valeur de pixel, et la valeur de pixel du pixel correspondant respectif de l'image d'entrée soit petite.
PCT/JP2009/066446 2009-09-18 2009-09-18 Appareil de traitement d'image et appareil d'affichage d'image Ceased WO2011033675A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2009/066446 WO2011033675A1 (fr) 2009-09-18 2009-09-18 Appareil de traitement d'image et appareil d'affichage d'image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2009/066446 WO2011033675A1 (fr) 2009-09-18 2009-09-18 Appareil de traitement d'image et appareil d'affichage d'image

Publications (1)

Publication Number Publication Date
WO2011033675A1 true WO2011033675A1 (fr) 2011-03-24

Family

ID=43758301

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2009/066446 Ceased WO2011033675A1 (fr) 2009-09-18 2009-09-18 Appareil de traitement d'image et appareil d'affichage d'image

Country Status (1)

Country Link
WO (1) WO2011033675A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9344626B2 (en) 2013-11-18 2016-05-17 Apple Inc. Modeless video and still frame capture using interleaved frames of video and still resolutions
JPWO2019044328A1 (ja) * 2017-08-31 2020-10-15 ソニー株式会社 医療用画像処理装置、医療用画像処理システム、及び医療用画像処理装置の駆動方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006081150A (ja) * 2004-08-11 2006-03-23 Sony Corp 画像処理装置および方法、記録媒体、並びにプログラム
JP2007114466A (ja) * 2005-10-20 2007-05-10 Toshiba Corp 手ぶれ補正機能を有する撮影装置
JP2009094617A (ja) * 2007-10-04 2009-04-30 Sony Corp 画像処理装置および方法、並びにプログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006081150A (ja) * 2004-08-11 2006-03-23 Sony Corp 画像処理装置および方法、記録媒体、並びにプログラム
JP2007114466A (ja) * 2005-10-20 2007-05-10 Toshiba Corp 手ぶれ補正機能を有する撮影装置
JP2009094617A (ja) * 2007-10-04 2009-04-30 Sony Corp 画像処理装置および方法、並びにプログラム

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9344626B2 (en) 2013-11-18 2016-05-17 Apple Inc. Modeless video and still frame capture using interleaved frames of video and still resolutions
TWI566598B (zh) * 2013-11-18 2017-01-11 蘋果公司 無模式之視訊及靜止訊框擷取
US9591219B2 (en) 2013-11-18 2017-03-07 Apple Inc. Modeless video and still frame capture
US10038845B2 (en) 2013-11-18 2018-07-31 Apple Inc. Modeless video and still frame capture
US10498960B2 (en) 2013-11-18 2019-12-03 Apple Inc. Modeless video and still frame capture
US10911673B2 (en) 2013-11-18 2021-02-02 Apple Inc. Modeless video and still frame capture
JPWO2019044328A1 (ja) * 2017-08-31 2020-10-15 ソニー株式会社 医療用画像処理装置、医療用画像処理システム、及び医療用画像処理装置の駆動方法
JP7160041B2 (ja) 2017-08-31 2022-10-25 ソニーグループ株式会社 医療用画像処理装置、医療用画像処理システム、及び医療用画像処理装置の駆動方法

Similar Documents

Publication Publication Date Title
KR101274061B1 (ko) 화상 처리장치 및 화상 처리방법
JP5531194B2 (ja) 画像処理装置、画像処理方法及び画像処理プログラム
US8233062B2 (en) Image processing apparatus, image processing method, and imaging apparatus
US8390698B2 (en) Image capturing apparatus, reproduction apparatus, image capturing method, and reproduction method
JP4674620B2 (ja) 画像処理装置、画像処理方法、及びプログラム
JP2009194896A (ja) 画像処理装置及び方法並びに撮像装置
EP2164040A1 (fr) Système et procédé pour une grande qualité d'image et l'interpolation vidéo
JP2010200179A (ja) 画像処理装置、画像処理方法、画像処理プログラムおよび画像処理プログラムが格納されたプログラム記憶媒体
JP2013003610A (ja) 画像処理装置および方法、プログラム、並びに記録媒体
JP2009081574A (ja) 画像処理装置、方法およびプログラム
JP2014143530A (ja) 情報処理装置と情報処理方法および撮像装置
JP2009088935A (ja) 画像記録装置、画像補正装置及び撮像装置
JP2012109656A (ja) 画像処理装置及び方法、並びに画像表示装置及び方法
JP2005150903A (ja) 画像処理装置、ノイズ除去方法及びノイズ除去プログラム
JP5919086B2 (ja) 画像処理装置及びその制御方法、並びにプログラム
JP6995490B2 (ja) 映像再生装置とその制御方法及びプログラム
WO2011033675A1 (fr) Appareil de traitement d'image et appareil d'affichage d'image
JP2010073074A (ja) 画像信号処理装置、画像表示装置、録画再生装置、画像信号処理方法
JP4927005B2 (ja) 変化要因情報のデータの生成法および信号処理装置
JP2009153046A (ja) ぶれ補正装置及び方法並びに撮像装置
JP5401696B2 (ja) 画像処理装置、画像処理方法及び画像処理プログラム
JP4606976B2 (ja) 画像処理装置
JP2009088933A (ja) 画像記録装置、画像補正装置及び撮像装置
JP2007179211A (ja) 画像処理装置、画像処理方法、およびそのプログラム
JP5495500B2 (ja) 変化要因情報のデータの生成法および信号処理装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09849534

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09849534

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP