US9270878B2 - Imaging method using multifocal lens - Google Patents

Imaging method using multifocal lens Download PDF

Info

Publication number: US9270878B2
Authority: US; United States
Prior art keywords: image; focusing; subject; imaging; region
Prior art date: 2011-10-28
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Active

Application number

US14/262,822

Other languages

English (en)

Other versions

US20140232928A1 (en

Inventor

Shuji Ono

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Fujifilm Corp

Original Assignee

Fujifilm Corp

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2011-10-28

Filing date

2014-04-28

Publication date

2016-02-23

2014-04-28 Application filed by Fujifilm Corp filed Critical Fujifilm Corp

2014-04-28 Assigned to FUJIFILM CORPORATION reassignment FUJIFILM CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ONO, SHUJI

2014-08-21 Publication of US20140232928A1 publication Critical patent/US20140232928A1/en

2016-02-23 Application granted granted Critical

2016-02-23 Publication of US9270878B2 publication Critical patent/US9270878B2/en

Status Active legal-status Critical Current

2032-10-23 Anticipated expiration legal-status Critical

Images

Classifications

- H04N5/23212—
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0075—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. increasing, the depth of field or depth of focus
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B3/00—Simple or compound lenses
- G02B3/10—Bifocal lenses; Multifocal lenses
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/28—Systems for automatic generation of focusing signals
- G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
- G02B7/38—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals measured at different points on the optical axis, e.g. focussing on two or more planes and comparing image data
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
- H04N25/704—Pixels specially adapted for focusing, e.g. phase difference pixel sets
- H04N5/3696—
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/28—Systems for automatic generation of focusing signals
- G02B7/34—Systems for automatic generation of focusing signals using different areas in a pupil plane
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
- G03B13/32—Means for focusing
- G03B13/34—Power focusing
- G03B13/36—Autofocus systems
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
- H04N5/2254—

Definitions

the present invention relates to a technology for controlling a focusing state by an imaging device including a multifocal lens.
Patent Literature 1 describes that a subject image is obtained by convolution processing in an imaging device including a multifocal lens.
Patent Literature 2 describes that an imaging signal is corrected with an inverse function based on a point spread function in an imaging device including a multifocal lens.
images with different focal lengths are obtained by using the imaging device including the multifocal lens, and an in-focus image is obtained by restoration processing.
Patent Literature 3 describes that a barcode is read in a short focus region provided to a center portion of a multifocal lens and normal picture taking is performed in a long focus region provided to a peripheral portion of the lens.
the present invention was made based on the circumstances described above, and has an object of providing an imaging method and image processing method, programs therefor, recording medium, and an imaging device capable of obtaining a desired image by taking advantage of the characteristics of a multifocal lens and also capable of extracting and selecting the obtained image according to the purpose or preference.
an imaging method is an imaging method using a multifocal lens having a plurality of regions, the plurality of regions having different focal lengths, and the imaging method includes a focusing state control step of controlling a focusing state of the multifocal lens, and an imaging step of obtaining an image of a subject in the controlled focusing state.
the imaging method includes a focusing state control step of controlling a focusing state of the multifocal lens, and an imaging step of obtaining an image of a subject in the controlled focusing state.
the focusing state of the multifocal lens having a plurality of different focal lengths is controlled to obtain an image of the subject. Therefore, a desired image, such as an accurately-focused image and an image focused at high speed, can be obtained by taking advantage of the characteristics of the multifocal lens.
the focusing state is controlled according to a relation between a depth of field of the multifocal lens and a width of a distance distribution of the subject.
focusing control is performed according to the relation between the depth of the field of the multifocal lens and the width of the distance distribution of the subject.
the distance distribution can be detected in any of various methods. For example, calculation may be performed by using phase difference information, or by comparing contrasts of a plurality of imaging signals obtained at different lens positions. Also, an image obtained in the second aspect is not restricted to a still picture, and a moving picture may be obtained by continuously performing focusing control according to changes in distance distribution of the subject. Furthermore, in the second aspect, the imaging device preferably has an image pickup element including a light-receiving element which selectively receives a light beam passing through any of the plurality of regions.
the focusing state control step when the depth of field of the multifocal lens as a whole is equal to the width of the distance distribution of the subject or more, the focusing state is controlled so that the width of the distance distribution of the subject is within the depth of field as the whole.
a relation between the depth of field as the whole multifocal lens and the distance distribution of the subject varies according to the design of the multifocal lens, the actual picture-taking situation, etc., and either one may be wide.
the imaging method according to the third aspect in consideration of these circumstances, when the depth of field of the multifocal lens as a whole is equal to the width of the distance distribution of the subject or more, focusing control is performed so that the width of the distance distribution of the subject is within the depth of subject to obtain an image. Therefore, an image appropriately focused according to the distance distribution of the subject can be obtained
wow the width of the distance distribution of the subject is covered by the depth of field as the multifocal lens as a whole can be defined according to the picture-taking purpose.
control may be performed so that the center of the depth of the field as a whole and the center of the depth of field of subject match each other or the depth of field as a whole is shifted to a front side or a rear side of the distance distribution of the subject.
control may be performed so that the focusing degree in any of the focal regions with respect to a specific subject such as a main subject is equal to a threshold or more or is maximum.
the imaging method in the imaging method according to a fourth aspect of the present invention in the first or second aspect, in the focusing state control step, when the depth of field of the multifocal lens as a whole is narrower than the width of the distance distribution of the subject, a range not within the depth of field as the whole extends off equally on a front side and a back side of the depth of field as the whole.
the fourth aspect defines an aspect of focusing control when the depth of field of the multifocal lens as a whole is narrower than the width of the distance distribution of the subject.
the focusing state control step when the depth of field of the multifocal lens as a whole is narrower than the width of the distance distribution of the subject, the focusing state is controlled based on a focusing priority ranking among subjects.
the focusing degree of the subject with the second priority ranking or lower can be made maximum. With this, an image appropriately focused according to the distance distribution of the subject can be obtained.
the imaging method according to a sixth aspect of the present invention in the fifth aspect further includes a step of detecting a face of a person, and a step of setting a focusing priority ranking of the person with the detected face higher than a ranking of a subject other than the person, wherein in the focusing state control step, the focusing state is controlled based on the set focusing priority ranking.
the imaging method according to a seventh aspect of the present invention in the fifth aspect further includes a step of causing a user of the imaging device to specify the focusing priority ranking, wherein in the focusing state control step, the focusing state is controlled based on the specified focusing priority ranking.
the focusing priority ranking of a person may be made higher or the focusing priority ranking of a subject other than the person may be made higher.
the focusing state control step the focusing state is controlled so that a main subject is focused in any of the plurality of regions. Note that whether the subject is a main subject can be determined based on whether the subject is a face of a person, based on a ratio occupying a picture-taking region, etc.
the focusing state control step the focusing state is controlled so that the main subject is focused in a region with a shortest focal length among the plurality of regions, and thereby in the imaging step, an image with a region on a front side of the main subject blurred is obtained.
an image in which a subject on a front side (a side near the imaging device) of the main subject is blurred according to the distance from the main subject can be obtained.
the focusing state control step the focusing state is controlled so that the main subject is focused in a region with a longest focal length among the plurality of regions, and thereby in the imaging step, an image with a region on a depth side of the main subject blurred is obtained.
an image in which a subject on a depth side (a side far from the imaging device) of the main subject is blurred according to the distance from the main subject can be obtained.
the focusing state is controlled based on a relation between an aperture value of the multifocal lens and a depth of field of the multifocal lens.
the field of depth varies depending on the aperture of the lens. In general, the depth of field becomes deeper, when the aperture value (F value) increases, and the depth of field becomes shallower when the aperture value decreases. Therefore, as in the eleventh aspect, by performing focusing control based on the relation between the aperture value of the multifocal lens and the depth of filed of the multifocal lens, an image appropriated focused according to the distance distribution of the subject can be obtained.
the imaging method according to a twelfth aspect of the present invention in any of the first to tenth aspects further includes a step of recording the obtained image and information indicating the focusing degree of the subject included in the obtained image.
the user can select and extract a desired image, such as an image with a focusing degree equal to a predetermined value or more, with reference to the focusing degree information after image obtainment.
an image processing method includes a step of extracting an image included in the recorded image based on the information recorded by the twelfth aspect. Also, the image processing method according to a fourteenth aspect of the present invention in the thirteenth aspect further includes a step of extracting an image including a subject having a specified focusing degree based on the recorded information. Furthermore, the image processing method according to a fifteenth aspect of the present invention in the thirteenth or fourteenth aspect further includes a step of generating a new image in which a specified subject has a specified focusing degree by using the extracted image. In the image processing method according to any of the above aspects, the user can obtain a desired focusing degree according to the purpose or preference. Note in the thirteenth to fifteenth aspects that a specific value may be specified as the focusing degree, or only an upper-limit value, only a lower-limit value, or a range formed of an upper-limit value and a lower-limit value may be specified.
an imaging program according to a sixteenth aspect of the present invention causes an imaging device to perform the imaging method according to any of the first to twelfth aspects.
an image processing program according to a seventeenth aspect of the present invention causes an image processing device to perform the image processing method according to any of the thirteenth to fifteenth aspects.
the programs according to the sixteenth and seventeenth aspects may be incorporated in the imaging device such as a digital camera, or may be used as image processing and editing software in a personal computer (PC) or the like.
PC personal computer
computer-readable codes of the imaging programs according to the sixteenth and seventeenth aspects are respectively recorded.
a non-transitory semiconductor storage medium or magneto-optical recording medium can be used, such as a ROM or RAM of a digital camera or PC as well as a CD, DVD, BD, HDD, SSD, or any of various memory cards.
an imaging device includes a multifocal lens having a plurality of regions, the plurality of regions having different focal lengths, focusing state control means which controls a focusing state of the multifocal lens, and imaging means which obtains an image of a subject in the controlled focusing state, wherein the focusing state control means controls the focusing state according to a relation between a depth of field of the multifocal lens and a width of a distance distribution of the subject.
the imaging device by controlling the focusing state according to the relation between the depth of field of the multifocal lens and the width of the distance distribution of the subject, for example, it is flexibly possible to obtain an image accurately focused in any of the plurality of focal regions according to the distance to the subject, to image a subject at a lens position different from a focusing position to obtain an image with a specific subject blurred, or others, thereby obtaining a desired image by taking advantage of the characteristics of the multifocal lens.
“depth of field of the multifocal lens” is assumed to include a depth of field of each focal region of the multifocal lens and a depth of field of the multifocal lens as a whole.
the image obtained in the twentieth aspect is not restricted to a still image, and a moving image may be obtained by continuously performing focusing control according to the changes of the distance distribution of the subject.
the imaging device according to the twentieth aspect preferably has an image pickup element including a light-receiving element which selectively receives a light beam passing through any of the plurality of regions.
the focusing state control step the focusing state is controlled so that a main subject is focused via any of the plurality of regions in response to a picture-taking instruction.
the front image device preferably has an image pickup element including a light-receiving element which selectively receives a light beam passing through any of the plurality of regions.
the “multifocal lens” is not restricted to a single lens but includes a lens configured of a plurality of lenses. “Controlling the multifocal lens” means that the subject is focused by moving at least part of the lens(es) configuring the multifocal lens. Also, in each aspect of the present invention, a “picture-taking instruction” may be inputted by a user pressing a shutter button, or may be generated by an imaging device at each picture-taking time during moving-picture taking. Furthermore, in each aspect of the present invention, whether the subject is a main subject can be determined based on whether the subject is a face of a person, based on a ratio occupying a picture-taking region, etc.
control is performed so that the main subject is focused via a region with a shortest required focusing time among the plurality of regions.
control since control is performed so that the main subject is focused via a region with the shortest required focusing time among the plurality of regions, focusing can be made at higher speed.
control is performed so that the main subject is focused via a main region having a largest area among the plurality of regions.
image quality depends on the area of each focal region.
the image is obtained by performing control so that the main subject is focused via the man region having the largest area among the plurality of regions. Therefore, an image accurately focused with high image quality can be obtained, and the maximum performance of the lens can be used.
control is performed so that the main subject is focused via a region with a shortest required focusing time among the plurality of regions, and the image of the subject is obtained in the imaging step, and subsequently in the focusing state control step, control is performed so that the main subject is focused via a main region having a largest area among the plurality of regions, and the image of the subject is obtained in the imaging step.
control is performed so that focusing is made via a region with the shortest required focusing time among the plurality of regions to obtain an image and, subsequently, control is performed so that focusing is made via a main region having the largest area among the plurality of regions to obtain an image. Therefore, an image focused at high speed without a time lag and an image accurately focused with high image quality but with a slight time lag can be obtained. From these images, the user can select a desired image according to the purpose or preference.
the imaging step after the picture-taking instruction and before the focusing state control step is performed, an image is obtained in all of the plurality of regions with the focusing state at the picture-taking instruction being kept. Since imaging is performed with the focusing state at the time of the picture-taking instruction being kept and a multifocal lens is used, an image focused to some extent can be obtained at high speed in any of the plurality of regions.
the imaging method in response to the picture-taking instruction, at least one of the following is continuously performed to obtain a moving image of the subject: performing control so that the main subject is focused via a region with a shortest required focusing time among the plurality of regions in the focusing state control step and obtaining an image of the subject in the imaging step; and performing control so that the main subject is focused via a main region having a largest area among the plurality of regions in the focusing state control step and obtaining the image of the subject in the imaging step.
At least one of the following image obtainment types is continuously performed to obtain a moving image of the subject: image obtainment with control so that the main subject is focused via a region with the shortest required focusing time among the plurality of regions and image obtainment with control so that the main subject is focused via a main region having the largest area among the plurality of regions. Therefore, at least one of the image focused at high speed and the image accurately focused with high image quality can be continuously obtained. Note that which image is to be obtained may be specified by the user, or both images may be continuously obtained and then selected and edited subsequently. With this, the user can obtain a desired image according to the purpose or preference.
the imaging method according to a twenty-seventh aspect of the present invention in any of the twenty-first to twenty-sixth aspects further includes a step of recording the obtained image and imaging time information indicating a time from the picture-taking instruction to image obtainment in association with each other.
the obtained image and the imaging time information indicating the time from the picture-taking instruction to image obtainment are recorded in association with each other. Therefore, the user can select and extract a desired image, such as an image with a picture-taking time is equal to a predetermined time or less, with reference to the imaging time information after image obtainment.
the imaging method according to a twenty-eighth aspect of the present invention in any of the twenty-first to twenty-seventh aspects further includes a step of recording the obtained image and focusing degree information indicating a focusing degree of the obtained image in association with each other.
the obtained image and the focusing degree information indicating a focusing degree of the obtained image are recorded in association with each other. Therefore, the user can select and extract a desired image, such as an image with a focusing degree is equal to a predetermined value or more, with reference to the focusing degree information after image obtainment.
the imaging method according to a twenty-ninth aspect of the present invention in any of the twenty-first to twenty-eighth aspects further includes a step of recording the obtained image and imaging region information indicating from which area the obtained image is obtained in association with each other.
the obtained image and the imaging region information indicating from which area the obtained image is obtained are recorded in association with each other. Therefore, the user can select and extract an image taken in a specific region (for example, in the main region) according to the purpose or preference.
an image processing method includes a step of extracting an image included in the recorded image based on the information recorded with the imaging method according to any of the twenty-seventh to twenty-ninth aspects.
an image is extracted with reference to the imaging time information, the focusing degree information, and the imaging region information recorded in association with the image. Therefore, the user can obtain a desired image according to the purpose or preference.
an imaging program causes an imaging device to perform the imaging method according to any of the twenty-first to thirtieth aspects.
an imaging program according to a thirty-second aspect of the present invention causes an imaging device to perform the imaging method according to the thirtieth aspect.
the program according to the thirty-second aspect may be used by being incorporated in the imaging device such as a digital camera or may be used in a personal computer (PC) or the like as image processing/editing software.
PC personal computer
computer-readable codes of the imaging programs according to the thirty-first and thirty-second aspects are respectively recorded.
a non-transitory semiconductor storage medium or magneto-optical recording medium can be used, such as a ROM or RAM of a digital camera or PC as well as a CD, DVD, BD, HDD, SSD, or any of various memory cards.
the imaging device includes a multifocal lens having a plurality of regions, the plurality of regions having different focal lengths, focusing state control means which controls a focusing state of the multifocal lens, and imaging means which obtains an image of a subject in the controlled focusing state, wherein the imaging means performs control so that a main subject is focused via any of the plurality of regions in response to a picture-taking instruction.
the imaging device preferably has an image pickup element including a light-receiving element which selectively receives a light beam passing through any of the plurality of regions.
the focusing state control means controls the focusing state with a phase-difference scheme.
a so-called contrast scheme and a so-called phase difference scheme are present. While the focusing state is detected by driving the lens in the contrast scheme, the focusing state is detected based on a phase difference of a subject image in the phase difference scheme. Therefore, focusing control at higher speed than the contrast scheme can be achieved, and accurate focusing at high speed, which is an effect of the present invention, can be further achieved.
the “multifocal lens” is not restricted to one lens but includes a lens configured of a plurality of lenses, and “control of the multifocal lens” means that at least part of the lens (lenses) configuring the multifocal lens is moved to focus on the subject.
a desired image can be obtained by taking advantage of the characteristics of the multifocal lens, and the obtained image call be extracted and selected according to the purpose or preference.
FIG. 1 is a block diagram depicting the structure of an imaging device 10 .
FIG. 2 is a schematic diagram depicting a taking lens 12 .
FIG. 3 is a schematic diagram depicting another example of a multifocal lens.
FIG. 4 is a schematic diagram depicting a relation between a subject distance and a focusing degree in the taking lens 12 .
FIG. 5 is a schematic view depicting the structure of an image pickup element 16 .
FIG. 6 is a schematic diagram depicting the arrangement of light-receiving elements in the image pickup element 16 .
FIG. 7 is a flowchart of an imaging method according to a first embodiment of the present invention.
FIG. 8 is a conceptual diagram depicting the imaging method according to the first embodiment of the present invention.
FIG. 9 is a flowchart of an imaging method according to a second embodiment of the present invention.
FIG. 10 is a flowchart of an imaging method according to a third embodiment of the present invention.
FIG. 11 is a diagram depicting an example of priority setting among subjects in the imaging method according to the third embodiment of the present invention.
FIG. 12 is a flowchart of an imaging method according to a fourth embodiment of the present invention.
FIG. 13 is a flowchart of an image processing method according to a fifth embodiment of the present invention.
FIG. 14 is a flowchart of an image processing method according to a sixth embodiment of the present invention.
FIG. 15 is a schematic diagram depicting a relation between a subject distance and a focusing degree in the taking lens 12 .
FIG. 16 is a flowchart of an imaging method according to a seventh embodiment of the present invention.
FIG. 17 is another flowchart of the imaging method according to the seventh embodiment of the present invention.
FIG. 18 is a flowchart of an imaging method according to an eighth embodiment of the present invention.
FIG. 19 is another flowchart of the imaging method according to the eighth embodiment of the present invention.
FIG. 20 is a flowchart of an imaging method according to a ninth embodiment of the present invention.
FIG. 21 is another flowchart of the imaging method according to the ninth embodiment of the present invention.
FIG. 22 is a conceptual diagram depicting the imaging method according to the ninth embodiment of the present invention.
FIG. 23 is a schematic diagram depicting an example of specifying a main subject in the imaging device 10 .
FIG. 24 is a conceptual diagram depicting an image extracting process in the present invention.
FIG. 1 is a block diagram depicting an embodiment of an imaging device 10 (an imaging device or an image processing device) according to a first embodiment of the present invention.
the operation of the imaging device 10 as a whole is controlled by a central processing unit (CPU) 40 (focusing state control means and imaging means), and a program (including a program for use in processing according to the present invention such as control of a focusing state, imaging, and image extraction and combining) and parameters required for the operation of the CPU 40 are stored in an EEPROM (electronically Erasable and Programmable Read Only Memory) 46 (recording medimn).
EEPROM electrotronically Erasable and Programmable Read Only Memory
a program (imaging program and image processing program) according to the present invention can be recorded on the EEPROM 46 of the imaging device 10 as well as a non-transitory semiconductor storage medium or magneto-optical recording medium such as a CD. DVD, BD, HDD, SSD, or any of various memory cards and can be used on an image processing device such as a digital camera or personal computer (PC).
a program imaging program and image processing program
the imaging device 10 is provided with an operating unit 38 such as a shutter button, a mode dial, a replay button, a MENU/OK key, a cross key, and a BACK key.
an operating unit 38 such as a shutter button, a mode dial, a replay button, a MENU/OK key, a cross key, and a BACK key.
a signal from this operating unit 38 is inputted to the CPU 40 and, based on the input signal, the CPU 40 controls each circuit of the imaging device 10 , as will be described further below.
the shutter button is an operation button for inputting an instruction for starting picture taking, and is configured of switches of a two-step stroke type having an S1 switch that is turned ON at the time of a half push and an S2 switch that is turned ON at the time of a full push.
the mode dial is means that selects any of a still-picture/moving-picture taking mode, a manual/auto picture-taking mode, and a picture-taking scene, etc.
the replay button is a button for switching to a replay mode for causing a still picture or moving picture of images taken and recorded to be displayed on a liquid-crystal monitor (LCD) 30 .
the MENU/OK key is an operation key having both of a function for making an instruction for causing a menu to be displayed on a screen of the liquid-crystal monitor 30 and a function for making an instruction for confirming and executing selected details and the like.
the cross key is an operating unit for inputting an instruction of any of four directions, that is, upward, downward, leftward, and rightward, and functions as cursor movement operation means, a zoom switch, a frame-advance button at the time of replay mode, or the like.
the BACK key is used to delete a desired target such as a selected item and cancel instruction details, return to the immediately preceding operation state, or the like.
buttons and keys can also be used for an operation required at the time of image extraction and combining processes or setting priorities among subjects, which will be described further below.
an image of image light representing a subject is formed via a taking lens 12 and an aperture 14 onto a light-receiving surface of a solid-state image pickup element (hereinafter referred to as a “CCD”) 16 (imaging means).
the taking lens 12 is driven by a lens drive unit 36 (focusing state control means), which is controlled by the CPU 40 , and focusing control or the like is performed, which will be described further below.
the lens drive unit 36 changes the focal position by moving the taking lens 12 in an optical axis direction by following an instruction from the CPU 40 .
the taking lens 12 is a multifocal lens (trifocal lens) having a region (hereinafter referred to as a near focal region) 12 a with a short focal length for near distance picture taking, a region (hereinafter referred to as an intermediate focal region) 12 b with a focal length longer than that of the near focal region and capable of taking pictures of humans and the like, and a region (hereinafter referred to as a far focal region) 12 c with a focal length further longer than that of the intermediate focal region and capable of taking pictures of landscapes and the like.
a near focal region 12 a with a short focal length for near distance picture taking
an intermediate focal region 12 a region with a focal length longer than that of the near focal region and capable of taking pictures of humans and the like
a region (hereinafter referred to as a far focal region) 12 c with a focal length further longer than that of the intermediate focal region and capable of taking pictures of landscapes and the like.
regions each in a half-moon shape when viewed from front are provided to upper and lower portions in FIG. 2 , and a band-shaped region including a lens center O is provided therebetween. These regions are, the near focal region 12 a , the intermediate focal region 12 b , and the far focal region 12 c , in sequence from an upper portion.
the intermediate focal region 12 b with the largest area is a main region. In these regions, specific values of focal lengths may be set according to the purpose of picture taking and the like.
the taking lens may be formed of a circular region 12 a ′ (intermediate focal region) including a lens center O′ and peripheral annular-shaped regions 12 b ′ and 12 c ′ thereof (one is a near focal region and the other is a far focal region).
an area ratio among the near focal region 12 a , the intermediate focal region 12 b , and the far focal region 12 c can be, for example, 30:50:20, but may be set at a ratio different from the above according to the characteristics of an optical system and the picture-taking purpose.
the number of focal regions is not restricted to three, and there may be two focal points or four or more focal points.
FIG. 4 is a a schematic diagram depicting a relation between a subject distance and a focusing degree in the taking lens 12 .
focusing degree curves Ca, Cb, and Cc are curves indicating focusing degrees at a distance D in a near focal region 12 a , an intermediate focal region 12 b , and a far focal region 12 c , respectively.
Distances to points Pa, Pb, and Pc where the respective curves are at their peaks represents a focal distance of the taking lens 12 in each region, and the height of each curve corresponds to a focusing degree and the spread of each curve corresponds to a depth of field. Therefore, if the taking lens 12 is controlled so that the subject distance and any of Pa.
the subject is accurately focused in the focal region. If the subject is present in the spread of any of of the curves Ca, Cb, and Cc (represented by Wa, Wb, and Wc, respectively), the subject is focused to some extent in the region according to the distance from each of Pa, Pb, and Pc. Also, if the subject is in the depth of field of the taking lens 12 as a whole, the subject is focused to some extent in any of the regions.
An area of a region surrounded by each of these curves corresponds to an area of each focal region.
the shape of each curve, the degree of overlap with another curve, and a depth of field DOF of the taking lens 12 as a whole defined by these can be set according the characteristics of the optical system and the picture-taking purpose.
These curves Ca, Cb, and Cc correspond to MTF (Modulation Transfer Function) curves of the respective regions.
the CCD 16 has a near-image light-receiving cell 16 a for receiving a light beam passing through the near focal region 12 a of the taking lens 12 , an intermediate-image light-receiving cell 16 b for receiving a light beam passing through the intermediate focal region 12 b thereof, and a far-image light-receiving cell 16 c for receiving a light beam passing through the far focal region 12 c .
the light-receiving cells 16 a , 16 b , and 16 c selectively receive light beams passing through the near focal region 12 a , the intermediate focal region 12 b , and the far focal region 12 c , via a microlens ML and light-shielding films 18 a , 18 b , and 18 c provided on the front surface of a light-receiving unit 17 , respectively.
the light-shielding films 18 a , 18 b , and 18 c have different shapes. Note that in place of providing light-shielding films on the front surface of the light-receiving unit 17 , light-shielding member may be provided on the front surface of the microlens ML.
the numbers of these light-receiving cells are provided at a ratio according to the areas of corresponding lens regions.
the number of elements of the intermediate-image light-receiving cells 16 b corresponding to the intermediate focal region 12 b is the largest.
an image obtained from the intermediate-image light-receiving cells 16 b is a main image, and by increasing the number of intermediate-image light-receiving cells 16 b according to the area ratio of the taking lens, a main image with high image quality can be obtained.
the numbers of light-receiving cells may be substantially substantially equal to the area ratio of the focal regions corresponding to the light-receiving cells.
the light-receiving cells are preferably arranged so that the image quality is not degraded in a specific region or direction in the obtained image.
a dropout of pixel data because cells corresponding to a plurality of focal regions are mixed is preferably compensated for by interpolation or the like.
the CPU 40 controls the aperture 14 via the aperture drive unit 34 and also controls a charge accumulation time (shutter speed) at a CCD 16 via a CCD control unit 32 and reading of an image signal from the CCD 16 .
the signal charge accumulated in the CCD 16 is read as a voltage signal according to the signal charge based on a read signal supplied from the CCD control unit 32 , and is supplied to an analog signal processing unit 20 .
An analog signal processing unit 20 samples and holds R, G, and B signals for each pixel by performing a correlated double sampling process on a voltage signal outputted from the CCD 16 , and amplifies and then supplies the resultant signals to an A/D converter 21 .
the A/D converter 21 converts the sequentially inputted analog R, G, and B signals to digital R. G, and B signals for output to an image input controller 22 .
a digital signal processing unit 24 performs predetermined signal processing on the digital image signals inputted via the image input controller 22 , such as offset processing, gain control processing including white balance correction and sensitivity correction, gamma correction processing, and YC processing.
Image data processed in the digital signal processing unit 24 is outputted to a VRAM 50 .
the VRAM 50 includes an A region and a B region each for storing image data representing one frame. Image data representing one frame is alternately rewritten between the A region and the B region. From the region other than the region where image data is being rewritten, the written image data is read.
the image data read from the VRAM 50 is encoded in a video encoder 28 and is outputted to the liquid-crystal monitor 30 , thereby causing a subject image to be displayed on the liquid-crystal monitor 30 .
a touch panel is adopted, displaying the obtained image. Also, as will be described further below, with a user's operation via a screen, operations are possible, such as specifying a main subject and other subject, setting focusing priorities among subjects, specifying a blurred region, and the like.
the CPU 40 starts an AF operation (focusing state control step), controlling a focusing state of the taking lens 12 via the lens drive unit 36 . Also, image data outputted from the A/D converter 21 at the time of a half press of the shutter button is captured into an AE detecting unit 44 .
the CPU 40 calculates a brightness (picture-taking Ev value) of the subject from an integrated value of G signals inputted from the AE detecting unit 44 , determines an aperture value and an electronic shutter (shutter speed) of the CCD 16 based on this picture-taking Ev value and, based on the result, controls a charge accumulation time at the aperture 14 and the CCD 16 .
An AF processing unit 42 (focusing state control means) is a portion where a phase-difference AF process is performed, controlling a focus lens in the taking lens 12 so that a defocus amount found from a phase difference in image data between a main picture element and a sub-picture element in a predetermined focus region of image data.
image data outputted from the A/D converter 21 in response to the press is inputted from the image input controller 22 to a memory (SDRAM) 48 and is temporarily stored therein.
SDRAM memory
an image file is generated after signal processing at the digital signal processing unit 24 such as YC processing, compression processing to a JPEG (joint photographic experts group) format at a compression/decompression processing unit 26 , etc.
the image file is read by a media controller 52 and recorded on a memory card 54 .
the image recorded on the memory card 54 can be replayed and displayed on the liquid-crystal monitor 30 by operating the replay button of the operating unit 38 .
FIG. 7 is a flowchart of the imaging process according to a first embodiment of the present invention, and execution is controlled by the CPU 40 based on a program (imaging program) stored in the EEPROM 46 (recording medium).
an image is obtained by controlling so that a main subject is focused in the intermediate focal region 12 b as a main region.
Imagery of the imaging process is depicted in (a) to (f) of FIG. 8 (imaging method and imaging program).
the main subject is a child Q1, and a dog Q2 and a wood Q3 are other subjects. Note in (a) of FIG. 8 that the focal regions 12 a and 12 c and the focal curves Ca and Cc corresponding thereto are not depicted.
the lens is first moved to an initial position (S 100 ). Any initial position can be set. For example, a lens position where the subject at a distance of 2 m is focused in the intermediate focal region 12 b as a main region can be set as an initial position.
picture taking is ready.
S 102 When the user presses the shutter button of the operating unit 38 to input a picture-taking instruction (S 102 ), a distance distribution of a subject is detected (S 104 ) based on an aperture value of the aperture 14 . Based on the detected distance distribution, the child Q1 as a main subject is focused in the main region (S 106 ; focusing state control step).
the child Q1 is in a width Wb of the intermediate focal region 12 b (also in the range of the depth of field DOF of the taking lens 12 as a whole), and the position of the child Q1 matches with the position of the peak Pb of the curve Cb.
a determination as to whether the subject is a main subject at S 106 can be made by determining whether the subject is a human face, based on a ratio occupying a picture-taking region, or by the user specifying via the liquid-crystal monitor 30 .
the distance distribution can be detected by any of various methods. For example, as described in Japanese Patent Application Laid-Open No. 2011-124712, calculation may be performed by using phase difference information or by comparing contrasts of a plurality of imaging signals obtained at different lens positions. The same goes for second to fourth embodiments, which will be described further below.
an image is obtained (S 108 ; imaging step), and the obtained image is recorded in association with focusing information (such as the width of the distance distribution of the subject, the depth of field of the taking lens 12 , and the focusing degree of the subject) in the memory card 54 (S 110 ).
focusing information such as the width of the distance distribution of the subject, the depth of field of the taking lens 12 , and the focusing degree of the subject.
the processes at S 104 to S 110 are continuously performed to obtain and record images according to the movement of the main subject (that is, changes of the distance distribution).
FIG. 9 (imaging method and imaging program) is a flowchart of the imaging process according to a second embodiment of the present invention, and execution is controlled by the CPU 40 based on a program stored in the EEPROM 46 .
focusing control is performed according to whether the depth of field of the taking lens 12 is equal to the width of the distance distribution of the subject or more.
the CPU 40 moves the taking lens 12 to an initial position (S 200 ).
S 202 When a picture-taking instruction is inputted (S 202 ), the depth of field DOF of taking lens 12 as a whole is calculated based on the aperture value of the aperture 14 (S 204 ), and a distance distribution of the subject is next detected (S 206 ).
S 210 focusing state control step
focusing control is performed so that the width SD is within the depth of field DOF, as depicted in (b) of FIG. 8 .
How the width SD of the distance distribution of the subject is covered by the depth of field DOF as the multifocal lens as a whole at the time of focusing control at S 210 can be defined according to the degree of difference in breadth between DOF and SD or the picture-taking purpose. For example, with the width SD of the distance distribution set within the depth of field DOF, control may be performed so that the center of DOF and the center of SD match each other or so that the focusing degree in any of the focal regions with respect to a specific subject such as a main subject is maximum.
imaging is performed (S 212 ; imaging step), and the obtained image and focusing degree information and focusing information (such as the width of the distance distribution of the subject, the depth of field of the taking lens 12 , and the focusing degree of the subject) regarding the respective subjects Q1 to Q3 are recorded in association with each other in the memory card 54 (S 214 ).
the procedure proceeds to S 216 (focusing state control step), where an image is obtained with focusing control so that SD extends off the depth of field forward and backward equally (S 218 ; imaging step), and the obtained image and the focusing degree information of the respective subjects Q1 to Q3 are recorded in association with each other in the memory card 54 (S 220 ).
focusing control is performed according to whether the depth of field DOF of the taking lens 12 as a whole is equal to the width SD of the distance distribution of the subject or more to obtain an image. Therefore, an appropriately focused image can be obtained according to the distance distribution of the subject.
FIG. 10 (imaging method and imaging program) is a flowchart of the imaging process according to a third embodiment of the present invention, and execution is controlled by the CPU 40 based on a program stored in the EEPROM 46 .
focusing control is performed according to whether the depth of field of the taking lens 12 is equal to the width of the distance distribution of the subject or more.
the focusing control is performed based on a focusing priority order among the subjects to obtain an image.
the procedure first proceeds to S 316 , where priorities among the subjects are specified. Priorities may be specified by performing face detection on an imaging device 10 side and setting the priority order of a person with the detected face higher than that of the subject other than that person or by accepting the specification of the user via the liquid-crystal monitor 30 having a touch panel function.
Examples of the priority setting described above are depicted in (a) to (c) of FIG. 11 .
the child Q1, the dog Q2, and the woods Q3 are present as subjects as in (a) of FIG. 11
an example in which face detection is performed on the imaging device 10 side for priority ranking is depicted in (b) of FIG. 11 .
the child Q1 is a main subject
the dog Q2 is the second in priority ranking
the wood Q3 is the third in priority ranking.
These subjects Q1 to Q3 are provided with a frame FL1 (a thick solid line), a frame FL2 (a thin solid lien), and a frame FL3 (a dotted line) according to the priority ranking, and the user can confirm the priority ranking among the subjects.
the size, position, ON/OFF of display, and others of the frames FL1 to FL 3 may be changed by the user operating on the liquid-crystal monitor 30 .
FIG. 11 An example in which the user sets priority ranking among subjects is depicted in (c) of FIG. 11 .
the priority ranking is set according to the order in which the user touches (here, the order is assumed to be the child Q1, the dog Q2, and then the woods Q3), and the frames FL1 to FL3 are displayed as with (b) of FIG. 11 .
the procedure proceeds to S 318 (focusing state control step), where focusing control is performed so that the subject with higher priority ranking is within the depth of field DOF of the taking lens 12 .
focusing control any of various schemes can be adopted to determine how control is further performed, with the subject with higher priority ranking being within the depth of field DOF of the taking lens 12 .
the subject Q1 with the first priority ranking can be accurately focused in any of the focal regions of the taking lens 12 or, as depicted in (d) of FIG. 8 , the subject Q2 with the second priority ranking can be within the depth of field DOF while the focusing degree of the subject Q1 with the first priority ranking is increased as high as possible.
the subjects as many as possible may be within the depth of field DOF.
an image of the subject is obtained (S 320 ; imaging step), and the obtained image and the focusing information the focusing information (such as the width of the distance distribution of the subject, the depth of field of the taking lens 12 , and the focusing degree of the subject) are recorded in association with each other in the memory card 54 (S 322 ).
an image appropriately focused according to the distance distribution of the subject can be obtained.
FIG. 12 (imaging method and imaging program) is a flowchart of the imaging process according to a fourth embodiment of the present invention, and execution is controlled by the CPU 40 based on a program stored in the EEPROM 46 .
an image with a region on a front side or a deep side of the main subject blurred is obtained.
processes at S 400 to S 406 are similar to those at S 200 to S 206 in the flowchart of FIG. 9 and those at S 300 to S 306 in the flowchart of FIG. 10 .
the depth of field DOF of the taking lens 12 as a whole is calculated at S 404 based on the aperture value of the aperture 14 .
the CPU 40 performs control so that the child Q1 as the main subject is focused in the far focal region 12 c of the taking lens 12 as depicted in (f) of FIG. 8 (S 416 ; focusing state control step), and imaging is performed (S 418 : imaging step).
the image obtained by imaging is recorded in association with the focusing information in the memory card 54 (S 420 ).
a subject in front of the child Q1 (for example, the dog Q2) is focused within the range of the depth of field DOF of the taking lens 12 , but a subject on a side deeper than the child Q1 (for example, the wood Q3) is blurred according to the distance from the child Q1.
a subject on a side deeper than the child Q1 (for example, the wood Q3) is blurred according to the distance from the child Q1.
the intermediate focal region 12 b in the taking lens 12 and its corresponding focusing degree curve Cb are not depicted.
the user can obtain a desired image, with the front side or the deep side of the main subject blurred according to the picture-taking purpose.
the user may be prompted to specify a blurring target region via the liquid-crystal monitor 30 for focusing control so that that region is blurred and then imaging.
FIG. 13 (image processing method and image processing program) is a flowchart of image processing according to a fifth embodiment of the present invention, and execution is controlled by the CPU 40 based on a program (image processing program) stored in the EEPROM (recording medium) 46 .
a program image processing program stored in the EEPROM (recording medium) 46 .
FIG. 13 the case is described in which an image is extracted from among images obtained with the scheme according to any of the first to fourth embodiments described above and, from the extracted image, a new image is generated by combining so that all subjects have a focusing degree equal to a threshold or more.
an image including a subject as a target (here, the case is described in which the child Q1, the dog Q2, and the woods Q3 are present as in (a) to (c) of FIG. 11 ) is detected in images recorded in the memory card 54 .
the subject as a target for example, the child Q1
a focusing degree of that target subject is detected, and it is determined whether the focusing degree is equal to a threshold or more (S 502 ).
the focusing degree of the target subject can be detected with reference to this focusing information.
a value set on the imaging device 10 side may be used as the threshold, or a value specified by the user via the operating unit 38 may be used. Also, a threshold may be set for each subject.
the procedure proceeds to 504 , where it is determined whether the detected focusing degree is maximum.
updating is performed with the process target image being taken as a combining source image regarding the target subject, and the procedure then proceeds to S 508 , where it is determined whether all images have been processed.
NO is determined at S 502 or S 504
the procedure proceeds to S 508 without updating the combining source image.
the procedure proceeds to S 510 .
NO is determined, the processes at S 502 to S 506 are repeated until the process on all images ends. With this, an image with a focusing degree of the target subject being equal to the threshold or more and maximum is extracted from the images recorded in the memory card 54 .
an image with all subjects having a focusing degree equal to the threshold or more can be obtained by combining.
image extraction and combining may be performed by displaying an image before the process and an image after the process on the liquid-crystal monitor 30 .
FIG. 14 (image processing method and image processing program) is a flowchart of image processing according to a sixth embodiment of the present invention, and execution is controlled by the CPU 40 based on a program (image processing program) stored in the EEPROM 46 .
a program image processing program stored in the EEPROM 46 .
FIG. 14 the case is described in which an image is extracted from images obtained with the scheme according to any of the first to fourth embodiments described above and a new image is obtained by combining so that the focusing degree of a specific subject (subject of interest) is equal to a threshold or more and the focusing degrees of each of other subjects are each equal to the threshold or less (for example, the case in which the main subject has a high focusing degree, and the other subjects each have a low focusing degree).
an image including a subject as a target (here, the case is described in which the child Q1, the dog Q2, and the woods Q3 are present as in (a) to (c) of FIG. 11 ) is detected in images recorded in the memory card 54 .
a subject as a target it is then determined whether the detected subject is a subject of interest (S 602 ).
the child Q1 as a main subject can be specified by the user as a subject of interest, but another subject may be specified as a subject of interest.
the procedure proceeds to S 604 .
the procedure proceeds to S 614 .
the focusing degree of the subject of interest is detected to determine whether the focusing degree is equal to a threshold or more.
a threshold may be set for each subject.
the procedure proceeds to 606 to determine whether the detected focusing degree is maximum.
the process target image is updated as a combining source image (S 608 ), and the procedure then proceeds to S 610 to determine whether all images have been processed.
NO is determined at S 604 and S 606
the procedure proceeds to S 610 without updating the combining source image.
the procedure proceeds to S 612 .
the processes at S 600 to S 608 are repeated until the process on all images ends. With this, an image with the focusing degree of the subject of interest is equal to the threshold or more and maximum is extracted from among the images recorded in the memory card 54 .
the procedure proceeds to S 614 , it is determined whether the focusing degree is equal to the threshold or less.
the procedure proceeds to S 616 to determine whether the focusing degree is minimum.
the procedure proceeds to S 618 , where the process target image is updated as a combining source image regarding a subject other than the subject of interest, and the procedure then proceeds to S 610 . Note that when NO is determined at S 614 and S 616 , the procedure proceeds to S 610 without updating the combining source image.
image combining is performed at S 612 .
This image combing is a process of extracting, for each target subject, a portion of the target subject from the extracted image and combining these portions as one image.
image extraction and combining may be performed by displaying an image before the process and an image after the process on the liquid-crystal monitor 30 .
a reference value of extraction (a threshold of the focusing degree) may be changed at each time. For example, instead of extracting an image with a focusing degree equal to the fixed threshold or more at all times, an image with a low focusing degree may be extracted at first, and an image with a high focusing degree may be gradually extracted as time elapses.
the subject initially blurred gradually becomes clear, and natural moving images can be obtained when abrupt focusing may provide uncomfortable feelings.
images may be extracted so that the focused subject may be gradually blurred.
a time-varying reference value may be set by the imaging device, or a value set by the user via the liquid-crystal monitor 30 or the operating unit 38 may be used.
the imaging device 10 is a device performing an imaging process and image processing.
the imaging process and image processing according to the present invention are not restricted to be performed by the imaging device, and can be performed at a camera-equipped portable telephone, a personal computer (PC), a portable game machine, or the like.
the structure of the imaging device 10 (imaging device and image processing device) according to the seventh embodiment is similar to that of the imaging device 10 according to the first to sixth embodiments described above (refer to FIG. 1 to 3 , 5 , 6 , etc.), and therefore detailed description is omitted and differences from these embodiments are now described.
FIG. 15 is a schematic diagram depicting a relation between a subject distance and a focusing degree in the taking lens 12 (multifocal lens).
focusing degree curves Ca, Cb, and Cc are curves indicating focusing degrees at a distance D in a near focal region 12 a , an intermediate focal region 12 b , and a far focal region 12 c , respectively.
Distances to points Pa, Pb, and Pc where the respective curves are at their peaks represents a focal distance of the taking lens 12 in each region, and the height of each curve corresponds to a focusing degree and the spread of each curve corresponds to a depth of field.
the taking lens 12 is controlled so that the subject distance and Pa, Pb, and Pc match each other, the subject is focused via each focal region.
the subject is present in the spread of each of the curves Ca, Cb, and Cc, the subject is focused in each of the regions according to the distance from each of Pa, Pb, and Pc.
An area of a region surrounded by each of these curves corresponds to an area of each focal region.
the shape of each curve and the degree of overlap with another curve can be set according the characteristics of the optical system and the picture-taking purpose.
the focusing degree curves Ca, Cb, and Cc are set so as to cover from a near distance equal to 1 m or less to a far distance (infinity). Note that these curves Ca, Cb, and Cc correspond to MTF (Modulation Transfer Function) curves of the respective regions.
MTF Modulation Transfer Function
FIG. 16 is a flowchart of an imaging process when still-picture taking is performed, and execution is controlled by the CPU 40 based on a program (imaging program) stored in the EEPROM 46 (recording medium).
the CPU 40 moves the taking lens 12 to an initial position (S 700 ).
S 702 images are obtained and recorded in all regions with the lens position at the time of the picture-taking instruction being kept (S 704 ).
focusing information which will be described further below, may be recorded in association with each other.
the CPU 40 detects a focusing state of the taking lens 12 , and times required for focusing at the respective focal regions 12 a , 12 b , and 12 c are calculated (S 706 ).
the CPU 40 detects a focusing state of the taking lens 12 , and times required for focusing at the respective focal regions 12 a , 12 b , and 12 c are calculated (S 706 ).
focusing control on the main region of the main subject S 709 : focusing state control step
S 710 imaging step
the obtained image and the focusing information are recorded in association with each other in the memory card 54 (S 712 ).
the procedure proceeds to S 714 , where the taking lens 12 is controlled so as to be focused via a region with the shortest required focusing time (S 714 ; focusing state control step) to obtain an image (S 716 ; imaging step), and the obtained image and the focusing information are recorded in association with each other in the memory card 54 (S 718 ).
processes at S 706 to S 716 as depicted in FIG. 17 are repeated until the end of moving-picture taking (NO at S 720 ).
Other processes are similar to those of the flowchart of FIG. 16 .
the time required for focusing can be calculated from a lens position at the time of a picture-taking instruction, an in-focus lens position, and a lens moving speed.
the main subject is focused via a region with the shortest required focusing time according to a picture-taking instruction to obtain an image. Therefore, the focused image at high speed (without a time lag) can be obtained. Also, since the main subject is focused via any of the focal region, the accurately focused image can be obtained.
FIG. 18 (imaging method and imaging program) is a flowchart of an imaging process when still-picture taking is performed, and execution is controlled by the CPU 40 based on a program (imaging program) stored in the EEPROM 46 (recording medium).
the CPU 40 moves the taking lens 12 to an initial position (S 800 ).
S 802 images are obtained and recorded in all regions with the lens position at the time of the picture-taking instruction being kept (S 804 ; imaging step).
S 804 imaging step
focusing information which will be described further below, may be recorded in association with each other.
the CPU 40 detects a focusing state of the taking lens 12 (S 806 ), and the main subject is focused via a main region (S 807 ; focusing state control step) to obtain an image (S 808 ; imaging step), and the obtained image and the focusing information are recorded in association with each other in the memory card 54 (S 810 ).
processes at S 806 to S 810 as depicted in a flowchart of FIG. 19 are repeated until the end of moving-picture taking (NO at S 812 ).
Other processes are similar to those of the flowchart of FIG. 18 .
the main subject is focused via the main region according to a picture-taking instruction to obtain an image. Therefore, the accurately focused image with high image quality can be obtained.
FIG. 20 (imaging method and imaging program) is a flowchart of an imaging process when still-picture taking is performed, and execution is controlled by the CPU 40 based on a program (imaging program) stored in the EEPROM 46 .
imaging is first performed in a focal region with the shortest required focusing time.
the focal region with the shortest required focusing time is not a main region, imaging in the main region is further performed.
the CPU 40 moves the taking lens 12 to an initial position (S 900 ).
This state is schematically depicted in (a) of FIG. 22 .
Any initial position can be set.
a lens position where the subject at a distance of 2 m is focused in the intermediate focal region 12 b as a main region can be set as an initial position.
picture taking is ready.
a picture-taking instruction is inputted (S 902 )
images are obtained and recorded in all regions (that is, in all of the near focal region 12 a , the intermediate focal region 12 b , and the far focal region 12 c ) with the lens position at the time of the picture-taking instruction being kept (S 904 ; imaging step).
each focal region of the taking lens 12 covers from a near distance to a far distance. Therefore, as depicted in (b) of FIG. 22 , when a main subject Q is located between the peak Pc of the far focal region and the peak Pb of the intermediate focal region and is not completely focused in any of the focal regions, the main subject Q is in focus to some extent in any of the focal regions even an image is obtained with the lens position at the time of the picture-taking instruction being kept (that is, even if an image is obtained without performing focusing control). Therefore, an image with a predetermined image quality can be obtained at high speed. Note that when the image obtained at S 304 is recorded, the focusing information, which will be described further below, may be recorded in association with each other.
the CPU 40 detects the focusing state of the taking lens 12 , and calculates a time required for focusing at each focusing region (S 906 ).
the procedure proceeds to S 909 , where focusing control of the main region is performed to obtain an image (S 910 ; imaging step), and the obtained image and the focusing information are recorded in association with each other in the memory card 54 (S 912 ).
the taking lens 12 is controlled so that focusing is achieved via a region with the shortest required focusing time (S 914 ; focusing state control step) to obtain an image (S 916 ; imaging step), and the obtained image and the focusing information are recorded in association with each other in the memory card 54 (S 918 ).
the main subject Q is located at a distance nearest to the peak Pc, and the focusing time in the far focal region 12 c is the shortest. Therefore, as depicted in (c) of FIG.
focusing is achieved via the far focal region 12 c to obtain an image (S 914 and S 916 ), and then, as depicted in (d) of FIG. 22 , focusing is achieved via the intermediate focal region 12 b as a main region to obtain an image (S 910 and S 912 ).
the main region is a region with the shortest required focusing time, an image is obtained in the main region and recorded (S 910 and S 912 ), and the process then may end without performing image obtainment in other regions.
the “focusing information” means imaging time information indicating a time from a picture-taking instruction to image obtainment, focusing information indicating a focusing degree of the obtained image, and imaging region information indicating the region of the taking lens 12 from which the image has been obtained. The same goes for seventh and eighth embodiments.
images in all of the focal regions are obtained with the lens position being kept at the initial position according to a picture-taking instruction, next an image in a region with the shortest required focusing time is obtained and, furthermore, when the main region is not a region with the shortest required focusing time, an image is also obtained in the main region.
an image with high image quality accurately focused with the focused image at high speed can be obtained.
an image focused to the maximum performance of the lens can be obtained.
a human when a human face is detected, that human may be recognized as a main subject, or a subject having the largest area may be recognized as a main subject. Also, a subject corresponding to a place touched by the user on the liquid-crystal monitor 30 having a touch panel function may be recognized as a main subject.
This main-subject specifying process is depicted in (a) to (c) of FIG. 23 (imaging method and imaging program). As depicted in (a) of FIG. 23 , an example is depicted in which the dog Q2 is present at front (near distance), the child Q1 is present at an intermediate distance, and the woods Q3 are present at a far distance.
FIG. 23 An example when the imaging device 10 recognizes the child Q1 as a main subject by face recognition is depicted in (b) of FIG. 23 , and the frame FL indicating a main subject is displayed so as to surround the face of the child Q1.
the frame FL is displayed with a point touched by the user being taken as a center. Note that the size of this frame FL may be changed and the position of the frame FL may be moved by a user's operation via the liquid-crystal monitor 30 .
FIG. 21 (imaging method and imaging program) is a flowchart of a moving-picture taking process according to the ninth embodiment. Steps where processes similar to those of the flowchart of FIG. 20 are provided with the same reference numeral, and detailed description is omitted herein.
the CPU 40 controls the taking lens 12 so that the main subject is focused in the near focal region 12 a as depicted in (e) of FIG. 22 (S 914 ) to obtain and record an image (S 916 and S 918 ). Furthermore, as depicted in (f) of FIG. 22 , the taking lens 12 is controlled so that focusing is achieved in the intermediate focal region 12 b as the main region (S 909 ) to obtain an record an image (S 910 and S 912 ).
an image is obtained in the region with the shortest required focusing time according to a picture-taking instruction in the case of moving-picture taking. Furthermore, when the main region is not a region with the shortest required focusing time, an image is obtained also in the main region. With this, an image with high image quality accurately focused with the focused image at high speed can be obtained. Also, by imaging by focusing via any of the focal regions of the taking lens 12 as a multifocal lens, an image focused to the maximum performance of the lens can be obtained. Note in the flowchart of FIG. 21 that while picture-taking in all regions after the picture-taking instruction (S 904 ) is omitted, this picture-taking may be performed as required.
the AF processing unit 42 performs a phase-difference AF process.
the AF processing unit 42 may perform a contrast AF process.
the AF processing unit 42 integrates high-frequency components of image data in a predetermined focus region to calculate an AF evaluation value indicating a focusing state, and controls the focusing state of the taking lens 12 so that this AF evaluation value is maximum.
A, B, C provided under the respective times t0 to tf represent images focused and obtained in the near focal region 12 a , the intermediate focal region 12 b and the far focal region 12 c , respectively.
Note in the third embodiment of the imaging process that when an image is obtained in the region with the shortest required focusing time and furthermore the main region is not the region with the shortest required focusing time, an image is obtained also in the main region, and therefore two images may be obtained at a time interval equal to an interval At of each frame of moving-picture taking or less. In the following description, such two images are collectively described as “images A and B taken at time t 1 ”.
the image and the focusing image are recorded in association with each other. Therefore, with reference to the focusing information recorded for each image, a desired image can be selected and extracted.
an image focused and imaged can be extracted at high speed with a short time lag.
An example of extracting an image with the shortest imaging time from an image group depicted in (a) of FIG. 24 is depicted in (b) of FIG. 24 .
the image C is first obtained, and then the image B is obtained. Therefore, the image C is extracted.
the image B imaged in the main region can also be extracted with reference to the imaging region information.
an image with a focusing degree equal to a threshold or more may be extracted.
an image with the highest focusing degree may be extracted.
an image when an image is extracted with reference to the focusing information described above, the information to be referred to may be changed at each time, and a reference value of extraction may be changed at each time.
an image when an image is extracted with reference to the focusing degree, instead of extracting an image with a focusing degree equal to the threshold or more at all times, an image with a low focusing degree may be extracted at first, and an image with a high focusing degree may be gradually extracted as time elapses.
the images extracted in this manner are replayed as moving pictures, the subject initially blurred gradually becomes clear, and natural moving images can be obtained when abrupt focusing may provide uncomfortable feelings.
images may be selected and extracted so that the focused subject may be gradually blurred.
This image selection and extraction may be performed by displaying an image before the process and an image after the process on the liquid-crystal monitor 30 .

Landscapes

Physics & Mathematics (AREA)
General Physics & Mathematics (AREA)
Optics & Photonics (AREA)
Engineering & Computer Science (AREA)
Computer Vision & Pattern Recognition (AREA)
Multimedia (AREA)
Signal Processing (AREA)
Studio Devices (AREA)
Focusing (AREA)
Cameras In General (AREA)
Automatic Focus Adjustment (AREA)

US14/262,822 2011-10-28 2014-04-28 Imaging method using multifocal lens Active US9270878B2 (en)

Applications Claiming Priority (5)

Application Number	Priority Date	Filing Date	Title
JP2011-237635		2011-10-28
JP2011237635		2011-10-28
JP2011237634		2011-10-28
JP2011-237634		2011-10-28
PCT/JP2012/077304 WO2013061947A1 (fr)	2011-10-28	2012-10-23	Procédé d'imagerie et procédé de traitement d'image, programme utilisant ceux-ci, support d'enregistrement et dispositif d'imagerie

Related Parent Applications (1)

Application Number	Title	Priority Date	Filing Date
PCT/JP2012/077304 Continuation WO2013061947A1 (fr)	2011-10-28	2012-10-23	Procédé d'imagerie et procédé de traitement d'image, programme utilisant ceux-ci, support d'enregistrement et dispositif d'imagerie

Publications (2)

Publication Number	Publication Date
US20140232928A1 US20140232928A1 (en)	2014-08-21
US9270878B2 true US9270878B2 (en)	2016-02-23

Family

ID=48167777

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
US14/262,822 Active US9270878B2 (en)	2011-10-28	2014-04-28	Imaging method using multifocal lens

Country Status (5)

Country	Link
US (1)	US9270878B2 (fr)
EP (1)	EP2772782B1 (fr)
JP (1)	JP5647739B2 (fr)
CN (1)	CN103907043B (fr)
WO (1)	WO2013061947A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
EP3893051A1 (fr) *	2020-04-09	2021-10-13	Sick Ag	Acquisition de données d'image d'un objet mobile
US11172112B2 (en)	2019-09-09	2021-11-09	Embedtek, LLC	Imaging system including a non-linear reflector

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
CN104204941B (zh) *	2012-03-21	2016-01-06	富士胶片株式会社	摄像装置
JP2014103353A (ja) *	2012-11-22	2014-06-05	Samsung R&D Institute Japan Co Ltd	認識装置、認識方法、実装装置及び実装方法
JP5743236B2 (ja)	2013-09-17	2015-07-01	オリンパス株式会社	撮影機器及び撮影方法
KR20150077646A (ko) *	2013-12-30	2015-07-08	삼성전자주식회사	이미지 처리 장치 및 방법
KR102149463B1 (ko) *	2014-02-19	2020-08-28	삼성전자주식회사	이미지를 처리하기 위한 전자 장치 및 방법
JP6415196B2 (ja) *	2014-09-08	2018-10-31	キヤノン株式会社	撮像装置および撮像装置の制御方法
JP6042936B2 (ja) *	2015-04-23	2016-12-14	オリンパス株式会社	撮影機器及び撮影方法
US20170054897A1 (en) *	2015-08-21	2017-02-23	Samsung Electronics Co., Ltd.	Method of automatically focusing on region of interest by an electronic device
JP6800628B2 (ja) *	2016-06-22	2020-12-16	キヤノン株式会社	追跡装置、追跡方法、及びプログラム
JPWO2018061876A1 (ja) *	2016-09-29	2019-08-29	株式会社ニコン	撮像装置
JP6838994B2 (ja) *	2017-02-22	2021-03-03	キヤノン株式会社	撮像装置、撮像装置の制御方法およびプログラム
JP2019086775A (ja) *	2017-11-06	2019-06-06	キヤノン株式会社	画像処理装置、その制御方法、プログラム及び記憶媒体
US11985432B2 (en)	2019-03-25	2024-05-14	Nec Corporation	Image processing device and image processing method suitably applied to biometric authentication
CN110324532B (zh) *	2019-07-05	2021-06-18	Oppo广东移动通信有限公司	一种图像虚化方法、装置、存储介质及电子设备
JP7421008B2 (ja) *	2021-03-26	2024-01-23	富士フイルム株式会社	撮像装置、撮像方法、及びプログラム

Citations (15)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US4902115A (en) *	1986-09-22	1990-02-20	Olympus Optical Co., Ltd.	Optical system for endoscopes
JPH05199443A (ja)	1992-01-21	1993-08-06	Konica Corp	電子カメラの合焦位置検出装置
JPH07119893A (ja)	1993-10-27	1995-05-12	Chiyoda Corp	低温液化ガス配管の制御方法
US5845155A (en)	1996-08-29	1998-12-01	Asahi Kogaku Kogyo Kabushiki Kaisha	Multipoint autofocusing system
JP2001116980A (ja)	1999-10-18	2001-04-27	Fuji Photo Film Co Ltd	自動焦点カメラ及び撮影方法
JP2003098426A (ja)	2001-09-20	2003-04-03	Olympus Optical Co Ltd	撮影レンズ、その撮影レンズを用いたカメラ及びそのカメラにおける絞り
US6614998B1 (en)	1999-10-18	2003-09-02	Fuji Photo Film Co., Ltd.	Automatic focusing camera and shooting method
US20070279618A1 (en)	2004-10-15	2007-12-06	Matsushita Electric Industrial Co., Ltd.	Imaging Apparatus And Image Improving Method
JP2008058540A (ja)	2006-08-30	2008-03-13	Kyocera Corp	撮像装置、および画像処理方法
JP2008172523A (ja)	2007-01-11	2008-07-24	Fujifilm Corp	多焦点カメラ装置及びそれに用いられる制御方法並びにプログラム
US20090284641A1 (en) *	2002-09-13	2009-11-19	Canon Kabushiki Kaisha	Focus control apparatus, image sensor, program and storage medium
US20110002678A1 (en) *	2009-07-03	2011-01-06	Fujifilm Corporation	Photographing control device and method, as well as program
US20110134282A1 (en)	2009-12-04	2011-06-09	Nobuhiro Morita	Imaging device
US20120120232A1 (en) *	2009-05-21	2012-05-17	Takashi Nishikawa	Shape measuring device, observation device, and image processing method
US20140098228A1 (en) *	2005-12-08	2014-04-10	Smart Drive Systems, Inc.	Memory management in event recording systems

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
JP2006154465A (ja) *	2004-11-30	2006-06-15	Olympus Corp	焦点検出装置およびその制御方法
JP4118322B2 (ja) *	2005-11-22	2008-07-16	松下電器産業株式会社	撮影装置、携帯端末装置、撮影方法、およびプログラム
JP5154392B2 (ja) *	2008-12-12	2013-02-27	株式会社キーエンス	撮像装置
JP5437781B2 (ja)	2009-12-09	2014-03-12	富士フイルム株式会社	画像処理装置、方法およびプログラム

2012
- 2012-10-23 JP JP2013540778A patent/JP5647739B2/ja active Active
- 2012-10-23 EP EP12844116.9A patent/EP2772782B1/fr active Active
- 2012-10-23 WO PCT/JP2012/077304 patent/WO2013061947A1/fr not_active Ceased
- 2012-10-23 CN CN201280053034.6A patent/CN103907043B/zh active Active
2014
- 2014-04-28 US US14/262,822 patent/US9270878B2/en active Active

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US4902115A (en) *	1986-09-22	1990-02-20	Olympus Optical Co., Ltd.	Optical system for endoscopes
JPH05199443A (ja)	1992-01-21	1993-08-06	Konica Corp	電子カメラの合焦位置検出装置
JPH07119893A (ja)	1993-10-27	1995-05-12	Chiyoda Corp	低温液化ガス配管の制御方法
US5845155A (en)	1996-08-29	1998-12-01	Asahi Kogaku Kogyo Kabushiki Kaisha	Multipoint autofocusing system
JP2001116980A (ja)	1999-10-18	2001-04-27	Fuji Photo Film Co Ltd	自動焦点カメラ及び撮影方法
US6614998B1 (en)	1999-10-18	2003-09-02	Fuji Photo Film Co., Ltd.	Automatic focusing camera and shooting method
JP2003098426A (ja)	2001-09-20	2003-04-03	Olympus Optical Co Ltd	撮影レンズ、その撮影レンズを用いたカメラ及びそのカメラにおける絞り
US20090284641A1 (en) *	2002-09-13	2009-11-19	Canon Kabushiki Kaisha	Focus control apparatus, image sensor, program and storage medium
US20070279618A1 (en)	2004-10-15	2007-12-06	Matsushita Electric Industrial Co., Ltd.	Imaging Apparatus And Image Improving Method
JP2008516299A (ja)	2004-10-15	2008-05-15	松下電器産業株式会社	撮像装置及び画像改質処理方法
US20140098228A1 (en) *	2005-12-08	2014-04-10	Smart Drive Systems, Inc.	Memory management in event recording systems
JP2008058540A (ja)	2006-08-30	2008-03-13	Kyocera Corp	撮像装置、および画像処理方法
JP2008172523A (ja)	2007-01-11	2008-07-24	Fujifilm Corp	多焦点カメラ装置及びそれに用いられる制御方法並びにプログラム
US20120120232A1 (en) *	2009-05-21	2012-05-17	Takashi Nishikawa	Shape measuring device, observation device, and image processing method
US20110002678A1 (en) *	2009-07-03	2011-01-06	Fujifilm Corporation	Photographing control device and method, as well as program
JP2011013514A (ja)	2009-07-03	2011-01-20	Fujifilm Corp	撮影制御装置および方法並びにプログラム
US20110134282A1 (en)	2009-12-04	2011-06-09	Nobuhiro Morita	Imaging device
JP2011118235A (ja)	2009-12-04	2011-06-16	Ricoh Co Ltd	撮像装置

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Extended European search report, dated Jun. 23, 2015, in corresponding European Patent Application No. 12844116.9.
International Search Report PCT/JP2012/077304 dated Feb. 5, 2013.
Written Opinion of the International Searching Authority PCT/ISA/237 dated Feb. 5, 2013.

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US11172112B2 (en)	2019-09-09	2021-11-09	Embedtek, LLC	Imaging system including a non-linear reflector
EP3893051A1 (fr) *	2020-04-09	2021-10-13	Sick Ag	Acquisition de données d'image d'un objet mobile
US11375102B2 (en)	2020-04-09	2022-06-28	Sick Ag	Detection of image data of a moving object

Also Published As

Publication number	Publication date
EP2772782B1 (fr)	2017-04-12
EP2772782A1 (fr)	2014-09-03
WO2013061947A1 (fr)	2013-05-02
CN103907043A (zh)	2014-07-02
EP2772782A4 (fr)	2015-07-22
CN103907043B (zh)	2016-03-02
JP5647739B2 (ja)	2015-01-07
JPWO2013061947A1 (ja)	2015-04-02
US20140232928A1 (en)	2014-08-21

Publication	Publication Date	Title
US9270878B2 (en)	2016-02-23	Imaging method using multifocal lens
JP4288612B2 (ja)	2009-07-01	画像処理装置および方法、並びにプログラム
US9888182B2 (en)	2018-02-06	Display apparatus
JP5609270B2 (ja)	2014-10-22	撮像装置、撮像システム、撮像装置の制御方法およびプログラム
JP6457776B2 (ja)	2019-01-23	撮像装置および撮像装置の制御方法
JP5647751B2 (ja)	2015-01-07	撮像装置及び撮像方法
JP5464934B2 (ja)	2014-04-09	撮像装置及び撮像装置の制御方法
JP6474693B2 (ja)	2019-02-27	焦点検出装置、焦点検出方法、および記録媒体
JP2014081420A (ja)	2014-05-08	追尾装置及びその方法
JP5437781B2 (ja)	2014-03-12	画像処理装置、方法およびプログラム
JP7281977B2 (ja)	2023-05-26	焦点調節装置および焦点調節方法
JP5936358B2 (ja)	2016-06-22	画像表示装置、画像表示方法、撮像装置およびその制御方法、プログラムおよびそれを記憶する記憶媒体
JP5832353B2 (ja)	2015-12-16	撮像装置
JP2017129788A (ja)	2017-07-27	焦点検出装置及び方法、及び撮像装置
JP2006025311A (ja)	2006-01-26	撮像装置、及び画像取得方法
JP2017040726A (ja)	2017-02-23	撮像装置及びその制御方法
JP5409483B2 (ja)	2014-02-05	撮像装置
JP6778340B2 (ja)	2020-10-28	撮像装置、撮像方法、及びプログラム
JP2020165997A (ja)	2020-10-08	撮像装置、絞り値設定方法及び絞り値設定プログラム
US20110228146A1 (en)	2011-09-22	Imaging apparatus
JP2007104248A (ja)	2007-04-19	電子カメラおよびプログラム
JP5278483B2 (ja)	2013-09-04	撮像装置、撮像方法及び撮像プログラム
JP2010134309A (ja)	2010-06-17	オートフォーカス装置、オートフォーカス方法及び撮像装置
JP5493273B2 (ja)	2014-05-14	撮像装置、撮影方法及びプログラム
JP2009239792A (ja)	2009-10-15	撮像装置、画像処理装置、およびプログラム

Legal Events

Date	Code	Title	Description
2014-04-28	AS	Assignment	Owner name: FUJIFILM CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ONO, SHUJI;REEL/FRAME:032764/0113 Effective date: 20140422
2016-02-03	STCF	Information on status: patent grant	Free format text: PATENTED CASE
2019-08-08	MAFP	Maintenance fee payment	Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4
2023-08-09	MAFP	Maintenance fee payment	Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8