JP2007266662A - Thumbnail generating device and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate an image where a user has a deep impression, as a thumbnail image for a moving image without giving needless troubles to the user. <P>SOLUTION: In a thumbnail generating device used for an imaging apparatus capable of photographing a still image while photographing the moving image, when the still image is photographed while the moving image is photographed; a thumbnail image is generated concurrently with the photographing timing of the still image, and the thumbnail image is related to the photographed moving and still images. When a plurality of still images are photographed during the photography of the moving image, the reduced image of the most characteristic still image is generated as the thumbnail image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a thumbnail generation device used for an imaging device such as a digital still camera or a digital video camera. The present invention also relates to an imaging apparatus including the thumbnail generation apparatus.

  An imaging device such as a digital video camera is often provided with a mode for displaying thumbnail images. A thumbnail image for a moving image is normally generated at an appropriate timing such as when recording of a moving image starts or ends (hereinafter, this generating method is referred to as Conventional Method 1). This type of imaging apparatus is disclosed in, for example, Patent Document 1 below. The user selects a moving image to be reproduced from a plurality of recorded moving images based on the thumbnail images.

  Further, after the recording is finished, a method in which a user sets a desired image (any frame image) as a thumbnail image after confirming a moving image (moving image) taken by the user (hereinafter referred to as conventional method 2). Has also been proposed.

  There has also been proposed a method (hereinafter referred to as Conventional Method 3) in which an imaging device is provided with a dedicated switch for generating a thumbnail and a thumbnail image is generated from a frame image at the timing when the dedicated switch is pressed (for example, Conventional Method 3) (for example, See Patent Document 2 below).

  On the other hand, an image pickup apparatus having a function of simultaneously taking still images during moving image shooting has also been proposed. This type of imaging apparatus is often capable of shooting and recording a high-resolution still image during moving image shooting. Accordingly, for example, it is possible to meet a demand for recording a decisive scene such as a goal as a high-resolution still image while recording a whole scene of a child running at an athletic meet as a moving image.

JP-A-10-28250 JP 2002-77803 A

  When the thumbnail image is generated at an appropriate timing as in the conventional method 1, the thumbnail image often does not represent the characteristics of the captured moving image, and a desired moving image file is efficiently selected based on the thumbnail image. In many cases.

  If an image that has a deep impression to the user can be used as a thumbnail image, it becomes easy to select a moving image file during reproduction. This can be achieved by conventional methods 2 or 3. However, with the conventional method 2, the user is required to take extra time. Further, even if the conventional method 3 is adopted, the user is eventually required to push the dedicated switch during recording. In addition, it is necessary to provide a dedicated switch.

  Accordingly, an object of the present invention is to provide a thumbnail generation device and an imaging device including the thumbnail generation device capable of generating an image that is deeply impressive to the user as a thumbnail image for a moving image without taking extra time for the user. To do.

  In order to achieve the above object, the thumbnail generating apparatus according to the present invention is configured to capture a moving image based on an imaging signal corresponding to a subject from an imaging unit provided in an imaging device capable of capturing a still image during moving image shooting. In a thumbnail generation device that generates an associated thumbnail image, when a still image is shot during shooting of the moving image, the thumbnail image is generated using the imaging signal at a predetermined timing based on the shooting timing of the still image. It is characterized by generating.

  During moving image shooting, the user captures a special state of the subject as a still image. Therefore, if configured as described above, it is possible to generate an image that is deeply impressed for the user or an image that well represents the features of the moving image as a thumbnail image without extra effort for the user.

  Specifically, for example, the thumbnail image is generated based on the imaging signal representing the still image.

  Further, for example, when there are a plurality of the still images taken during the shooting of the moving image, the selection unit is configured to select any one of the plurality of still images as an adopted still image. The thumbnail image is generated using the imaging signal at a predetermined timing with reference to the shooting timing of the adopted still image.

  Specifically, for example, which of the plurality of still images is selected as the adopted still image is set in advance.

  More specifically, for example, a predetermined contrast detection area is provided in each of the plurality of still images, and the contrast detection is performed for each still image based on the imaging signal representing the captured still image. Contrast detection means for detecting the contrast amount of the image in the region is provided, and the selection means selects the adopted still image based on the comparison between the contrast amounts.

  Further, specifically, for example, a predetermined detection area is provided in each of the plurality of still images, and the detection area is detected for each still image based on the imaging signal representing the captured still image. Area dividing means for dividing the image into a plurality of luminance areas or a plurality of color areas, and the selecting means has a luminance area having the maximum size in each detection area or a maximum size in each detection area A color region is specified for each still image, and the adopted still image is selected based on a comparison of the sizes of the luminance regions having the maximum size or the color regions having the maximum size.

  More specifically, for example, the imaging device includes a sound input unit that receives sound input from the outside, and the thumbnail generation device corresponds to the shooting timing of the still image for each captured still image. Sound detecting means for detecting the intensity or the magnitude of the sound or the intensity or the magnitude of the frequency component of the predetermined band of the sound, and the selecting means comprises the sound intensity or the magnitudes of the sounds or the sound The adopted still image is selected based on a comparison between the strengths or sizes of frequency components in a predetermined band.

  Also, for example, when there are a plurality of still images taken during the shooting of the moving image, some or all of the plurality of still images are adopted still images, there are a plurality of the adopted still images, and the adoption For each of the still images, a thumbnail element image is formed based on the imaging signal at a predetermined timing based on the shooting timing of the employed still image, and the thumbnail images are generated by combining the thumbnail element images.

  In order to achieve the above object, an imaging apparatus according to the present invention is an imaging apparatus capable of shooting a still image during moving image shooting, and includes the above-described imaging means and the above-described thumbnail generation device.

  According to the present invention, it is possible to generate, as a thumbnail image, an image that deeply gives an impression to the user or an image that well expresses the characteristics of a moving image, without taking extra time for the user.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In the drawings to be referred to, the same parts are denoted by the same reference numerals.

  The first, second and third embodiments will be described later. First, the contents common to all of them will be described with reference to FIGS.

  FIG. 1 is an overall block diagram of an imaging apparatus 1 according to an embodiment of the present invention. The imaging device 1 is, for example, a digital still camera or a digital video camera. The imaging device 1 can shoot moving images and still images, and can also shoot still images simultaneously during moving image shooting.

  The imaging device 1 includes an imaging unit 11, an AFE (Analog Front End) 12, a video signal processing unit 13, a microphone (sound input means) 14, an audio signal processing unit 15, a compression processing unit 16, and an internal memory. As an example, an SDRAM (Synchronous Dynamic Random Access Memory) 17, a memory card 18, a decompression processing unit 19, a video output circuit (video output circuit) 20, an audio output circuit 21, and a TG (timing generator) 22 , A CPU (Central Processing Unit) 23, a bus 24, a bus 25, an operation unit 26, a display unit 27, and a speaker 28. The operation unit 26 includes a recording button 26a, a shutter button 26b, an operation key 26c, and the like.

  The bus 24 is connected to the imaging unit 11, AFE 12, video signal processing unit 13, audio signal processing unit 15, compression processing unit 16, expansion processing unit 19, video output circuit 20, audio output circuit 21, and CPU 23. Each part connected to the bus 24 exchanges various signals (data) via the bus 24.

  The bus 25 is connected to the video signal processing unit 13, the audio signal processing unit 15, the compression processing unit 16, the expansion processing unit 19, and the SDRAM 17. Each part connected to the bus 25 exchanges various signals (data) via the bus 25.

  The TG 22 generates a timing control signal for controlling the timing of each operation in the entire imaging apparatus 1, and provides the generated timing control signal to each unit in the imaging apparatus 1. Specifically, the timing control signal is given to the imaging unit 11, the video signal processing unit 13, the audio signal processing unit 15, the compression processing unit 16, the expansion processing unit 19, and the CPU 23. The timing control signal includes a vertical synchronization signal Vsync and a horizontal synchronization signal Hsync.

  The CPU 23 comprehensively controls the operation of each unit in the imaging apparatus 1. The operation unit 26 receives an operation by a user. The operation content given to the operation unit 26 is transmitted to the CPU 23. The SDRAM 17 functions as a frame memory. Each unit in the image pickup apparatus 1 temporarily records various data (digital signals) in the SDRAM 17 during signal processing as necessary via an SDRAM controller (not shown) that controls writing and reading of data to and from the SDRAM 17. To do.

  The memory card 18 is an external recording medium, for example, an SD (Secure Digital) memory card. The memory card 18 is detachable from the imaging device 1. The recorded content of the memory card 18 can be freely read out by an external personal computer or the like via a terminal of the memory card 18 or a communication connector (not shown) provided in the imaging device 1. In this embodiment, the memory card 18 is illustrated as an external recording medium. However, the external recording medium is composed of one or a plurality of randomly accessible recording media (semiconductor memory, memory card, optical disk, magnetic disk, etc.). can do.

  FIG. 2 is an internal configuration diagram of the imaging unit 11 of FIG. The imaging unit 11 includes an optical system 35 including a plurality of lenses including a zoom lens 30 and a focus lens 31, an aperture 32, an imaging element 33, and a driver 34. The driver 34 includes a motor for realizing movement of the zoom lens 30 and the focus lens 31 and adjustment of the opening amount of the diaphragm 12.

  Incident light from the subject (imaging target) enters the image sensor 33 through the zoom lens 30 and the focus lens 31 that constitute the optical system 35, and the diaphragm 32. The TG 22 generates a drive pulse for driving the image sensor 33 in synchronization with the timing control signal, and applies the drive pulse to the image sensor 33.

  The image sensor 33 is composed of, for example, a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like. The image sensor 33 photoelectrically converts an optical image incident through the optical system 35 and the diaphragm 32 and outputs an electrical signal obtained by the photoelectric conversion to the AFE 12. More specifically, the imaging device 33 includes a plurality of pixels (light receiving pixels; not shown) that are two-dimensionally arranged in a matrix, and in each photographing, each pixel receives a signal charge having a charge amount corresponding to the exposure time. store. The electrical signal from each pixel having a magnitude proportional to the amount of the stored signal charge is sequentially output to the subsequent AFE 12 in accordance with the drive pulse from the TG 22.

  The image sensor 33 is a single-plate image sensor capable of color photography. Each pixel constituting the image sensor 33 is provided with, for example, one of red (R), green (G), and blue (B) color filters (not shown). Note that a three-plate image sensor can also be adopted as the image sensor 33.

  The AFE 12 includes an amplifier circuit (not shown) that amplifies the analog electric signal that is an output signal of the image pickup unit 11 (that is, an output signal of the image pickup element 33), and an A / And a D (analog-digital) conversion circuit (not shown). The output signal of the imaging unit 11 converted into a digital signal by the AFE 12 is sequentially sent to the video signal processing unit 13. Further, the CPU 23 adjusts the amplification degree of the amplification circuit based on the signal level of the output signal of the imaging unit 11.

  Hereinafter, a signal corresponding to the subject output from the imaging unit 11 or the AFE 12 is referred to as an imaging signal.

  FIG. 3 is an internal block diagram of the video signal processing unit 13. The video signal processing unit 13 includes a moving image / still image processing unit 41 and a thumbnail generation unit 42.

  The moving image / still image processing unit 41 generates a video signal representing a video (captured image) obtained by shooting by the imaging unit 11 based on the imaging signal from the AFE 12, and sends the generated video signal to the compression processing unit 16. . The video signal is composed of a luminance signal Y representing the luminance of the photographed image and color difference signals U and V representing the color of the photographed image. The moving image / still image processing unit 41 generates both a moving image video signal and a still image video signal.

  The microphone 14 converts the sound (sound) given from the outside into an analog electric signal and outputs it. The audio signal processing unit 15 converts an electrical signal (audio analog signal) output from the microphone 14 into a digital signal. The digital signal obtained by this conversion is sent to the compression processing unit 16 as an audio signal representing the audio input to the microphone 14.

  The compression processing unit 16 compresses the video signal from the video signal processing unit 13 (moving image / still image processing unit 41) using a predetermined compression method. For video, for example, compression of video signals is performed using a compression method such as MPEG (Moving Picture Experts Group), and for still images, for example, a compression method such as JPEG (Joint Photographic Experts Group) is used. To compress the video signal. During video or still image shooting, the compressed video signal is sent to the memory card 18.

  For example, the image size of each captured image constituting the moving image is reduced through thinning processing or the like in the moving image / still image processing unit 41 or the like as necessary. For example, such a reduction process is not performed on the still image (may be performed).

  The compression processing unit 16 compresses the audio signal from the audio signal processing unit 15 using a predetermined compression method such as AAC (Advanced Audio Coding). At the time of moving image shooting, the video signal from the video signal processing unit 13 and the audio signal from the audio signal processing unit 15 are compressed in the compression processing unit 16 while being correlated with each other in time, and the compressed signals are stored in the memory card. 18 is sent.

  The recording button 26a is a push button switch for the user to instruct the start and end of shooting of a moving image (moving image), and the shutter button 26b is a button for the user to instruct shooting of a still image (still image). It is a button switch. Moving image shooting is started and ended according to the operation on the recording button 26a, and still image shooting is performed according to the operation on the shutter button 26b. One frame image is obtained in one frame. The length of each frame is 1/60 seconds, for example. In this case, a collection of frame images (stream images) that are sequentially acquired at a period of 1/60 seconds constitutes a moving image.

  The operation mode of the imaging apparatus 1 includes a shooting mode capable of shooting moving images and still images, and a playback mode for reproducing and displaying moving images or still images stored in the memory card 18 on the display unit 27. The reproduction mode includes a thumbnail image display mode for reproducing and displaying a thumbnail image associated with a moving image or a still image stored in the memory card 18 on the display unit 27. Transition between the modes is performed according to the operation on the operation key 26c.

  When the user presses the recording button 26a in the shooting mode, under the control of the CPU 23, the video signal of each frame after the pressing and the corresponding audio signal are sequentially recorded on the memory card 18 via the compression processing unit 16. Is done. That is, the captured image (that is, the frame image) of each frame is sequentially stored in the memory card 18 together with the audio signal. When the user presses the recording button 26a again after starting the moving image shooting, the moving image shooting ends. That is, recording of the video signal and the audio signal to the memory card 18 is completed, and shooting of one moving image is completed.

  In the shooting mode, when the user presses the shutter button 26b, a still image is shot. Specifically, under the control of the CPU 23, the video signal of one frame immediately after being pressed is recorded on the memory card 18 through the compression processing unit 16 as a video signal representing a still image. It is also possible to simultaneously shoot a still image while shooting a moving image. In this case, signal processing related to the moving image and signal processing related to the still image are performed in parallel based on the imaging signal of the same frame. Note that a common circuit may be used in a time-sharing manner so that signal processing related to moving images and signal processing related to still images may be performed at separate timings. For example, signal processing related to a still image is performed after moving image shooting is completed. In this case, for example, an imaging signal (photographed image) necessary for the signal processing performed afterwards may be temporarily stored in the SDRAM 17.

  When the user performs a predetermined operation on the operation key 26 c in the reproduction mode, a compressed video signal representing a moving image or a still image recorded on the memory card 18 is sent to the expansion processing unit 19. The decompression processing unit 19 decompresses the received video signal and sends it to the video output circuit 20. In the shooting mode, the generation of the video signal by the video signal processing 13 is normally performed regardless of whether or not a moving image or a still image is being shot, and the video signal is sent to the video output circuit 20. It is done.

  The video output circuit 20 converts a given digital video signal into a video signal (for example, an analog video signal) in a format that can be displayed on the display unit 27 and outputs the video signal. The display unit 27 is a display device such as a liquid crystal display, and displays an image corresponding to the video signal output from the video output circuit 20. That is, the display unit 27 is an image based on the imaging signal currently output from the imaging unit 11 (an image representing the current subject), a moving image (moving image) or a still image (still image) recorded on the memory card 18. Image).

  When a moving image is reproduced in the reproduction mode, a compressed audio signal corresponding to the moving image recorded on the memory card 18 is also sent to the expansion processing unit 19. The decompression processing unit 19 decompresses the received audio signal and sends it to the audio output circuit 21. The audio output circuit 21 converts a given digital audio signal into an audio signal in a format that can be output by the speaker 28 (for example, an analog audio signal) and outputs the audio signal to the speaker 28. The speaker 28 outputs the sound signal from the sound output circuit 21 to the outside as sound (sound).

  It can be considered that the display unit 27 and the speaker 28 are provided in an external television or the like. In this case, the video signal output from the video output circuit 20 and the audio signal output from the audio output circuit 21 are supplied to an external television or the like via a connector (not shown).

  The video signal processing unit 13 also detects an AF evaluation value according to the contrast amount in the focus detection area in the captured image, and detects an AE evaluation value according to the brightness of the captured image. An evaluation value detection circuit, a motion detection circuit that detects the motion of an image, and the like are included. The CPU 23 adjusts the position of the focus lens 31 via the driver 34 in FIG. 2 in accordance with the AF evaluation value, thereby forming an optical image of the subject on the imaging surface (light receiving surface) of the image sensor 33. Further, the CPU 23 adjusts the opening amount of the diaphragm 32 (and the amplification degree of the amplification circuit of the AFE 12 as necessary) via the driver 34 of FIG. Control).

  When shooting a still image, the thumbnail generation unit 42 in FIG. 3 generates a thumbnail image for the still image based on the imaging signal at the still image shooting timing under the control of the CPU 23. More specifically, a video signal composed of a luminance signal Y and color difference signals U and V representing a thumbnail image is generated. This thumbnail image is a reduced image obtained by reducing one still image (still image) that has been shot and recorded by thinning processing or the like. A thumbnail image (a video signal representing a thumbnail image) for a still image is recorded on the memory card 18 through compression (such as JPEG) by the compression processing unit 16 while being associated with the captured still image. The thumbnail image associated with the photographed still image is referred to as a still image thumbnail image.

  In addition, when shooting a moving image, the thumbnail generation unit 42 generates a thumbnail image for the moving image based on the imaging signal at a predetermined timing under the control of the CPU 23. More specifically, a video signal composed of a luminance signal Y and color difference signals U and V representing a thumbnail image is generated. This generation timing will be described in detail later. A thumbnail image of a moving image is recorded on the memory card 18 through compression (such as JPEG) by the compression processing unit 16 while being associated with the captured moving image. The thumbnail image generated for recording in association with the captured moving image is referred to as a moving image thumbnail image. Note that the image size of the moving image thumbnail image is made smaller than the image size of each image constituting the moving image (or the image size of the still image) through thinning-out processing or the like. However, this reduction in image size is not essential.

  In the thumbnail image display mode, the moving image and still image thumbnail images stored in the memory card 18 are displayed on the display unit 27 through the expansion processing by the expansion processing unit 19 and the conversion processing by the video output circuit 20.

  FIG. 4 shows a display example of the display screen of the display unit 27 in the thumbnail image display mode. In FIG. 4, the display area of the display unit 27 is divided into four, and thumbnail images TN1, TN2, TN3, and TN4 are displayed at the upper left, upper right, lower left, and lower right of the display area, respectively. When another thumbnail image is stored in the memory card 18, the other thumbnail image is displayed by performing a predetermined operation on the operation key 26c. A mark 71 attached to the thumbnail images TN2 and TN3 indicates that the thumbnail image is a thumbnail image for a moving image. Therefore, the thumbnail images TN2 and TN3 are moving image thumbnail images, and the thumbnail images TN1 and TN4 are still image thumbnail images.

  In FIG. 4, 72 is a cursor on the display screen. The cursor 72 moves on the display screen by operating the operation key 26c. FIG. 4 shows a state in which the thumbnail image TN3 is selected by the cursor 72. A number such as “001” displayed at the lower right of each thumbnail image indicates the file number of each thumbnail image.

  When a predetermined key operation is performed while a certain thumbnail image is selected by the cursor 72, a moving image or a still image associated with the thumbnail image is read from the memory card 18. Then, the read moving image or still image is displayed using the entire display screen of the display unit 27. For example, when a key operation is performed so that a moving image corresponding to the thumbnail image TN3 is reproduced, the display screen of the display unit 27 transitions from the state illustrated in FIG. 4 to the state illustrated in FIG. 5 and corresponds to the thumbnail image TN3. The moving image is reproduced and displayed using the entire display screen of the display unit 27. In this state, when a key operation for stopping reproduction is performed, the display screen returns to the state shown in FIG.

  Next, a method for generating a moving image thumbnail image will be described. As examples illustrating the generation methods that can be employed, first, second, and third examples are illustrated below. Hereinafter, unless otherwise stated, description will be made with a focus on a certain moving image.

<< First Example >>
First, the first embodiment will be described. FIG. 6 is a flowchart showing the procedure for generating the moving image thumbnail image according to the first embodiment. The first embodiment assumes a case where only one still image is shot during moving image shooting.

  First, in step S1, the CPU 23 determines whether or not the recording button 26a is pressed in the shooting mode. When not pressed, the process of step S1 is repeated, and when it is pressed, the process proceeds to step S2 (Yes in step S1).

  When the process proceeds to step S2, signal processing relating to the moving image is started (recording is started). Specifically, a video signal representing a captured image of each frame under the control of the CPU 23 during the period from the transition to step S2 to the transition to step S7 due to the pressing of the recording button 26a again. That is, a moving image) and a corresponding audio signal are sequentially recorded on the memory card 18 via the compression processing unit 16.

  After the process of step S2, the process proceeds to step S3. In step S3, the CPU 23 determines whether the recording button 26a has been pressed again, that is, whether recording has been instructed. If the recording button 26a is pressed again (Yes in step S3), the process proceeds to step S7. If the recording button 26a is not pressed again (No in step S3), the process proceeds to step S4.

  In step S4, the CPU 23 determines whether or not the shutter button 26b for instructing to capture a still image is pressed. If the shutter button 26b has not been pressed (No in step S4), the process returns to step S3. If the shutter button 26b has been pressed (Yes in step S4), the process proceeds to step S5.

  In step S5, signal processing related to a still image is performed in parallel with signal processing related to a moving image. Specifically, in parallel with the signal processing related to the moving image, under the control of the CPU 23, the moving image / still image processing unit 41 displays a video representing a still image from the imaging signal of the current frame (hereinafter referred to as a still image shooting frame). Generate a signal. In step S6, the thumbnail generation unit 42 generates a moving image thumbnail image from the imaging signal of the still image shooting frame. As a result, the moving image thumbnail image becomes a reduced image of the captured still image (or the same image as the still image).

  Steps S5 and S6 are performed in parallel. The video signal representing the still image generated in step S5 is recorded on the memory card 18 via the compression processing unit 16. The video signal representing the thumbnail image for moving image generated in step S 6 is temporarily recorded in the SDRAM 17. When the process of step S6 is completed, the process returns to step S3.

  In step S7, which is shifted from step S3 when an instruction to stop recording is given, the signal processing relating to the moving image ends (recording ends). That is, the process of recording the video signal representing the captured image of each frame and the audio signal corresponding to the video signal, which has been performed from step S <b> 2, is stopped. In subsequent step S8, the moving image thumbnail image (video signal representing the moving image thumbnail image) generated in step S6 is recorded in the memory card 18 while being associated with the captured moving image. When the process of step S8 is completed, the process of FIG. 6 ends.

  During moving image shooting, the user captures a special state of the subject as a still image. A still image taken during moving image shooting can be said to be an image having a special meaning for the user. According to the present embodiment, an image corresponding to a still image captured at the same time is associated with a moving image as a moving image thumbnail image. Therefore, the moving image thumbnail image is an image that clearly represents the characteristics of the moving image or an image that has a deep impression to the user. Become. As a result, it is possible to facilitate the search for a desired moving image and improve user convenience. In addition, when shooting a still image, the user inevitably operates the shutter button 26b, so that an unnecessary operation burden is not imposed on the user when selecting a moving image thumbnail image.

  As a typical example, the example in which the moving image thumbnail image is generated from the imaging signal of the still image shooting frame has been described above. However, the moving image thumbnail image is generated from the imaging signal before k frames or after the k frame of the still image shooting frame. You may make it produce | generate (k is a natural number with which the numerical value was predetermined, for example, is 1-several 10). In this case, the moving image thumbnail image generated in step S6 is a reduced image (or the captured image itself) of the captured image before or after k frames of still image shooting. This is the same in the second and third embodiments described later. Even if the image from which the moving image thumbnail image is created does not match the completely captured still image, the same effect as described above can be obtained.

  Further, in the first embodiment and the second and third embodiments described later, if the shutter button 26b is never pressed during moving image shooting, a reduced image (or a shot image) of a shot image at a preset timing is used. Itself) is recorded in the memory card 18 as a thumbnail image for a moving image while being associated with the captured moving image. For example, if a thumbnail image for a movie is generated based on the image of the frame immediately before or after the movie is taken, the image at the beginning or end of the movie, or the image of the frame after a certain time has elapsed since the movie was started. Good.

  The still image thumbnail image is generated, for example, in step S6. The generated still image thumbnail image is recorded in the memory card 18 via the compression processing unit 16 while being associated with the still image generated in step S5. Typically, the moving image thumbnail image and the still image thumbnail image generated in step S6 are the same.

<< Second Example >>
Next, a second embodiment will be described. FIG. 7 is a flowchart showing a procedure for generating a moving image thumbnail image according to the second embodiment.

  The generation procedure according to the second embodiment includes steps S1 to S7 and steps S10 to S12. Since the process of step S1-S7 is the same as those of the production | generation procedure which concerns on 1st Example, the overlapping description regarding them is abbreviate | omitted.

  The second embodiment assumes a case where a plurality of still images are shot during moving image shooting. For the sake of concrete explanation, a case where two still images are shot during moving image shooting will be given as an example as appropriate.

  When the first shutter button 26b is pressed during moving image shooting, a video signal representing the first still image is generated in step S5, and the video signal is recorded in the memory card 18. On the other hand, in step S6, a first moving image thumbnail image is generated from an imaging signal of a frame in which the first still image is captured (hereinafter referred to as a first still image capturing frame).

  Thereafter, when the second shutter button 26b is pressed while moving image shooting is continuously performed, a video signal representing the second still image is generated in step S5, and the video is displayed. The signal is recorded on the memory card 18. On the other hand, in step S6, a second moving image thumbnail image is generated from an imaging signal of a frame in which the second still image is captured (hereinafter referred to as a second still image capturing frame).

  The first and second still image shooting frames are different frames, and the first and second moving image thumbnail images are different moving image thumbnail images. Video signals representing the first and second moving image thumbnail images are temporarily recorded in the SDRAM 17. When n still images are shot during moving image shooting, n moving image thumbnail images are created (n is an integer of 3 or more) in the same manner as described above.

  In step S7, which is shifted from step S3 when an instruction to stop recording is given, the signal processing relating to the moving image ends (recording ends). In the second embodiment, after step S7, the process proceeds to step S10. In step S10, it is determined whether or not a plurality of still images have been shot during moving image shooting. If only one still image is shot during moving image shooting (No in step S10), the process proceeds to step S12, and the same processing as step S8 in the first embodiment is performed. That is, the moving image thumbnail image (video signal representing the moving image thumbnail image) generated in step S6 is recorded on the memory card 18 while being associated with the captured moving image.

  As described above, when a plurality of still images have been taken (Yes in step S10), the process proceeds from step S10 to step S11. In step S11, one moving image thumbnail image is selected from a plurality of moving image thumbnail images created according to a predetermined rule. When the selection in step S11 is completed, the process proceeds to step S12, and one moving image thumbnail image (video signal representing the moving image thumbnail image) selected in step S11 is stored in the memory card 18 while being associated with the captured moving image. To be recorded. When the process of step S12 is completed, the process of FIG. 7 ends.

  Below, the 1st, 2nd, 3rd, and 4th selection method employable as a selection method in Step S11 is illustrated. The still image corresponding to the moving image thumbnail image selected in step S11 is referred to as an adopted still image. If the first still image is an adopted still image, the first moving image thumbnail image is selected as the one to be associated with the moving image in step S11. Actually, the adopted still image is selected in step S11. Thus, the moving image thumbnail image to be selected is automatically determined in step S11.

  As is clear from the above description, the moving image thumbnail image selected in step S11 is a predetermined timing (the still image shooting frame itself or its frame) based on the shooting timing (still image shooting frame) of the adopted still image. Generated on the basis of imaging signals in the preceding and following frames).

  The selection means for selecting the adopted still image is realized mainly by the CPU 23 and the video signal processing unit 13.

[First selection method]
First, the first selection method will be described. In the first selection method, it is determined in advance which still image is selected as the adopted still image from among a plurality of still images taken.

  For example, the first still image or the last captured still image during moving image shooting is selected as the adopted still image. For example, when the first, second,..., Nth still images are captured in this order during moving image shooting (n is an integer of 3 or more), the first or nth still image is adopted. Selected as. Of course, any one of the second to (n-1) th still images may be selected as the adopted still image.

[Second selection method]
Next, the second selection method will be described. In the second selection method, the contrast amount of the image is referred to.

  In FIG. 8, 75 represents the entire area of the image obtained from the imaging signal for one frame, and 76 represents a partial area of the entire area. 76 is referred to as a contrast detection region. The contrast detection area 76 is provided near the center of the entire area 75, for example. In FIG. 8, the contrast detection area 76 is a single rectangular area, but the contrast detection area 76 may be configured by a plurality of areas provided in the entire area 75. Further, the entire area 75 itself may be used as the contrast detection area 76. Further, the entire area 75 may be regarded as the entire imaging area of the imaging element 33 in FIG.

  A contrast detection area 76 is defined for each of the still images shot during moving image shooting. A contrast detection unit 50 used when the second selection method is employed is shown in FIG. The contrast detection unit 50 is provided in the video signal processing unit 13 of FIG. The contrast detection unit 50 includes an extraction unit 51, an HPF (high pass filter) 52, and an integration unit 53.

  The extraction unit 51 is provided with a video signal generated by the moving image / still image processing unit 41 or an imaging signal from the AFE 12, which represents a captured still image. The extraction unit 51 extracts a luminance signal from a given video signal or imaging signal. At this time, only the luminance signal in the contrast detection area 76 is extracted. The HPF 52 extracts only a predetermined high frequency component from the luminance signal extracted by the extraction unit 51. As is well known, the extracted high frequency component increases substantially in proportion to the contrast amount of the image in the contrast detection region 76.

  The integrating unit 53 obtains a contrast evaluation value corresponding to the contrast amount of the image in the contrast detection region 76 by integrating the high frequency components extracted by the HPF 52. The contrast evaluation value is calculated for each captured still image and transmitted to the CPU 23. The contrast evaluation value increases as the contrast amount increases.

  The CPU 23 compares the contrast evaluation values calculated for each still image. Then, the still image corresponding to the largest contrast evaluation value, that is, the still image having the largest contrast amount of the image in the contrast detection area 76 is selected as the adopted still image, and thereby the moving image thumbnail to be selected in step S11. Determine the image.

  In a shooting composition with the sky or the like as the background, when comparing the case where a subject such as a person is placed small and the case where the subject is placed greatly enlarged, the latter usually has a higher contrast amount of the image. growing. In general, the latter is a photo that gives a deeper impression to the user. Therefore, if the second selection method is employed, the moving image thumbnail image is an image that clearly represents the characteristics of the moving image or an image that has a deep impression for the user. As a result, it is possible to facilitate the search for a desired moving image and improve user convenience.

[Third selection method]
Next, the third selection method will be described. In the third selection method, area division based on luminance information or color information is used.

  In FIG. 10, 75 represents the entire area of the image obtained from the imaging signal for one frame, and 77 represents a partial area of the entire area. 77 is called a detection region. The detection area 77 is provided near the center of the entire area 75, for example. Further, the entire area 75 may be regarded as the entire imaging area of the imaging element 33 in FIG.

  A detection area 77 is defined for each of the still images shot during moving image shooting. A description will be given with reference to FIG. 11 showing a flowchart of the processing procedure of the third selection method.

  First, attention is paid to one still image among a plurality of still images taken. In step S51, the detection area 77 is divided into a plurality of luminance areas or a plurality of color areas based on the imaging signal in the still image shooting frame.

  A case where the detection area 77 is divided into a plurality of luminance areas will be described. The brightness of each part in the detection area 77 is specified based on the imaging signal from the pixel corresponding to the detection area 77. The luminance signal corresponding to each pixel can take a value of 0 to 255, for example, but the numerical range of 0 to 255 is classified into a plurality of stages. For example, it is classified into nine luminance categories in increments of 30. As shown in FIG. 12, pixels with luminance signal values 0 to 30 are in the first luminance segment, pixels with luminance signal values 31 to 60 are in the second luminance segment,..., And luminance signal values are 211 to 240. Pixels are classified into the eighth luminance category, and pixels having luminance signal values of 241 to 255 are classified into the ninth luminance category.

  Thus, when the luminance signal values are classified into nine luminance categories, the detection area 77 is divided into nine luminance areas including the first, second,..., And ninth luminance areas. become. The first, second,..., And ninth luminance areas are classified into pixels classified into the first luminance category, pixels classified into the second luminance category,..., And the ninth luminance category, respectively. It is composed of pixels. The same luminance region is composed of pixels having similar luminance information. Depending on the imaging signal, the detection area 77 is divided into less than 9 luminance areas. For example, when a uniform flat image is given, the detection area 77 is occupied by a single luminance area.

  In step S52 following step S51, the luminance region having the maximum size (area) (hereinafter referred to as the maximum divided region A) is specified by counting the number of pixels constituting each luminance region. The size (area) of a certain luminance area is proportional to the number of pixels constituting the luminance area.

  The processing in steps S51 and S52 is performed for each captured still image, and the maximum divided area A is specified for each still image. In step S53, it is determined whether steps S51 and S52 have been performed for all still images. When steps S51 and S52 are performed on all still images, the process proceeds to step S54. In step S54, for example, the CPU 23 compares the sizes of the maximum divided areas A. As a result of the comparison, the still image corresponding to the largest divided area A having the largest size is selected as the adopted still image.

  For example, when the sizes of the maximum divided areas A corresponding to the first and second still images are “10” and “15”, respectively, the second still image is selected as the adopted still image.

  The same processing is performed when the detection area 77 is divided into a plurality of color areas. A case where the detection area 77 is divided into a plurality of color areas will be described.

  The color (hue) of each part in the detection area 77 is specified based on the imaging signal from the pixel corresponding to the detection area 77. Based on this imaging signal, the color corresponding to each pixel is classified into a plurality of color classifications. For example, classification is performed by nine color classifications of the first to ninth color classifications.

  When the colors corresponding to each pixel are classified into nine color sections, the detection area 77 is divided into nine color areas including the first, second,..., And ninth color areas. become. The first, second,..., And ninth color regions are classified into pixels classified into the first color classification, pixels classified into the second color classification,..., And ninth color classification, respectively. It is composed of pixels. The same color area is composed of pixels having similar color information. Depending on the imaging signal, the detection area 77 is divided into less than 9 color areas. For example, when a uniform flat image is given, the detection area 77 is occupied by a single color area.

  In step S52, the number of pixels constituting each color region is counted to identify the color region (referred to as the maximum divided region B) having the maximum size (area). The size (area) of a certain color area is proportional to the number of pixels constituting the color area.

  The processes in steps S51 and S52 are performed for each captured still image, and the maximum divided area B is specified for each still image. In step S53, it is determined whether steps S51 and S52 have been performed for all still images. When steps S51 and S52 are performed on all still images, the process proceeds to step S54. In step S54, for example, the CPU 23 compares the sizes of the maximum divided areas B. As a result of the comparison, the still image corresponding to the largest divided region B having the largest size is selected as the adopted still image.

  For example, a composition in which a subject 78 having a green background on a white background is relatively small as shown in FIG. 13A and a subject 78 having a green background on a white background as shown in FIG. 13B. Consider a comparison with a composition that is relatively large. In this case, the maximum divided areas A and B are larger in the latter case where the subject 78 is enlarged.

  The user often performs shooting so that the subject of interest occupies a large area on the image, and the image obtained by such shooting has a deeper impression on the user. Therefore, if the third selection method is adopted, the moving image thumbnail image is an image that clearly represents the characteristics of the moving image or an image that has a deep impression for the user. As a result, it is possible to facilitate the search for a desired moving image and improve user convenience.

  Note that the processing (area dividing means) in step S51 is mainly realized by the CPU 23 or the video signal processing unit 13.

[Fourth selection method]
Next, the fourth selection method will be described. In the fourth selection method, attention is focused on the intensity of sound input via the microphone 14 during moving image shooting.

  The audio signal processing unit 15 has a function of detecting the intensity or magnitude of a sound or the intensity or magnitude of a frequency component in a predetermined band of sound input via the microphone 14 during moving image shooting. A sound intensity value corresponding to the intensity or magnitude or the intensity or magnitude of a frequency component of a predetermined band of sound is calculated for each captured still image. In the following description, it is assumed that the sound intensity value is a value corresponding to the intensity of the sound or the intensity of the frequency component in a predetermined band of the sound. When the sound intensity value is a value corresponding to the volume of the sound or the frequency component of the predetermined band of the sound, the following “strength” word related to the sound may be read as “volume” .

  The sound intensity value corresponding to a certain still image is, for example, the average value or peak value (maximum value) of the sound intensity or the intensity of the frequency component in a predetermined band of the sound during the sound evaluation period corresponding to the still image. The average value or peak value (maximum value). The sound evaluation period is a period including a still image shooting frame, for example, a period of the still image shooting frame itself or a period of a plurality of predetermined frames centering on the still image shooting frame.

  The sound intensity value increases as the sound intensity increases. The frequency component of the predetermined band is, for example, a human voice frequency band component or an audible frequency band component. The audio signal processing unit 15 extracts the frequency component of the predetermined band from the audio signal input via the microphone 14 using a bandpass filter or the like.

  The CPU 23 compares the sound intensity values calculated for each still image shot during moving image shooting. Then, for example, the still image corresponding to the maximum sound intensity value is selected as the adopted still image.

  When shooting a concert or party venue, it can be considered that the state in which the sound intensity is high coincides with the state in which the shooting site is exciting. For this reason, it can be considered that a still image corresponding to a large sound intensity value has a deep impression for the user. Therefore, if the fourth selection method is employed, the moving image thumbnail image is an image that clearly represents the characteristics of the moving image or an image that has a deep impression on the user. As a result, it is possible to facilitate the search for a desired moving image and improve user convenience.

<< Third Example >>
Next, a third embodiment will be described. FIG. 14 is a flowchart showing a procedure for generating a moving image thumbnail image according to the third embodiment.

  The generation procedure according to the third embodiment includes steps S1 to S7 and steps S10, S21 and S22. The processes in steps S1 to S7 are the same as those in the generation procedure according to the first embodiment, and the processes in step S10 are the same as those in the generation procedure according to the second embodiment. To do.

  In the third embodiment, similarly to the second embodiment, it is assumed that a plurality of still images are shot during moving image shooting.

  In the second embodiment, the case has been described in which the first still image is captured in the first still image capturing frame and then the second still image is captured in the second still image capturing frame. Also in the third embodiment, it is assumed that the first and second still images are taken. Then, after shooting the second still image, the third still image is shot in the third still image shooting frame, and then the fourth still image is shot in the fourth still image shooting frame. Assume a case.

  In step S6, first, second, third, and fourth moving image thumbnail images are generated from the imaging signals of the first, second, third, and fourth still image capturing frames, respectively. In this embodiment, each moving image thumbnail image generated in step S6 is a part of the moving image thumbnail image to be associated with one moving image. Therefore, in the present embodiment, the first, second, third, and fourth moving image thumbnail images generated in step S6 are respectively referred to as the first, second, third, and fourth thumbnail element images. Call.

  In the third embodiment, as in the second embodiment, the process proceeds to step S10 after the process in step S7. In step S10, it is determined whether or not a plurality of still images have been shot during moving image shooting. If only one still image is shot during moving image shooting (No in step S10), the process proceeds to step S22, and the same processing as step S8 in the first embodiment is performed.

  As described above, when a plurality of still images have been taken (Yes in step S10), the process proceeds from step S10 to step S21. In step S21, the final thumbnail image for moving image is generated by synthesizing the four thumbnail element images generated in step 6. When the composition in step S21 is completed, the process proceeds to step S22, and the moving image thumbnail image (video signal representing the moving image thumbnail image) obtained by the composition in step S21 is recorded on the memory card 18 while being associated with the captured moving image. Is done. When the process of step S22 is completed, the process of FIG.

  Examples of the first and second synthesis methods that can be employed as the synthesis method in step S21 will be described. In this embodiment, the first to fourth still images are all adopted still images.

[First synthesis method]
First, the first synthesis method will be described. In FIG. 15, a thumbnail image for moving image obtained by the synthesis by the first synthesis method is represented by 80. In FIG. 15, reference numerals 81, 82, 83, and 84 denote first, second, third, and fourth thumbnail element images, respectively. The image size of the moving image thumbnail image 80 is, for example, the same as the image size of the thumbnail TN3 in FIG. For this reason, when the moving image thumbnail image 80 is created, the image size of each thumbnail element image is appropriately reduced through a thinning process or the like.

  When the first composition method is adopted, in the thumbnail image display mode, for example, the moving image thumbnail image 80 is displayed in a portion where the thumbnail TN3 is displayed.

  When five or more still images are taken, four still images are selected as adopted still images, and a total of four thumbnail element images corresponding to each adopted still image are combined to form a thumbnail image for moving images. Is generated. For example, the first four still images or the last four still images are selected as adopted still images.

[Second synthesis method]
Next, the second synthesis method will be described. When the second composition method is employed, the moving image thumbnail image obtained by the composition in step S21 is composed of a plurality of images having the same image size as the thumbnail TN3 in FIG. When the first to fourth still images are taken, the moving image thumbnail images obtained by the composition are composed of first to fourth thumbnail element images each having the same image size as the thumbnail TN3.

  When the first to fourth still images are shot during shooting of the moving image corresponding to the thumbnail TN3, when the thumbnail image display mode is selected, first, the first thumbnail element image is displayed in the portion of the thumbnail TN3. . Then, when a predetermined time (for example, 1 second) elapses, the second thumbnail element image is updated and displayed in the thumbnail TN3 portion. Similarly, every time a predetermined time elapses, the image displayed in the thumbnail TN3 portion is sequentially updated to the third and fourth thumbnail element images, and then the first thumbnail element image is again displayed. Is displayed.

  The update of the display is performed only when the cursor 72 is in the thumbnail TN3 portion, or regardless of the position of the cursor 72.

<< Deformation, etc. >>
The contents described in the above-described embodiments can be applied to other embodiments as long as no contradiction arises.

  The timing at which the moving image thumbnail image is actually generated is arbitrary. A moving image thumbnail image may be generated by receiving an imaging signal from the AFE 12 during moving image shooting, or an image that becomes a base of the moving image thumbnail image at the time of moving image shooting is selected and later (for example, a display unit) The thumbnail image for moving image may be generated using the image (when displayed on the screen 27). In any case, the imaging signals that are the basis for generating the same moving image thumbnail image are the same.

  As described above, the recorded contents of the memory card 18 can be freely read by an external personal computer or the like. As described above, the recorded content of the memory card 18 can be reproduced (video display and audio output) by the display unit 27 and the speaker 28. However, the recorded content of the memory card 18 is reproduced by a personal computer including the display unit and the speaker. It can be similarly realized by a computer or the like.

  Note that the thumbnail generation apparatus mainly includes a thumbnail generation unit 42. It may be considered that the thumbnail generation device includes the CPU 23.

  The moving image thumbnail image may be a moving image (hereinafter referred to as a thumbnail moving image). A thumbnail sound may be created based on an audio signal corresponding to the captured moving image, and the thumbnail sound may be simultaneously output via the speaker 28 when displaying the thumbnail moving image. Even when the thumbnail image is a still image, the thumbnail sound can be output.

1 is an overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is an internal block diagram of the imaging part of FIG. FIG. 2 is an internal block diagram of a video signal processing unit in FIG. 1. It is a figure which shows the example of a display of the display screen of the display part of FIG. 1 in thumbnail image display mode. It is a figure which shows the reproduction | regeneration display screen of a moving image which can be changed from the display screen of FIG. It is a flowchart showing the production | generation procedure of the thumbnail image for moving images based on 1st Example of this invention. It is a flowchart showing the production | generation procedure of the thumbnail image for moving images based on 2nd Example of this invention. It is a figure for demonstrating the 2nd selection method based on 2nd Example of this invention. It is a figure for demonstrating the 2nd selection method based on 2nd Example of this invention. It is a figure for demonstrating the 3rd selection method based on 2nd Example of this invention. It is a flowchart for demonstrating the 3rd selection method based on 2nd Example of this invention. It is a figure for demonstrating the 3rd selection method based on 2nd Example of this invention. It is a figure for demonstrating the 3rd selection method based on 2nd Example of this invention. It is a flowchart showing the production | generation procedure of the thumbnail image for moving images based on 3rd Example of this invention. It is a figure which shows the example of the synthesized thumbnail image for moving images based on 3rd Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Imaging part 12 AFE
13 Video signal processing unit 14 Microphone 15 Audio signal processing unit 26 Operation unit 26a Recording button 26b Shutter button 26c Operation key 41 Video / still image processing unit 42 Thumbnail generation unit

Claims (9)

In a thumbnail generation device that generates a thumbnail image associated with a captured moving image based on an imaging signal corresponding to a subject from an imaging unit provided in an imaging device capable of capturing a still image during moving image shooting,
An apparatus for generating a thumbnail, comprising: generating a thumbnail image using the image pickup signal at a predetermined timing based on a shooting timing of the still image when a still image is shot during shooting of the moving image. The thumbnail generation apparatus according to claim 1, wherein the thumbnail image is generated based on the imaging signal representing the still image. When there are a plurality of the still images taken during the shooting of the moving image, a selection unit that selects any one of the plurality of still images as an adopted still image,
The thumbnail generation apparatus according to claim 1, wherein the thumbnail image is generated using the imaging signal at a predetermined timing with reference to a shooting timing of the selected adopted still image. 4. The thumbnail generating apparatus according to claim 3, wherein which of the still images is selected as the adopted still image is preset. A predetermined contrast detection area is provided in each of the plurality of still images, and the contrast amount of the image in the contrast detection area for each still image based on the imaging signal representing the captured still image. Contrast detection means for detecting
4. The thumbnail generation apparatus according to claim 3, wherein the selection unit selects the adopted still image based on a comparison between the contrast amounts. A predetermined detection area is provided in each of the plurality of still images, and a plurality of luminance areas or a plurality of areas are detected in the detection area for each still image based on the imaging signal representing the captured still image. Area dividing means for dividing the area into color areas,
The selection unit specifies a luminance region having the maximum size in each detection region or a color region having the maximum size in each detection region for each still image, and the luminance regions having the maximum size are 4. The thumbnail generating apparatus according to claim 3, wherein the adopted still image is selected based on a comparison of sizes of color regions having the maximum size. The imaging device includes sound input means for receiving sound input from the outside,
The thumbnail generation device detects, for each captured still image, the intensity or the magnitude of the sound or the intensity or the magnitude of a frequency component of the predetermined band of the sound corresponding to the shooting timing of the still image. Sound detecting means for
The selection means selects the adopted still image based on a comparison between strengths or magnitudes of the sounds or between strengths or magnitudes of frequency components of a predetermined band of the sounds. 3. A thumbnail generation apparatus according to 3. If there are multiple still images taken during the video recording,
Some or all of the plurality of still images are adopted still images, and there are a plurality of the adopted still images,
For each of the adopted still images, forming a thumbnail element image based on the imaging signal at a predetermined timing based on the photographing timing of the adopted still image,
The thumbnail generation apparatus according to claim 1, wherein the thumbnail image is generated by combining the thumbnail element images. In an imaging device that can capture still images during video recording,
An imaging apparatus comprising the imaging means and thumbnail generation apparatus according to claim 1.