JP7131595B2 - playback equipment and electronic equipment - Google Patents

Description

The present invention relates to playback devices and electronic equipment .

2. Description of the Related Art There is known an imaging apparatus that performs imaging with different exposure times for each imaging area (see Patent Document 1). However, there is no mention of recording of imaging conditions for each imaging area.

JP 2006-197192 A

According to the first aspect of the present invention, a playback device is configured to : first data having image data of a subject, data relating to a first imaging condition set when imaging the first subject in the first imaging area, and the second subject in the second imaging area a control unit for recording second data having data relating to a second imaging condition set at the time of imaging in a recording unit; and recording in the recording unit based on the second data recorded in the recording unit. a reproduction unit that reproduces an image of the image data included in the received first data on a display unit .
According to a second aspect of the invention, an electronic device comprises the playback device described above.

1 is a block diagram showing the configuration of an imaging device according to a first embodiment of the present invention; FIG. 2 is a plan view schematically showing an imaging surface of an imaging element; FIG. 4 is a schematic diagram showing the configuration of an image file according to the embodiment; FIG. 4 is an explanatory diagram of a still image capturing function A; FIG. 4 is a diagram schematically showing the configuration of an image file created when an image is captured using a still image capturing function A; FIG. 4 is an explanatory diagram of a moving image capturing function A; FIG. 4 is a diagram schematically showing the configuration of an image file created when an image is captured using the moving image capturing function A; FIG. 4 is an explanatory diagram of a still image capturing function B; FIG. FIG. 4 is a diagram showing an example of a layout of a large group; FIG. 10 is a diagram schematically showing the configuration of an image file created when an image is captured using a still image capturing function B; 4 is an explanatory diagram of a moving image capturing function B; FIG. 4 is an explanatory diagram of a moving image capturing function B; FIG. FIG. 4 is a diagram schematically showing the configuration of an image file created when an image is captured using a moving image capturing function B; FIG. 4 is an explanatory diagram of a mixed imaging function; FIG. 4 is a diagram schematically showing the configuration of an image file created when an image is captured using the mixed imaging function; FIG. 10 is a diagram schematically showing a directory structure of a memory card according to the second embodiment; FIG. FIG. 10 is a diagram schematically showing the structure of each file according to the second embodiment; FIG. FIG. 10 is a diagram schematically showing the structure of each file according to the second embodiment; FIG. FIG. 11 is an explanatory diagram of Modification 2; FIG. 11 is an explanatory diagram of Modification 3; FIG. 11 is an explanatory diagram of Modification 4; FIG. 12 is an explanatory diagram of Modification 7; 1 is a cross-sectional view of a stacked imaging device; FIG. It is a figure explaining the pixel arrangement|sequence and block of an imaging chip. It is a circuit diagram corresponding to the unit of the imaging chip. FIG. 10 is a diagram schematically showing the structure of an image file in modification 1; FIG. 10 is a diagram schematically showing the structure of the data portion of the image file in Modification 1; FIG. 11 is an explanatory diagram of Modification 6; FIG. 11 is a block diagram illustrating the configuration of a remote image capturing system according to a third embodiment; FIG. FIG. 11 is a diagram schematically showing an example of an image captured in the third embodiment; FIG. FIG. 11 is a diagram schematically showing an example of an image captured in the third embodiment; FIG. FIG. 11 is a diagram schematically showing an example of an image captured in the third embodiment; FIG.

(First embodiment)
First, the stacked imaging device 22 mounted on the electronic device (for example, the imaging device 10) according to the embodiment will be described. The stacked imaging device 22 is described in WO2013/164915 previously filed by the applicant of the present application. FIG. 23 is a cross-sectional view of the stacked imaging device 22. As shown in FIG. The imaging device 22 includes a back-illuminated imaging chip 2111 that outputs pixel signals corresponding to incident light, a signal processing chip 2112 that processes pixel signals, and a memory chip 2113 that stores pixel signals. These imaging chip 2111, signal processing chip 2112, and memory chip 2113 are stacked and electrically connected to each other by a connection portion 2109 having conductivity such as Cu.

Incidentally, as shown in FIG. 23, the incident light is mainly incident in the Z-axis plus direction indicated by the white arrow. As shown in the coordinate axes, the left direction perpendicular to the Z-axis is the positive X-axis direction, and the frontward direction perpendicular to the Z-axis and the X-axis is the positive Y-axis direction. In the following figures, the coordinate axes are displayed with reference to the coordinate axes in FIG. 23 so that the direction of each figure can be understood.

The imaging chip 2111 is, for example, a CMOS image sensor. Specifically, the imaging chip 2111 is a back-illuminated MOS image sensor. The imaging chip 2111 has a microlens layer 2101 , a color filter layer 2102 , a passivation layer 2103 , a semiconductor layer 2106 and a wiring layer 108 . In the imaging chip 2111, the microlens layer 2101, the color filter layer 2102, the passivation layer 2103, the semiconductor layer 2106, and the wiring layer 2108 are arranged in this order in the positive Z-axis direction.

The microlens layer 2101 has a plurality of microlenses L. FIG. The microlens L converges incident light onto a photoelectric conversion unit 2104, which will be described later. The color filter layer 2102 has a plurality of color filters F. The color filter layer 2102 has multiple types of color filters F with different spectral characteristics. Specifically, the color filter layer 2102 includes a first filter (R) having spectral characteristics to transmit mainly red component light, and a second filter (Gb, Gr) having spectral characteristics to mainly transmit green component light. ) and a third filter (B) having a spectral characteristic to transmit mainly blue component light. The color filter layer 102 has, for example, a first filter, a second filter, and a third filter arranged in a Bayer arrangement. The passivation layer 2103 is composed of a nitrogen film or an oxide film and protects the semiconductor layer 2106 .

The semiconductor layer 2106 has a photoelectric conversion portion 2104 and a readout circuit 2105 . The semiconductor layer 2106 has a plurality of photoelectric conversion units 2104 between a first surface 2106a as a light incident surface and a second surface 2106b opposite to the first surface 2106a. In the semiconductor layer 2106, a plurality of photoelectric conversion units 2104 are arranged in the X-axis direction and the Y-axis direction. The photoelectric conversion unit 2104 has a photoelectric conversion function of converting light into charge. Also, the photoelectric conversion unit 2104 accumulates electric charges resulting from the photoelectric conversion signal. The photoelectric conversion unit 2104 is, for example, a photodiode. The semiconductor layer 2106 has the readout circuit 105 on the second surface 2106 b side of the photoelectric conversion unit 2104 . The semiconductor layer 2106 has a plurality of readout circuits 2105 arranged in the X-axis direction and the Y-axis direction. The readout circuit 2105 is composed of a plurality of transistors, reads image data generated by charges photoelectrically converted by the photoelectric conversion unit 2104 , and outputs the image data to the wiring layer 2108 .

The wiring layer 2108 has multiple metal layers. The metal layer is, for example, Al wiring, Cu wiring, or the like. The wiring layer 2108 outputs the image data read by the reading circuit 2105 . Image data is output from the wiring layer 2108 to the signal processing chip 2112 via the connection section 2109 .

Note that the connection unit 2109 may be provided for each photoelectric conversion unit 2104 . Also, the connection unit 2109 may be provided for each of the plurality of photoelectric conversion units 2104 . When the connecting portions 2109 are provided for each of the plurality of photoelectric conversion portions 2104 , the pitch of the connecting portions 2109 may be larger than the pitch of the photoelectric conversion portions 2104 . Also, the connection portion 2109 may be provided in a peripheral region of the region where the photoelectric conversion portion 2104 is arranged.

The signal processing chip 2112 has a plurality of signal processing circuits. The signal processing circuit performs signal processing on image data output from the imaging chip 2111 . The signal processing circuit includes, for example, an amplifier circuit that amplifies the signal value of image data, a correlated double sampling circuit that performs noise reduction processing of image data, and an analog/digital (A/D) converter that converts analog signals into digital signals. circuit and the like. A signal processing circuit may be provided for each photoelectric conversion unit 2104 .

Also, a signal processing circuit may be provided for each of the plurality of photoelectric conversion units 2104 . A signal processing chip 2112 has a plurality of through electrodes 2110 . The through electrode 2110 is, for example, a through silicon electrode. The through electrodes 2110 connect circuits provided in the signal processing chip 2112 to each other. The through electrodes 2110 may also be provided in the peripheral region of the imaging chip 2111 and the memory chip 2113 . Note that some of the elements that constitute the signal processing circuit may be provided in the imaging chip 2111 . For example, in the case of an analog/digital conversion circuit, the imaging chip 2111 may be provided with a comparator for comparing the input voltage and the reference voltage, and the signal processing chip 2112 may be provided with circuits such as a counter circuit and a latch circuit.

The memory chip 2113 has multiple storage units. The storage unit stores image data subjected to signal processing by the signal processing chip 2112 . The storage unit is, for example, volatile memory such as DRAM. A storage unit may be provided for each photoelectric conversion unit 2104 . Also, a storage unit may be provided for each of the plurality of photoelectric conversion units 2104 . The image data stored in the storage unit is output to the subsequent image processing unit.

FIG. 24 is a diagram for explaining the pixel array of the imaging chip 2111 and the unit area 2131. FIG. In particular, it shows how the imaging chip 2111 is observed from the rear surface (imaging surface) side. For example, 20 million or more pixels are arranged in a matrix in the pixel area. In the example of FIG. 24, four adjacent pixels of 2 pixels×2 pixels form one unit area 2131 . The grid lines in the figure show the concept of grouping adjacent pixels to form a unit area 2131 . The number of pixels forming the unit area 2131 is not limited to this, and may be about 1000, for example, 32 pixels×32 pixels, may be more or less, or may be 1 pixel.

As shown in the partially enlarged view of the pixel area, the unit area 2131 in FIG. 24 includes a so-called Bayer array consisting of four pixels, green pixels Gb and Gr, blue pixels B and red pixels R. FIG. The green pixels Gb and Gr are pixels having a green filter as the color filter F, and receive light in the green wavelength band among incident light. Similarly, the blue pixel B is a pixel that has a blue filter as the color filter F and receives light in the blue wavelength band, and the red pixel R is a pixel that has a red filter as the color filter F and receives light in the red wavelength band. receive light.

In this embodiment, a plurality of blocks are defined so as to include at least one unit area 2131 per block. That is, the minimum unit of one block is one unit area 2131 . As described above, among the values that can be taken as the number of pixels forming one unit area 2131, the smallest number of pixels is one pixel. Therefore, when one block is defined in units of pixels, the minimum number of pixels that can define one block is one pixel. Each block can control the pixels contained in each block with different control parameters. In each block, all unit areas 2131 within the block, that is, all pixels within the block are controlled under the same imaging conditions. That is, it is possible to acquire photoelectric conversion signals with different imaging conditions for a pixel group included in a certain block and a pixel group included in another block. Examples of control parameters include frame rate, gain, thinning rate, number of addition rows or columns for adding photoelectric conversion signals, charge accumulation time or number of accumulations, digitization bit number (word length), and the like. The imaging device 22 can freely perform thinning in the column direction (Y-axis direction of the imaging chip 2111) as well as in the row direction (X-axis direction of the imaging chip 2111). Furthermore, the control parameters may be parameters in image processing.

FIG. 25 is a diagram illustrating a circuit in the unit area 2131. FIG. In the example of FIG. 25, one unit area 2131 is formed by 4 pixels of adjacent 2 pixels×2 pixels. As described above, the number of pixels included in the unit area 2131 is not limited to this, and may be 1000 pixels or more, or a minimum of 1 pixel. The two-dimensional positions of the unit area 2131 are indicated by symbols A to D.

The reset transistors (RST) of the pixels included in the unit area 2131 are configured to be turned on and off individually for each pixel. In FIG. 25, a reset wiring 2300 for turning on/off the reset transistor of pixel A is provided, and a reset wiring 2310 for turning on/off the reset transistor of pixel B is provided separately from the reset wiring 2300 . Similarly, a reset wiring 2320 for turning on/off the reset transistor of the pixel C is provided separately from the reset wirings 2300 and 2310 . A dedicated reset wiring 2330 for turning on/off the reset transistor is provided for the other pixels D as well.

The transfer transistors (TX) of the pixels included in the unit area 2131 are also configured to be turned on and off individually for each pixel. In FIG. 25, a transfer wiring 2302 for turning on/off the pixel A transfer transistor, a transfer wiring 2312 for turning on/off the pixel B transfer transistor, and a transfer wiring 2322 for turning on/off the pixel C transfer transistor are separately provided. A dedicated transfer wiring 2332 for turning on/off the transfer transistor is also provided for the other pixels D. FIG.

Furthermore, the selection transistors (SEL) of the pixels included in the unit area 2131 are configured to be turned on and off individually for each pixel. In FIG. 25, a selection wiring 2306 for turning on/off the selection transistor of pixel A, a selection wiring 2316 for turning on/off the selection transistor for pixel B, and a selection wiring 2326 for turning on/off the selection transistor for pixel C are separately provided. A dedicated selection wiring 2336 for turning on/off the selection transistor is provided for the other pixels D as well.

Note that the power wiring 2304 is commonly connected to the pixels A to D included in the unit area 2131 . Similarly, the output wiring 2308 is commonly connected to the pixels A to D included in the unit area 2131 . Also, the power supply wiring 2304 is commonly connected among the plurality of unit regions, but the output wiring 2308 is individually provided for each unit region 2131 . A load current source 2309 supplies current to the output wiring 2308 . The load current source 2309 may be provided on the imaging chip 2111 side or may be provided on the signal processing chip 2112 side.

By individually turning on and off the reset transistor and the transfer transistor of the unit area 2131, the charge accumulation including the charge accumulation start time, the accumulation end time, and the transfer timing is controlled for the pixels A to D included in the unit area 2131. can do. In addition, by individually turning on and off the selection transistors of the unit area 2131 , the photoelectric conversion signals of the pixels A to D can be output through the common output wiring 2308 .

Here, a so-called rolling shutter system is known, which controls charge accumulation in a regular order for rows and columns for pixels A to D included in the unit area 2131 . If a column is specified after pixels are selected for each row by the rolling shutter method, photoelectric conversion signals are output in the order of "ABCD" in the example of FIG.

By configuring the circuit based on the unit area 2131 in this way, the charge accumulation time can be controlled for each unit area 2131 . In other words, it is possible to output photoelectric conversion signals with different frame rates between the unit areas 2131 . In addition, while the unit regions 2131 included in some blocks of the imaging chip 2111 are allowed to perform charge accumulation (imaging), the unit regions 2131 included in other blocks are allowed to rest, so that a predetermined block of the imaging chip 2111 It is possible to pick up an image only and output the photoelectric conversion signal. Furthermore, it is also possible to switch blocks (target blocks for accumulation control) in which charge accumulation (imaging) is performed between frames, to sequentially perform imaging in different blocks of the imaging chip 2111, and to output photoelectric conversion signals.

As described above, the output wiring 2308 is provided corresponding to each unit area 2131 . Since the imaging device 22 has an imaging chip 2111, a signal processing chip 2112, and a memory chip 2113 stacked, by using electrical connection between the chips using the connection portion 2109 for the output wiring 2308, each chip can be arranged in the plane direction. Wiring can be routed without increasing the size.

FIG. 1 is a block diagram showing the configuration of an imaging device according to the first embodiment. The imaging device 10 is a lens-integrated camera. The imaging device 10 includes an imaging optical system 21, an imaging device 22, a control section 23, a liquid crystal monitor 24, a memory card 25, an operation section 26, a DRAM 27, a flash memory 28, and a recording section 29. .

The imaging optical system 21 is composed of a plurality of lenses and forms a subject image on the imaging surface of the imaging device 22 . Note that FIG. 1 illustrates the imaging optical system 21 as one lens.

The imaging device 22 is, for example, an imaging device such as a CMOS or a CCD, and captures a subject image formed by the imaging optical system 21 and outputs an imaging signal. The control unit 23 is an electronic circuit that controls each unit of the imaging device 10, and is composed of a CPU and its peripheral circuits. A predetermined control program is written in advance in the flash memory 28, which is a non-volatile storage medium. The control unit 23 controls each unit by reading a control program from the flash memory 28 and executing it. This control program uses the DRAM 27, which is a volatile storage medium, as a working area.

The liquid crystal monitor 24 is a display device using a liquid crystal panel. The control unit 23 causes the imaging device 22 to repeatedly capture a subject image at predetermined intervals (for example, 1/60th of a second). Then, various image processing is performed on the imaging signal output from the imaging device 22 to create a so-called through image, which is displayed on the liquid crystal monitor 24 . The liquid crystal monitor 24 displays, for example, a setting screen for setting imaging parameters (imaging conditions) in addition to the through image.

The control unit 23 creates an image file, which will be described later, based on the imaging signal output from the imaging device 22, and records the image file in the memory card 25, which is a portable recording medium. The operation unit 26 has various operation members such as push buttons, and outputs operation signals to the control unit 23 in response to the operation of these operation members. The recording unit 29 is composed of, for example, a microphone or the like, converts environmental sounds into audio signals, and inputs the audio signals to the control unit 23 . Instead of recording the image file 40 in the memory card 25, which is a portable recording medium, the image file 40 may be recorded in a hard disk, etc., which is a recording medium (not shown) incorporated in the imaging apparatus 10. FIG.

2A is a plan view schematically showing the imaging surface 30 of the imaging device 22, and FIG. 2B is an enlarged plan view of a partial area 30a of the imaging surface 30. FIG. As shown in FIG. 2B, a large number of imaging pixels 31 are arranged two-dimensionally on the imaging surface 30 . The imaging pixels 31 each have a color filter (not shown). There are three types of color filters: red (R), green (G), and blue (B). It shows the type of color filters that you have. As shown in FIG. 2B, on the imaging surface 30 of the imaging element 22, the imaging pixels 31 having such color filters are arranged according to the so-called Bayer array.

The image pickup pixel 31 having a red filter photoelectrically converts light in the red wavelength band among incident light and outputs a light reception signal (photoelectric conversion signal). Similarly, the image pickup pixel 31 having a green filter photoelectrically converts light in the green wavelength band among incident light and outputs a light receiving signal. Further, the image pickup pixel 31 having a blue filter photoelectrically converts light in the blue wavelength band among incident light and outputs a light receiving signal.

The imaging device 22 of the present embodiment is configured to be individually controllable by the control unit 23 for each unit group 32 composed of a total of four imaging pixels 31 of 2×2 pixels adjacent to each other. For example, when charge accumulation is started simultaneously for two unit groups 32 different from each other, one unit group 32 reads out the charge, that is, reads out the received light signal 1/30 second after the start of charge accumulation, and the other unit group 32 reads out the charge. In group 32, charges can be read out 1/15 second after the start of charge accumulation. In other words, the imaging element 22 can set different exposure times (charge accumulation times, ie, so-called shutter speeds) for each unit group 32 in one imaging.

The imaging element 22 can vary the amplification factor of the imaging signal (so-called ISO sensitivity) for each unit group 32 in addition to the exposure time described above. The imaging element 22 can change the timing of starting charge accumulation and the timing of reading out the received light signal for each unit group 32 . In other words, the imaging device 22 can change the frame rate for each unit group 32 when capturing a moving image.

In summary, the imaging element 22 is configured such that imaging conditions such as exposure time, amplification factor, and frame rate can be varied for each unit group 32 . For example, a readout line (not shown) for reading an imaging signal from a photoelectric conversion unit (not shown) included in the imaging pixel 31 is provided for each unit group 32 so that the imaging signal can be read independently for each unit group 32. Then, the exposure time (shutter speed) can be made different for each unit group 32 . Further, an amplifier circuit (not shown) for amplifying an imaging signal generated by photoelectrically converted charges is provided independently for each unit group 32, and the amplification factor of the amplifier circuit is configured to be independently controllable for each amplifier circuit. For example, the signal amplification factor (ISO sensitivity) can be varied for each unit group 32 .

Note that the number of imaging pixels 31 forming the unit group 32 does not have to be 2×2=4 pixels described above. The unit group 32 may have at least one imaging pixel 31 , or may have more than four imaging pixels 31 . Also, the imaging conditions that can be varied for each unit group 32 may be other than those described above. For example, if the imaging device 22 is provided with a liquid crystal panel having sections that can be controlled independently for each unit group 32 (one section corresponds to one unit group 32) and is used as a neutral density filter that can be turned on and off, the unit It becomes possible to control the brightness (aperture value) for each group 32 .

Next, the image file 40 created by the controller 23 and recorded in the memory card 25 will be described. FIG. 3 is a schematic diagram showing the structure of an image file according to this embodiment. The image file 40 consists of two blocks, a header portion 41 and a data portion 42 .

The header section 41 is a block located at the head of the image file 40, and stores the file basic information section 43, the mask section 44, and the imaging information section 45 in the order described above. In the file basic information section 43, for example, the size and offset of each section (header section 41, data section 42, mask section 44, imaging information section 45, etc.) in the image file 40 are recorded. Imaging condition information, mask information, and the like, which will be described later, are recorded in the mask portion 44 . The imaging information section 45 records information about imaging, such as the model name of the imaging device 10 and information about the imaging optical system 21 (for example, information about optical characteristics such as aberration). The data section 42 is a block positioned after the header section 41, and records image information, audio information, and the like.

Next, the imaging functions of the imaging apparatus 10 and the image file 40 created (recorded) by each imaging function will be described. A user can perform a predetermined operation on an operation member of the operation unit 26 to switch (select) each imaging function described below. The control unit 23 takes an image based on the selected imaging function, creates an image file 40 and records it in the memory card 25 .

(1) Still image capturing function A (single still image)
The still image capturing function A is a function of dividing an image capturing screen into a plurality of partial areas, setting image capturing conditions individually for each of the plurality of partial areas, and capturing a still image.

FIG. 4A schematically shows an imaging screen 50 (imaging range) of the imaging device 22 and a subject 51. As shown in FIG. A procedure for capturing an image of the subject 51 shown in FIG. 4A using the still image capturing function A will be described. The control unit 23 images the subject 51 once before the main imaging. The imaging performed prior to this main imaging is hereinafter referred to as preliminary imaging. Note that the preliminary imaging may also serve as imaging performed for creating a live view image (so-called through image), for example.

The control unit 23 executes a predetermined image analysis process on the image of the subject 51 (the image in which the subject 51 is captured) obtained by preliminary imaging. The image analysis process is a process of detecting a main subject portion and a background portion by, for example, a well-known subject detection technique (a technique of calculating a feature amount and detecting a range in which a predetermined subject exists). By the image analysis processing, the imaging screen 50 is divided into a main subject area 52 in which the main subject portion exists and a background area 53 in which the background portion exists.

Although FIG. 4A shows an area roughly including the subject 51 as the main subject area 52 , the main subject area 52 may have a shape that follows the outline of the subject 51 . That is, the main subject area 52 may be set so as not to include objects other than the subject 51 as much as possible.

The control unit 23 sets different imaging conditions for each unit group 32 within the main subject area 52 and for each unit group 32 within the background area 53 . For example, each former unit group 32 is set to a higher shutter speed than each latter unit group 32 . By doing so, image blurring is less likely to occur in the main subject area 52 in the actual imaging.

Also, when the main subject area 52 is backlit due to the influence of a light source such as the sun existing in the background area 53, the former unit groups 32 may be set to a relatively high ISO sensitivity. , set a slow shutter speed. Also, a relatively low ISO sensitivity or a high shutter speed is set for each of the latter unit groups 32 . By doing so, it is possible to prevent blocked-up shadows in the backlit main subject area 52 and blown-out highlights in the background area 53 with a large amount of light.

Note that the image analysis processing may be processing different from the above-described processing for detecting the main subject portion and the background portion. For example, it may be a process of detecting a portion with a certain brightness or more (too bright portion) or a portion with less than a certain brightness (too dark portion) in the entire imaging screen 50 . When the image analysis process is such a process, the control unit 23 sets the exposure value (Ev value) of the unit group 32 included in the former area to be lower than that of the unit group 32 included in the other area. Set the shutter speed and ISO sensitivity. For the unit groups 32 included in the latter area, the shutter speed and ISO sensitivity are set so that the exposure value (Ev value) is higher than those of the unit groups 32 included in the other areas. By doing so, the dynamic range of the image obtained by the main imaging can be made wider than the original dynamic range of the imaging device 22 .

FIG. 5 is a diagram schematically showing the configuration of an image file 40 created when an image is captured using the still image capturing function A. As shown in FIG. Identification information 60, imaging condition information 61, and mask information 62a are recorded in the mask portion 44 in the order described above. The identification information 60 is information indicating that the image file 40 was created by the still image capturing function A. FIG.

The imaging condition information 61 is information representing what kind of application (purpose, role) exists in the unit group 32 . For example, as described above, when the imaging screen 50 (FIG. 4A) is divided into the main subject area 52 and the background area 53, each unit group 32 belongs to the main subject area 52 or the background area 53. either belong to In other words, the unit group 32 has a usage of either “capturing a still image of the main subject portion” or “capturing a still image of the background portion”. The image pickup condition information 61 indicates that the unit group 32 has two types of uses, ie, "capturing a still image of the main subject portion" and "capturing a still image of the background portion" when creating the image file 40, and This is information representing a unique number assigned for each of these uses. For example, the number 1 is assigned to the use of "capturing a still image of the main subject portion" and the number of 2 is assigned to the use of "capturing a still image of the background portion".

The mask information 62a is information representing the use (purpose, role) of each unit group 32 . In the present embodiment, the mask information 62a is "information expressing the numbers assigned to the imaging condition information 61 in the form of a two-dimensional map according to the positions of the unit groups 32". That is, when the two-dimensionally arranged unit groups 32 are specified by two-dimensional coordinates (x, y) defined by two integers x and y, the use of the unit group 32 existing at the position (x, y) is as follows. It is represented by a number present at the (x, y) position of the mask information 62a. For example, if the mask information 62a contains the number "1" at the position of the coordinates (3, 5), the unit group 32 at the coordinates (3, 5) contains the message "Capturing a still image of the main subject portion". ” was given. In other words, it can be seen that the unit group 32 located at coordinates (3, 5) belongs to the main subject area 52 .

An example of the mask information 62a corresponding to the imaging screen 50 shown in FIG. 4(a) is shown in FIG. 4(b). "1" is stored at the position of the unit group 32 belonging to the main subject area 52, and "2" is stored at the position of the unit group 32 belonging to the background area 53, respectively.

The data section 42 stores mask information 62b, image information 64, Tv value map 65, Sv value map 66, Bv value map 67, and Av value information 68 in the order described above. . The mask information 62 b is the same information as the mask information 62 a stored in the mask section 44 . The reason why the same mask information 62a and 62b are stored in both the mask portion 44 and the data portion 42 is that the image file 40 can be easily handled.

Although the details will be described later, in the image file 40 created by another function, the mask portion 44 and the data portion 42 may store different mask information 62a and 62b. In the still image capturing function A, for example, if the mask information 62b is stored in the data section 42 and the mask information 62a is not stored in the mask section 44, the structure of the image file 40 changes for each function. This complicates the handling of the image file 40. Therefore, in this embodiment, the same mask information 62a and 62b is intentionally entered in both the mask portion 44 and the data portion 42, and the structure of the image file 40 for each function is defined. minimizes the difference between Note that one of the mask information 62a and 62b may be omitted, in which case the size of the storage area occupied by the image file 40 can be reduced. Also, even if both of the mask information 62a and 62b are recorded, the identification information indicates whether it is necessary to read both of the mask information 62a and 62b. If it is determined that there is one, the file reading time can be shortened by skipping the reading of one of them.

In the following description, the mask information 62a stored in the mask section 44 and the mask information 62b stored in the data section 42 are collectively referred to as mask information 62. FIG.

The image information 64 is information obtained by recording the imaging signal output from the imaging element 22 by the actual imaging in a form before various image processing, and is so-called RAW image data. The Tv value map 65 is information expressing the Tv value representing the shutter speed set for each unit group 32 in the form of a two-dimensional map according to the position of the unit group 32 . For example, the shutter speed set for the unit group 32 located at the coordinates (x, y) can be determined by checking the Tv value stored at the coordinates (x, y) in the Tv value map 65 .

The Sv value map 66 is information expressing the Sv value representing the ISO sensitivity set for each unit group 32 in the form of a two-dimensional map like the Tv value map 65 . The Bv value map 67 is a two-dimensional map of the subject brightness measured for each unit group 32 at the time of main imaging, that is, the Bv value representing the brightness of subject light incident on each unit group 32, in the same manner as the Tv value map 65. is information expressed in the form of The Av value information 68 is information representing the aperture value at the time of actual imaging. In this embodiment, the Av value is not a value that exists for each unit group 32, unlike the Tv value, Sv value, and Bv value. Therefore, unlike the Tv value, the Sv value, and the Bv value, the Av value stores only a single value, and is not information in which a plurality of values are mapped two-dimensionally.

As described above, the control unit 23 captures an image using the still image capturing function A, so that the image information 64 generated by the image sensor 22 capable of setting the capturing conditions for each unit group 32 and the image information 64 for each unit group 32 An image file 40 associated with data on imaging conditions (imaging condition information 61, mask information 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) is recorded in the memory card 25. FIG. Such an image file storage format is referred to as a collective storage format (time-series type) in this specification.

In the above description, the image information 64 is RAW image data, but it may be compressed (developed) image data instead of RAW image data.

(2) Movie shooting function A (single movie)
The moving image capturing function A is a function of dividing an imaged screen into a plurality of partial areas, setting imaging conditions individually for the plurality of partial areas, and capturing a moving image. The difference from the still image capturing function A is that moving images are captured instead of still images. Since a moving image is captured instead of a still image, there is a possibility that the “use of each unit group 32” described in the still image capturing function A may change for each frame.

FIG. 6A schematically shows an imaging screen 50 (imaging range) of the imaging device 22 and a subject 51. As shown in FIG. The control unit 23 performs preliminary imaging before main imaging. Then, predetermined image analysis processing is performed on the image of the subject 51 (the image in which the subject 51 is captured) obtained by the preliminary imaging. By the image analysis processing, the imaging screen 50 is divided into a main subject area 52 in which the main subject portion exists and a background area 53 in which the background portion exists. The control unit 23 sets different imaging conditions for each unit group 32 in the main subject area 52 and each unit group 32 in the background area 53, performs main imaging for the first frame, and creates image data. do. An example of mask information 62 at this time is shown in FIG. In the mask information 62 shown in FIG. 6B, as an example, the unit group 32 belonging to the main subject area 52 is assigned the number "1", and the unit group 32 belonging to the background area 53 is assigned the number "2". .

Next, the control unit 23 executes image analysis processing on the image data of the first frame to detect the main subject portion and the background portion. As a result, the image data of the first frame is divided into a main subject area 52 and a background area 53, as shown in FIG. 6(c). The control unit 23 sets different imaging conditions for each unit group 32 in the main subject area 52 and each unit group 32 in the background area 53, performs main imaging for the second frame, and creates image data. do. An example of mask information 62 at this time is shown in FIG.

The mask information 62 (FIG. 6B) corresponding to the result of the preliminary imaging and the mask information 62 (FIG. 6D) corresponding to the result of the main imaging of the first frame are imaged at different times. Therefore, when the object 51 is moving or when the user moves the imaging device 10, the contents of these two pieces of mask information 62 are different. In other words, the mask information 62 is dynamic information that changes over time. Therefore, in a certain unit group 32, different imaging conditions are set for the main imaging of the first frame and the main imaging of the second frame.

The control unit 23 records the mask information 62b, the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68 of each frame in the image file 40 together with the image information 64 of each frame. Therefore, after imaging, the information at the time of imaging can be fully acquired from the image file 40, and such information can be effectively used for moving image reproduction.

Note that the processing of the third and subsequent frames at the time of actual imaging is the same as the above-described processing of the second frame, so description thereof will be omitted. The control unit 23 repeatedly executes the above-described processing until the imaging is completed (for example, until a predetermined time elapses or the user performs a predetermined imaging end operation).

FIG. 7 is a diagram schematically showing the structure of an image file 40 created when an image is captured using the moving image capturing function A. As shown in FIG. The difference from the still image capturing function A shown in FIG. 5 will be described in detail below.

The identification information 60 indicates that the image file 40 was created by the moving image capturing function A. FIG. The imaging condition information 61 is obtained by adding a frame rate to the imaging condition information 61 of the still image imaging function A. FIG. In other words, when the image file 40 is created, the imaging condition information 61 has two uses in the unit group 32, for example, "moving the main subject portion at 60 fps" and "moving the background portion at 30 fps". and the unique number assigned to each of these uses. For example, the number 1 is assigned to the use of "moving the main subject at 60 fps" and the number 2 is assigned to the use of "moving the background at 30 fps".

The mask information 62a is the same information as the still image capturing function A described above. However, in the case of moving image shooting, as described above, the mask information 62 is dynamic information that changes for each frame. In this embodiment, the mask information 62a representing the imaging conditions set for each unit group 32 when imaging the first frame, that is, the mask information 62 illustrated in FIG. . The reason for doing so is to prevent the handling of the image file 40 from becoming complicated, as described in the explanation of the still image capturing function A.

In the data section 42, blocks 70 for one frame are stored in the order of imaging for each frame. One block 70 consists of mask information 62 , image information 64 , Tv value map 65 , Sv value map 66 , Bv value map 67 and Av value information 68 . The data section 42 stores audio information 71 together with blocks 70 for each frame. The audio information 71 is divided for each frame of information, multiplexed with the block 70, and stored in the data section 42 so as to facilitate reproduction of moving images. Note that the multiplexing of the audio information 71 may be performed for each predetermined number of frames instead of one frame. Each piece of information in the block 70 is the same as in the case of the still image capturing function A, except that it is recorded for each frame, so description thereof will be omitted.

Note that the image file 40 is not limited to recording the mask information 62b of each frame together with the image information of each frame. For example, when the same imaging scene continues, as in the case of video imaging by a surveillance camera, the mask information 62 of the initial frame is used until the imaging scene changes, that is, the mask information 62 changes. In this case, the control unit 23 records the mask information 62a set at the time of imaging the first frame in the header portion 41 . If the mask information 62 set for the second frame imaging is the same as the mask information 62a for the first frame, the control section 23 sets the block 70 for the second frame in the data section 42 to mask mask information 62a. The information 62b is not recorded, and information indicating that there is no mask information 62b (mask presence/absence information) is added. For example, when the number "1" is given as the mask presence/absence information, it indicates that the mask information 62b is recorded in the data section 42, and when the number "0" is given, the data section 42 indicates that the mask information 62b is not given. That is, "0" is given as the mask presence/absence information to the block 70 of the second frame.

If the imaged scene of the subsequent frames is the same as that of the first frame, the mask information 62b is not recorded in each block 70 of the data section 42, and "0" is set as the mask presence/absence information. is given. When the imaged scene changes from the scene imaged in the first frame, the control unit 23 assigns "1" as the mask presence/absence information to the block 70 of the frame in the data unit 42, and the scene is changed. The mask information 62b set according to the change is recorded. As a result, when the same imaging scene continues, it is possible to reduce the size of the storage area by reducing the mask information 62b of the data section 42. FIG.

Furthermore, if the imaging scene is the same as the immediately preceding frame, the immediately preceding mask information 62 may be used. In this case, the control unit 23 records the mask information 62a set at the time of imaging the first frame in the header portion 41 . If the mask information 62 set for the second frame imaging is the same as the mask information 62a for the first frame, the control section 23 sets the block 70 for the second frame in the data section 42 to mask mask information 62a. The information 62b is not recorded, and information indicating that the mask information 62b is the same as the mask information 62a of the immediately preceding frame (mask same information) is added. For example, if the number "1" is assigned as the mask same information, it indicates that the mask information 62b is recorded in the data section 42 because the imaging scene is different from the previous frame. If the number "0" is given, it indicates that the mask information 62b is not given to the data section 42 because the imaging scene is the same as the immediately preceding frame. That is, "0" is assigned to the block 70 of the second frame as mask identity information.

When the imaging scene of the third frame is different from the imaging scene of the second frame, the control unit 23 sets the mask information 62 of the third frame, and stores the block 70 of the third frame in the data unit 42. , "1" is assigned as mask identity information, and mask information 62b is recorded. In subsequent frames as well, when an imaged scene different from the imaged scene of the immediately preceding frame is obtained, the control unit 23 assigns "1" as mask identity information to the block 70 of the frame in the data unit 42, and The mask information 62b is recorded, and when the same imaging scene is obtained, "0" is given as mask same information and the mask information 62b is not recorded. As a result, when the same imaging scene continues, it is possible to reduce the size of the storage area by reducing the mask information 62b of the data section 42. FIG.

As described above, the control unit 23 captures images using the moving image capturing function A, thereby obtaining the image information 64 generated by the image sensor 22 capable of setting the capturing conditions for each unit group 32, and the captured images for each unit group 32. An image file 40 associated with data relating to conditions (imaging condition information 61, mask information 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) is recorded in the memory card 25. FIG. Such an image file storage format is referred to as a collective storage format (time-series type) in this specification.

(3) Still image capturing function B (multiple still images)
The still image capturing function B is a function of simultaneously capturing a plurality of still images of the same subject under different capturing conditions in one capturing.

FIG. 8A schematically shows the imaging surface 30 of the imaging element 22. As shown in FIG. Further, FIG. 8B shows a schematic diagram in which a partial region 30b of the imaging surface 30 is enlarged. In the still image capturing function B, the plurality of unit groups 32 arranged two-dimensionally are further classified into a plurality of large groups 81 . At this time, the unit groups 32 are classified so that the unit groups 32 belonging to a certain large group 81 are uniformly arranged on the entire imaging surface 80 . For example, in FIG. 8B, all the unit groups 32 are divided into blocks 82 consisting of four 2×2 unit groups 32, and the upper left unit group 32 of each block 82 is divided into the first large group 811. , the lower left unit group 32 into the second large group 812, the upper right unit group 32 into the third large group 813, and the lower right unit group 32 into the fourth large group 814. . In FIG. 8(b), one schematically illustrated square represents one unit group 32, and the number written in the square represents the type of large group 81 to which the unit group 32 belongs. .

During main imaging, the control unit 23 controls the unit groups 32 belonging to the first large group 811, the unit groups 32 belonging to the second large group 812, and the unit groups 32 belonging to the third large group 813. , and the unit groups 32 belonging to the fourth large group 814 have different imaging conditions. For example, the main imaging is performed by setting the shutter speed and the ISO sensitivity to mutually different values. The control unit 23 records the image information thus captured in the image file 40 . The image information recorded here is intended to be used by aggregating each pixel value for each large group 81, as schematically shown in FIG. 8(c).

For example, as shown in FIG. 8C, if only the pixel values corresponding to the unit groups 32 belonging to the first large group 811 are extracted from the image information 64 and arranged two-dimensionally, the pixels of the image sensor 22 are A first image information 641 consisting of 1/4 of the number of pixel values is obtained. Similarly, if only the pixel values corresponding to the unit groups 32 belonging to the second large group 81 are extracted from the image information 64 and arranged two-dimensionally, then from the pixel values of 1/4 the number of pixels of the image sensor 22, As a result, second image information 642 is obtained in which the same subject 51 as in the first image information 641 is imaged under different imaging conditions from the first image information 641 described above. Similarly, third image information 643 and fourth image information 644 are obtained. These four pieces of image information 641, 642, 643, and 644 are all images of the same subject 51 captured under different imaging conditions. That is, as described at the beginning, it can be said that four still images of the same subject 51 under different imaging conditions are simultaneously captured in one imaging.

The image information 64 in the image file 40 is an image obtained by arranging the pixel outputs from the imaging pixels 31 of the imaging device 22 according to the positions of the imaging pixels 31 . In other words, the process of creating the four pieces of image information 641, 642, 643, and 644 described above is performed when the image file 40 is read out from the memory card 25 for reproduction or development. Moreover, the image information 64 is not necessarily used only for creating the four pieces of image information 641, 642, 643, and 644. FIG. If the image information 64 is used (reproduced, etc.) as it is, the imaging conditions are different for each of the adjacent unit groups 32, resulting in an unnatural image with, for example, a checkerboard pattern. However, since the image file 40 records the imaging conditions (for example, Tv value, Sv value, etc.) for each unit group 32, if these imaging conditions are combined with the image information 64 for development, such It is possible to prevent the image from becoming unnatural. For example, a unit group 32 having a higher exposure value (Ev value) than other unit groups 32 may be developed with a lower brightness than the other unit groups 32 .

An example in which the unit group 32 is classified into four large groups 811, 812, 813, and 814 has been described above. Images may be taken at the same time. Also, the layout of the large groups 81 (method of classifying the unit groups 32) is not limited to classifying the 2×2 unit groups 32 into different large groups 81 one by one.

Examples of this point are shown in FIGS. In FIG. 9(a), all the unit groups 32 are divided into a total of 9 sets of 3×3, and in each set, the 9 unit groups 32 included in the set are divided into 1st to 9th Allocated to large group 81 . By adopting such a layout, nine images 641 to 649 with different imaging conditions can be simultaneously captured by one imaging. Also, in FIG. 9B, all the unit groups 32 are divided into a total of 9 sets of 3×3, and in each set, the upper left corner unit group 32 is assigned to the first large group 81, A total of four unit groups 32 of 2×2 on the lower right are assigned to the second large group 81 . The remaining four unit groups 32 are not used for imaging. In this way, two images 641 and 642 with different imaging conditions can be simultaneously captured in one imaging operation, but the image 642 corresponding to the second large group 81 is the same as that of the first large group. Compared to the image 641 corresponding to 81, the image has four times the number of pixels. In other words, two images 641 and 642 with different imaging conditions can be simultaneously captured in one imaging, and the two images 641 and 642 have different numbers of pixels.

FIG. 10 is a diagram schematically showing the configuration of an image file 40 created when an image is captured using the still image capturing function B. As shown in FIG. The difference from the still image capturing function A shown in FIG. 5 will be described in detail below.

The identification information 60 indicates that the image file 40 was created by the still image capturing function B. FIG. The imaging condition information 61 is information representing what kind of application the unit group 32 has. In the still image capturing function B, each unit group 32 is, for example, used for "constituting first image information 641", for "constituting second image information 642", or for "third image information 642". It has either a usage of "composing the image information 643" or a usage of "composing the fourth image information 644". The imaging condition information 61 is information indicating that the unit group 32 had these four types of uses when the image file 40 was created, and a unique number assigned to each of these uses. For example, numbers 1 to 4 are assigned to uses "composing first to fourth image information 641 to 644", respectively.

The mask information 62a in the still image capturing function B is information representing the use of each unit group 32, as in the case of the still image capturing function A. FIG. That is, the mask information 62a is "information in which the numbers assigned to the imaging condition information 61 are represented in the form of a two-dimensional map according to the positions of the unit groups 32". For example, if the mask information 62a contains the number "1" at the coordinates (3, 5), the unit group 32 at the coordinates (3, 5) belongs to the first large group 811, that is, the first image. It can be seen that it constitutes information 641 .

In this embodiment, the large group 81 having the number "0" is a special large group 81 representing the unit group 32 not used for imaging. That is, the unit group 32 to which the number "0" is assigned in the mask information 62a is not used for imaging (the imaging signal is not read out during actual imaging), and is not recorded in the data section 42. This means that the image information 64 does not include information about the unit group 32 (or invalid dummy information is recorded as information about the unit group 32).

For example, if it is sufficient to perform imaging under three imaging conditions at the same time, and there is no need to perform imaging under four imaging conditions, "4" is written in the unit group 32 shown in FIG. 8(b). The number "0" may be assigned to the mask information 62a of the unit group 32 that has been obtained.

The configuration of the data section 42 is the same as that of the still image capturing function A. FIG. That is, the data section 42 stores mask information 62b, image information 64, Tv value map 65, Sv value map 66, Bv value map 67, and Av value information 68. FIG. The mask information 62b is the same information as the mask information 62a stored in the mask section 44. FIG.

As the mask information 62b, instead of the same information as the mask information 62a of the mask section 44, information indicating validity/invalidity of each of the unit groups 32 may be stored. For example, the unit group 32 that is not used for imaging (the imaging signal is not read out at the time of imaging) has a numerical value of “0”, and the unit group 32 that is used for imaging (the imaging signal has been read out at the time of imaging) has a numerical value of “0”. A map obtained by assigning "1" and arranging it two-dimensionally according to the position of the unit group 32 may be stored in the data section 42 as the mask information 62b. The same applies to the moving image capturing function B and the mixed capturing function, which will be described later.

As described above, the control unit 23 captures an image using the still image capturing function B, so that the image information 64 generated by the image sensor 22 capable of setting the capturing conditions for each unit group 32 and the image information 64 for each unit group 32 An image file 40 associated with data on imaging conditions (imaging condition information 61, mask information 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) is recorded in the memory card 25. FIG. Such an image file saving format is called a batch saving format (image set type) in this specification.

(4) Moving image capturing function B (multiple moving images)
The moving image capturing function B is a function of simultaneously capturing moving images of the same subject under different imaging conditions in a single image capturing. The difference from the still image capturing function B is that moving images are captured instead of still images. Although moving images are captured, the unit groups 32 classified into a certain large group 81 are not classified into different large groups 81 for each frame as in the case of the moving image capturing function A. However, depending on the frame rate setting, a unit group 32 included in one frame (valid in one frame) may not be included in another frame (invalid in another frame). be. The moving image capturing function B will be described below for each frame rate setting.

(4-1) When the frame rate is the same for all the large groups 81 FIG. 11 is an explanatory diagram of the moving image capturing function B when the frame rate is the same for all the large groups 81 . In this case, the imaging conditions that differ for each large group 81 are imaging conditions other than the frame rate (for example, shutter speed, ISO sensitivity, etc.). Even if the exposure times are different, the frame rate, that is, the signal readout cycle is the same. Therefore, in all the large groups 81, the imaging signal readout is performed at a predetermined cycle T1 corresponding to the frame rate.

Since imaging is performed at the same frame rate in all unit groups 32, all unit groups 32 are used for imaging in all frames. In other words, the imaging signals are read out from all the unit groups 32 in all frames, and the imaging signals read out from all the unit groups 32 are included in the image information 64 of all frames. For example, the first image information 64 is created at time t1 after a predetermined cycle T1 from imaging start time t0. The image information 64 includes an image of the first frame (the frame labeled as #1 in FIG. 11; the same shall apply hereinafter) in the first large group 81, an image of the first frame in the second large group 81, The image of the first frame in the second large group 81 and the image of the first frame in the fourth large group 81 are included. The same applies to the second and subsequent image information 64 .

(4-2) Case where the frame rate is not unified for each large group 81 FIG. 12 is an explanatory diagram of the moving image capturing function B when different frame rates are set for all the large groups 81 . In this example, a frame rate of 60 fps is set for the first large group 811, 50 fps for the second large group 812, 24 fps for the third large group 813, and 25 fps for the fourth large group 814. ing.

In this way, when the frame rates of the large groups 81 are different, the control section 23 records each frame based on the fastest frame rate. That is, the image information 64 is recorded every predetermined period T2 (about 16.7 milliseconds) corresponding to 60 fps. For example, at time t11 after a predetermined cycle T2 from imaging start time t0, image information 64 based on the imaging signal read from the unit group 32 belonging to the first large group 811 is created and recorded in the image file 40. be done. At the time t11, the other large groups 812, 813, and 814 have not yet read the imaging signals of the first frame, so the image information 64 does not include those imaging signals. Note that in FIG. 12 , the symbol “X” indicates that the imaging signal is not read out from a specific unit group 32 and the imaging signal is not included in the image information 64 .

At time t12 after a predetermined cycle T2 from time t11, not only the second (second frame) main imaging of the first large group 811 but also the first (first frame) imaging of the second large group 812 (50 fps) is performed. ) has also been completed. Therefore, the control unit 23 records the imaging signals read from the unit groups 32 belonging to the first large group 811 and the unit groups 32 belonging to the second large group 812 in the image file 40 . The imaging signals are not read out from the unit groups 32 belonging to the third large group 813 or the unit groups 32 belonging to the fourth large group 814 and are not recorded in the image file 40 either.

In this way, when the frame rates are different between the large groups 81, part of the image information 64 may be missing (invalid). The control unit 23 indicates that the imaging signal corresponding to the specific unit group 32 is not included in the image information 64 by the mask information 62b recorded for each frame. A specific configuration of the mask information 62b will be described later.

FIG. 13 is a diagram schematically showing the configuration of an image file 40 created when an image is captured using the moving image capturing function B. As shown in FIG. Differences from the moving image pickup function A shown in FIG. 7 and the still image pickup function B shown in FIG. 10 will be described in detail below.

The identification information 60 indicates that the image file 40 was created by the moving image capturing function B. FIG. The imaging condition information 61 is information representing what kind of application the unit group 32 has. The imaging condition information 61 for the moving image imaging function B is obtained by adding a frame rate to the imaging condition information 61 for the still image imaging function B. FIG. In other words, when the image file 40 is created, the imaging condition information 61 is stored in the unit group 32, for example, "constituting first image information 641 that is a moving image of 60 fps", "second image information 642 that is a moving image of 50 fps". , ``configuring third image information 643 that is a moving image of 24 fps'', and ``configuring fourth image information 644 that is a moving image of 25 fps'', and This is information representing a unique number assigned for each of these uses. For example, numbers 1 to 4 are assigned to uses "composing first to fourth image information 641 to 644", respectively.

The mask information 62a in the moving image capturing function B is information representing the usage of each unit group 32, as in the case of the still image capturing function B. FIG. That is, the mask information 62a is "information in which the numbers assigned to the imaging condition information 61 are represented in the form of a two-dimensional map according to the positions of the unit groups 32". For example, if the mask information 62a contains the number "1" at the coordinates (3, 5), the unit group 32 at the coordinates (3, 5) belongs to the first large group 811, that is, the first image. It can be seen that it constitutes information 641 .

The configuration of the data section 42 is the same as that of the moving image capturing function A. FIG. That is, in the data section 42, blocks 70 for one frame are stored in the order of imaging for each frame. One block 70 consists of mask information 62 b , image information 64 , Tv value map 65 , Sv value map 66 , Bv value map 67 and Av value information 68 . The data section 42 stores audio information 71 together with blocks 70 for each frame.

The mask information 62b may store not only the numbers (for example, numbers 1 to 4) specified by the imaging condition information 61 described above, but also the number "0". The number "0" indicates that the unit group 32 is not used for imaging (the imaging signal is not read during imaging) in the corresponding frame. As described above, when capturing a plurality of moving images with different frame rates, the image information 64 of a certain frame may not store an image signal corresponding to a specific unit group 32 . In such a case, the control unit 23 sets the numerical value of the mask information 62 corresponding to the unit group 32 to "0" in the frame. Here, regarding the unit group 32 in which the numerical value of the mask information 62b is set to "0", information other than the image information 64, that is, the Tv value in the Tv value map 65, the Sv value in the Sv value map 66, the Bv No valid value is recorded for the Sv value in the value map 67 either.

For the unit group 32 in which the mask information 62b is set to "0", the imaging signal in the previous frame of the unit group 32 may be recorded in the image information 64. FIG. Similarly, the Tv value in the Tv value map 65, the Sv value in the Sv value map 66, and the Sv value in the Bv value map 67 may be recorded in the previous frame. The same applies to the mixed imaging function, which will be described later.

Also, even if the frame rates are different between the large groups 81 as in (4-2) described above, it is not necessary to record the mask information 62b in the data section 42 . For example, a frame rate of 60 fps is set for the first large group 811, 50 fps for the second large group 812, 24 fps for the third large group 813, and 25 fps for the fourth large group 814. Even in this case, imaging signals are read out from the unit groups 32 constituting all the large groups 81 at a predetermined cycle. That is, a frame in which none of the unit groups 32 stores "0" as the mask information 62 appears with regularity at predetermined intervals. The control section 23 (recording control section, generation section) generates information indicating this regularity and records it in the header section 41 , but does not record the mask information 62 b in the data section 42 . In this case, as the information indicating the regularity, the control unit 23, taking the case shown in FIG. Information indicating that 811 and 812 are valid, large groups 813 and 814 are invalid, and that the large groups 811 to 814 are valid for the third frame may be recorded in the header portion 41 . By recording the information having regularity, it becomes unnecessary to record the mask information 62b for each frame, and the size of the recording area occupied by the image file 40 can be reduced.

As described above, the control unit 23 captures images using the moving image capturing function B, so that the image information 64 generated by the image sensor 22 capable of setting the capturing conditions for each unit group 32 and the captured images for each unit group 32 An image file 40 associated with data relating to conditions (imaging condition information 61, mask information 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) is recorded in the memory card 25. FIG. Such an image file saving format is called a batch saving format (image set type) in this specification.

(5) Mixed imaging function (video and still image)
The mixed imaging function is a function that combines the still image imaging function B and the moving image imaging function B, and is a function that simultaneously captures a still image and a moving image of the same subject under different imaging conditions in one imaging operation.

In the mixed imaging function, as in the still image imaging function B and the moving image imaging function B, the control unit 23 further classifies a plurality of unit groups 32 arranged two-dimensionally into a plurality of large groups 81 . The control unit 23 performs moving image capturing in the same manner as the moving image capturing function B for some of the large groups 81 . The control unit 23 uses the remaining large group 81 to perform still image capturing in the same manner as the still image capturing function B during the moving image capturing. This still image capturing may be performed, for example, repeatedly at regular intervals (automatic capturing), or may be performed in response to a specific operation performed by the user (manual capturing).

FIG. 14 is an explanatory diagram of the mixed imaging function. Here, four large groups 811 to 814 are assumed, of which the first large group 811 captures a moving image of 60 fps, the second large group 812 captures a moving image of 50 fps, the third and fourth large groups 813, A still image is taken at 814 .

The control unit 23 records each frame based on the fastest frame rate (for example, 60 fps), similarly to the moving image capturing function B. While the still image is not captured, the readout of the captured signal is not performed for the unit groups 32 belonging to the third and fourth large groups 813 and 814 . That is, the image information 64 recorded for each frame does not include the imaging signals of the unit groups 32 belonging to the third and fourth large groups 813 and 814 corresponding to still images. When still image capturing is performed, the control unit 23 reads out the image signal from the unit groups 32 belonging to the third and fourth large groups 813 and 814 at the timing when the still image capturing is completed, and the next frame is included in the image information 64 corresponding to .

FIG. 15 is a diagram schematically showing the configuration of an image file 40 created when an image is captured using the mixed imaging function. Differences from the moving image capturing function B shown in FIG. 13 will be described in detail below.

The identification information 60 indicates that this image file 40 was created by the mixed imaging function. The imaging condition information 61 is information representing what kind of application the unit group 32 has. For example, when the image file 40 is created, the imaging condition information 61 in the mixed imaging function is set to the unit group 32 such that "first image information 641 that is a 60 fps moving image" and "second image information that is a 30 fps moving image" are added to the unit group 32. that there were four types of uses: "composing image information 642", "composing third image information 643 that is a still image", and "composing fourth image information 644 that is a still image", and , is information representing a unique number assigned to each of these uses. For example, numbers 1 to 4 are assigned to uses "composing first to fourth image information 641 to 644", respectively.

The mask information 62a in the mixed imaging function is information representing the use of each unit group 32, as in the case of the moving image imaging function B. FIG. That is, the mask information 62a is "information in which the numbers assigned to the imaging condition information 61 are represented in the form of a two-dimensional map according to the positions of the unit groups 32". For example, if the mask information 62a contains the number "1" at the coordinates (3, 5), the unit group 32 at the coordinates (3, 5) belongs to the first large group 811, that is, the first image. It can be seen that it constitutes information 641 .

In the mixed imaging function, an index portion 73 is added to the header portion 41 . The index portion 73 records index information 74 indicating which block 70 stores a still image among the plurality of blocks 70 (each corresponding to a plurality of frames) recorded in the data portion 42 . there is The index information 74 includes, for example, one or more pieces of information such as "a still image is included in the third image information 643 included in the image information 64 of the fifth frame." only) included. The index section 73 is provided so that a still image can be searched quickly from many blocks 70 .

Note that the index information 74 does not have to specify the recording position of the still image based on the number of frames. For example, it is also possible to specify the recording position of the still image based on the playback time of the moving image. In this case, the index information 74 is, for example, information such as "a still image is included in the third image information 643 included in the image information 64 at 3 minutes and 15 seconds".

The control unit 23 additionally writes the frame number and the time at which the still image was captured as index information 74 in the index unit 73 every time the still image is captured during the imaging by the mixed imaging function. Instead of directly adding to the index portion 73 of the image file 40 in the memory card 25, it is temporarily stored in the DRAM 27 and added to the index portion 73 of the image file 40 in the memory card 25 at the end of the mixed imaging function. Information in the DRAM 27 may be transferred.

The configuration of the data section 42 is the same as that of the moving image capturing function B. FIG. That is, in the data section 42, blocks 70 for one frame are stored in the order of imaging for each frame. One block 70 consists of mask information 62 , image information 64 , Tv value map 65 , Sv value map 66 , Bv value map 67 and Av value information 68 . The data section 42 stores audio information 71 together with blocks 70 for each frame.

As described above, the control unit 23 captures the image information 64 generated by the image sensor 22 capable of setting the imaging conditions for each unit group 32 and the imaging conditions for each unit group 32 by performing imaging using the mixed imaging function. An image file 40 associated with data (imaging condition information 61, mask information 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) is recorded in the memory card 25. Such an image file saving format is called a batch saving format (image set type) in this specification.

Next, image reproduction processing by the control unit 23 will be described. The image reproduction process is a process of creating an image of the subject from the image file 40 recorded in the memory card 25 by the various imaging functions described above. The control unit 23 displays the created image on, for example, the liquid crystal monitor 24 or records it on the memory card 25 as a file different from the image file 40 described above.

The control unit 23 opens the image file 40 (FIGS. 5, 7, 10, 13, and 15) and first reads the file basic information section 43. FIG. As a result, the offsets and sizes of the mask portion 44 and the data portion 42 of the image file 40 are found. Next, the control section 23 reads out the identification information 60 from the mask section 44 of the image file 40 . Thereby, the control unit 23 recognizes which imaging function created the image file 40 . Subsequent processing differs for each imaging function. Therefore, the image reproduction processing will be described below for each of the imaging functions described above.

(1) Still image capturing function A (single still image)
Upon recognizing that the image file 40 is a file created by the still image capturing function A shown in FIG. As a result, the control section 23 can identify which range (which unit group 32) is the main subject portion or the background portion in the entire imaging screen, and the main subject portion and the background portion are included in the image. You can change the production. For example, the main subject portion can be emphasized by subjecting the main subject portion to sharp edge enhancement processing and the background portion to blurring processing.

Next, the control unit 23 reads the image information 64 , the Tv value map 65 , the Sv value map 66 , the Bv value map 67 and the Av value information 68 from the data unit 42 . Based on the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68, the image information 64 is subjected to so-called development processing. When the image information 64 is RAW data, the control unit 23 performs well-known demosaic processing on the image information 64 that does not have color information, for example, to create an image that has color information. Further, the control unit 23 performs image processing such as adjustment of color and brightness, noise reduction, etc., based on the Sv value map 66 and the like. For example, a unit group 32 with a large Sv value (high sensitivity) is more susceptible to noise than other unit groups 32 . Therefore, the control unit 23 increases (strengthens) the strength of noise reduction as the Sv value increases. The control unit 23 displays the image created as described above, for example, on the liquid crystal monitor 24 or records it on the memory card 25 .

As described above, when reproducing the image file 40 created by the still image capturing function A, the control unit 23 puts the The recorded imaging condition information 61 and mask information 62a are read. Since the mask portion 44 is recorded before the data portion 42, it is possible to minimize the seek that occurs during the reproduction process.

As described above, the data section 42 stores the same mask information 62b as the mask information 62a stored in the header section 41 . Therefore, the control section 23 may read the mask information 62b from the data section 42 instead of the mask information 62a.

(2) Movie shooting function A (single movie)
Upon recognizing that the image file 40 is a file created by the moving image capturing function A shown in FIG. The control unit 23 determines which range (which unit group 32) is the main subject portion or the background portion in the entire imaging screen. Next, the control section 23 reads out the imaging condition information 61 from the mask section 44 . Thereby, the control unit 23 recognizes the frame rates of the main subject portion and the background portion. Next, the control unit 23 reads out the image information 64, the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68 in order from the top block 70 of the data unit 42 to form a moving image. Create each frame.

When creating each frame, the control section 23 first reads the mask information 62b from the block 70 . Then, in the frame, it is identified which range (which unit group 32) is the main subject portion or the background portion. After that, as described in the description of the still image capturing function A, the control unit 23 performs different image processing for the main subject portion and the background portion. The control unit 23 displays the moving image composed of the frames created as described above, for example, on the liquid crystal monitor 24 or records it on the memory card 25 .

As described above, when reproducing the image file 40 created by the moving image capturing function A, the control unit 23 reads out the mask information 62b before information such as the image information 64 recorded in the block 70. . Since the mask information 62b is recorded before the image information 64 and the like, it is possible to minimize the seek that occurs during reproduction processing.

Since the mask information 62b of the head block of the data section 42 and the mask information 62a recorded in the mask section 44 are the same information, the control section 23 does not read the mask information 62a from the mask section 44. good.

(3) Still image capturing function B (multiple still images)
Upon recognizing that the image file 40 is the file created by the still image capturing function B shown in FIG. Thereby, the control unit 23 identifies how many kinds of still images are captured at the same time and which unit group 32 constitutes which still image. That is, it identifies how many large groups 81 exist and to which large group 81 each unit group 32 belongs.

Next, the control unit 23 reads the image information 64 , the Tv value map 65 , the Sv value map 66 , the Bv value map 67 and the Av value information 68 from the data unit 42 . Then, the control unit 23 performs a so-called developing process on the image information 64 based on the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68 for each large group 81 to produce a still image. create. Thereby, a plurality of still images (for example, four still images) are created. The control unit 23 displays the image created as described above, for example, on the liquid crystal monitor 24 or records it on the memory card 25 .

As described above, when reproducing the image file 40 created by the still image capturing function B, the control unit 23 puts the The recorded imaging condition information 61 and mask information 62a are read. Since the mask portion 44 is recorded before the data portion 42, it is possible to minimize the seek that occurs during the reproduction process.

As described above, the data section 42 stores the same mask information 62b as the mask information 62a stored in the header section 41 . Therefore, the mask information 62b may be read from the data section 42 instead of the mask information 62a.

(4) Moving image capturing function B (multiple moving images)
Upon recognizing that the image file 40 is the file created by the moving image capturing function B shown in FIG. Thereby, the control unit 23 identifies how many types of moving images are captured at the same time, which unit group 32 constitutes which moving image, and the frame rate of each moving image. That is, it identifies how many large groups 81 exist, which large group 81 each unit group 32 belongs to, and at what frame rate each large group 81 was captured. Next, the control unit 23 sequentially reads out the image information 64, the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68 from the top block 70 of the data unit 42, and constructs each moving image. Create each frame to

When creating each frame, the control section 23 first reads the mask information 62b from the block 70 . Then, the control unit 23 identifies which large group 81 the pixel signal corresponding to is included in the image information 64 of the block 70 . After that, the control unit 23 creates a frame corresponding to each large group 81, but does not create a frame for the large group 81 for which the image information 64 of the block 70 does not contain a pixel signal. The control unit 23 displays the moving image composed of the frames created as described above, for example, on the liquid crystal monitor 24 or records it on the memory card 25 .

As described above, when reproducing the image file 40 created by the moving image capturing function B, the control unit 23 reads the mask information 62a, 62b prior to information such as the image information 64 recorded in the block 70. read out. Since the mask information 62a and 62b are recorded prior to the image information 64 and the like, it is possible to minimize the seek that occurs during reproduction processing.

Since the mask information 62b of the head block of the data section 42 and the mask information 62a recorded in the mask section 44 are the same information, the control section 23 does not read the mask information 62a from the mask section 44. good.

(5) Mixed imaging function (video and still image)
Upon recognizing that the image file 40 is the file created by the mixed imaging function shown in FIG. As a result, the control unit 23 can determine how many kinds of moving images and how many kinds of still images are captured at the same time, which unit group 32 constitutes which still images and moving images, and the frame rate of each moving image. Identify. That is, the control unit 23 determines how many large groups 81 exist, whether each large group 81 is a still image or a moving image, if it is a moving image, how many frame rates each unit group 32 belongs to. Identify what you belong to. Next, the control unit 23 sequentially reads out the image information 64, the Tv value map 65, the Sv value map 66, the Bv value map 67, and the Av value information 68 from the top block 70 of the data unit 42, and constructs each moving image. Create each frame and each still image.

The control unit 23 first reads the mask information 62b from the block 70 when creating each frame of a moving image or a still image. Then, it is identified which large group 81 the pixel signal corresponding to is included in the image information 64 of the block 70 . After that, the control unit 23 creates frames and still images corresponding to each large group 81, and creates frames and still images for the large groups 81 for which pixel signals are not included in the image information 64 of the block 70. do not do. The control unit 23 displays moving images and still images composed of frames created as described above, for example, on the liquid crystal monitor 24 and records them on the memory card 25 .

As described above, when reproducing the image file 40 created by the mixed imaging function, the control unit 23 reproduces the mask information 62a and 62b prior to information such as the image information 64 recorded in the block 70. read out. Since the mask information 62a and 62b are recorded prior to the image information 64 and the like, it is possible to minimize the seek that occurs during reproduction processing.

Since the mask information 62b of the head block of the data section 42 and the mask information 62a recorded in the mask section 44 are the same information, the control section 23 does not read the mask information 62a from the mask section 44. good.

The image reproduction process is a process of creating an image of the subject from the image files 40 recorded on the memory card 25 by the various imaging functions described above. Alternatively, it may be a process of creating a still image or moving image. The control unit 23 may perform compression processing after still images or moving images are created.

Note that an electronic device (hereinafter referred to as a playback device) different from the imaging device 10 may execute the playback processing described above. For example, when the memory card 25 is removed from the imaging device 10, and the memory card 25 is inserted into a playback device such as a PC provided outside the imaging device 10, the playback device reads out the image file 40 from the memory card 25 and reproduces the image file 40 as described above. Processing may be performed and images may be reproduced. Alternatively, the image information 64 and the like may be exchanged by performing data communication such as wireless communication between the imaging device 10 and the reproducing device.

According to the imaging device according to the first embodiment described above, the following effects are obtained.
(1) The imaging element 22 has a plurality of unit groups 32 (imaging regions), and imaging conditions can be set for each unit group 32 . The control unit 23 controls image information 64 (image data) generated by the imaging element 22, imaging condition information 61 regarding imaging conditions, mask information 62, Tv value map 65, Sv value map 66, Bv value map 65, and Bv Data (imaging condition data) such as the value map 67 are recorded in association with each other. In this way, it is possible to provide a user-friendly image pickup apparatus 10 that allows the user to know what kind of image pickup conditions are applied to each pixel when reproducing an image file 40, which is the result of image pickup. can be done.

(2) The information about the imaging conditions recorded in association with the image information 64 includes, for example, information about the exposure when the subject is imaged by the image sensor 22 and information about the brightness of the subject imaged by the image sensor 22. included. Specifically, a Bv value map 67 that is information about the brightness of the subject imaged by the imaging device 22, a Tv value map 65 that is the charge accumulation time by a photoelectric conversion unit (not shown), and an amplification factor by an amplification unit (not shown) Sv value map 66 and the like are included. All of these pieces of information can be said to be information relating to the imaging operation of the imaging device 22 . Since this is done, more appropriate image processing can be performed when the image file 40 is reproduced.

(3) The control unit 23 records information about imaging conditions that change each time imaging is performed in association with the image information 64 . Since this is done, appropriate information is added to each image file 40, and more appropriate image processing can be performed at the time of reproduction.

(4) The control unit 23 records a plurality of pieces of information about the imaging conditions corresponding to the image information 64 in the single image file 40 in chronological order. With this arrangement, for example, when moving images are recorded in the image file 40, image processing based on the information can be easily performed.

(5) In the image file 40 having the header portion 41 and the data portion 42 (image data portion) for recording the image information 64, the control portion 23 stores at least one of the header portion 41 and the data portion 42 with: Records information about imaging conditions. Since this is done, when reproducing the image file 40, for example, it is possible to know what imaging conditions are applied to each pixel.

(6) The control unit 23 records the imaging condition information 61 and the mask information 62 regarding the use of each of the plurality of unit groups 32 in association with the image information 64 . Since this is done, when reproducing the image file 40, for example, it is possible to know what imaging conditions are applied to each pixel.

(7) The mask information 62 contains dynamic information that changes over time. Specifically, information indicating whether or not the image information 64 includes pixel values corresponding to pixel signals read from the imaging pixels 31 belonging to the unit group 32, or information representing whether or not each of the plurality of unit groups 32 contains information indicating to which of a plurality of different groups the is classified. Since this is done, image processing using dynamic information can be performed, for example, when the image file 40 is reproduced.

(8) The mask information 62 contains static information that does not change over time. Specifically, it includes information representing roles of the plurality of unit groups 32 . The mask information 62a also includes information indicating to which of a plurality of different groups each of the plurality of unit groups 32 is classified at the beginning of imaging. Since this is done, image processing using static information can be performed, for example, when the image file 40 is reproduced.

(9) The control unit 23 records a plurality of pieces of mask information 62b respectively corresponding to a plurality of pieces of image information 64 in a single image file 40 in chronological order. Since this is done, the imaging conditions can be tracked in chronological order, for example, when the image file 40 is reproduced.

(10) In the image file 40 having the header portion 41 and the data portion 42 (image data portion) for recording the image information 64, the control portion 23 causes at least one of the header portion 41 and the data portion 42 to: The mask information 62 is recorded. Since this is done, when reproducing the image file 40, for example, it is possible to know what imaging conditions are applied to each pixel.

(11) The plurality of unit groups 32 include a unit group 32 imaged at a first frame rate and a unit group 32 imaged at a second frame rate slower than the first frame rate. The unit 23 records a plurality of pieces of image information 64 based on the first frame rate. Since this is done, information about all frames can be reliably recorded without omission.

(12) The control unit 23 records the audio information 71 (audio data) corresponding to the imaging period of the plurality of pieces of image information 64 in association with the plurality of pieces of image information 64 . Since this is done, it is possible to reproduce moving images including sound.

(13) The control unit 23 stores at least one of the information on the imaging pattern of the image information 64, the information on the storage method of the image information 64, and the information on the imaging condition for each unit group 32 as header part 41 and data. It is recorded in the header section 41 of the image file 40 consisting of two blocks, the section 42 and the section 42 . Since this is done, when reproducing the image file 40, for example, it is possible to know what imaging conditions are applied to each pixel.

(Second embodiment)
The imaging device according to the second embodiment has the same configuration as the imaging device 10 according to the first embodiment, but the still image imaging function B, the moving image imaging function B, and the image pickup function in the mixed imaging function The recording method of the file 40 is different from that of the first embodiment. This point will be described in detail below.

As described above, the still image imaging function B, the moving image imaging function B, and the mixed imaging function are functions for capturing a plurality of still images and moving images of the same subject at the same time in one imaging operation. In the present embodiment, the control unit 23 divides and records a plurality of still images and moving images captured in this way into a plurality of image files 40 instead of a single image file 40 . At this time, the control section 23 associates and records the respective image files 40 that have been divided and recorded. Therefore, although the plurality of image files 40 are recorded as separate files for the sake of convenience, the information that they are files obtained by a single imaging is not lost, as in the first embodiment. In other words, when the plurality of image files 40 are handled later, they can be recognized and handled as having been obtained by a single imaging, as in the first embodiment.

FIG. 16 is a diagram schematically showing the directory structure of the memory card 25. As shown in FIG. A root directory 90 of the memory card 25 contains a DCIM directory 91a. Further, in the DCIM directory 91a, there is a subdirectory 91b for saving images. The control unit 23 creates one imaging set directory 92 in the subdirectory 91b each time imaging is performed by the still image imaging function B, the moving image imaging function B, and the mixed imaging function. That is, one imaging set directory 92 corresponds to one imaging.

In the imaging set directory 92 , one management data file 93 and subdirectories 94 are created for each use of the unit group 32 . For example, if there are four usages of the unit group 32, four subdirectories 94 are created. At least one image file 40 corresponding to the purpose of the unit group 32 is created in each subdirectory 94 . For example, if the purpose of the unit group 32 is moving image pickup, only one moving image file 401 is recorded in the subdirectory 94 corresponding to that purpose. Further, when the purpose of the unit group 32 is still image capturing, still image files 402 corresponding to the number of times of capturing are recorded in the subdirectory 94 corresponding to the purpose. In the case of the still image capturing function B, only one still image file 402 is recorded for each application in one image capturing operation. will be

FIG. 17A is a diagram schematically showing the structure of the management data file 93. As shown in FIG. The management data file 93 is a file in which information for associating the image files 40 recorded in each subdirectory 94 is recorded. 45. The file basic information section 43, the mask section 44, and the imaging information section 45 are the same as the sections with the same names in the image file 40 described with reference to FIG. 15 and the like. Layout information 96 is recorded in the index portion 73 to indicate which application of the unit group 32 each subdirectory 94 corresponds to.

FIG. 17B is a diagram schematically showing the structure of a still image file 402 recorded in the subdirectory 94. As shown in FIG. Mask information 62 b , image information 64 , Tv value map 65 , Sv value map 66 , Bv value map 67 , and Av value information 68 are recorded in still image file 402 . The Av value information 68 is the same as that described with reference to FIG. 10, so description thereof will be omitted.

The mask information 62b, the image information 64, the Tv value map 65, the Sv value map 66, and the Bv value map 67 each contain only values corresponding to one large group 81 from the information with the same name described in FIG. is extracted and arranged in two dimensions. For example, in the image file 40 described with reference to FIG. 10, the mask information 62b is "information expressing the numbers assigned to the imaging condition information 61 in the form of a two-dimensional map according to the positions of the unit groups 32". The number of values included in the mask information 62 b was the same as the number of unit groups 32 . On the other hand, the mask information 62b in the still image file 402 is information in which only the values corresponding to the large group 81 corresponding to the main subdirectory 94 are extracted from all these values and expressed in the form of a two-dimensional map. . As for the image information 64 , Tv value map 65 , Sv value map 66 , and Bv value map 67 , one still image file 402 similarly includes only values corresponding to one large group 81 .

FIG. 18 is a diagram schematically showing the structure of the moving image file 401 recorded in the subdirectory 94. As shown in FIG. In the moving image file 401, blocks 70 for one frame are stored in the order of shooting for each frame. One block 70 consists of mask information 62 b , image information 64 , Tv value map 65 , Sv value map 66 , Bv value map 67 and Av value information 68 . In addition, audio information 71 is stored in the moving image file 401 together with blocks 70 for each frame. The Av value information 68 is the same as that described with reference to FIG. 13, so description thereof will be omitted.

The mask information 62b, the image information 64, the Tv value map 65, the Sv value map 66, and the Bv value map 67 each contain only values corresponding to one large group 81 from the information with the same name described in FIG. is extracted and arranged in two dimensions. Since this point is the same as the case of the still image file 402 described above, the description is omitted.

As described above, the control unit 23 controls the image information 64 generated by the image sensor 22 capable of setting the imaging conditions for each unit group 32, and the data regarding the imaging conditions for each unit group 32 (imaging condition information 61, mask information, 62, Tv value map 65, Sv value map 66, Bv value map 67, etc.) are recorded in the memory card 25 in association with each other. Unlike the first embodiment, although it is not in the form of a single image file 40, the management data file 93, moving image file 401, and still image file 402 recorded on the memory card 25 are the management data file 93 are associated with each other by the layout information 96 within. Such an image file saving format is called a division saving format in this specification.

According to the imaging device according to the second embodiment described above, the same effects as those of the first embodiment can be obtained.

(Third Embodiment)
In this embodiment, a remote image capturing system having the image capturing device 10 that generates the image file 40 described in the first and second embodiments will be described. In the remote image capturing system of the present embodiment, a photographer captures images in response to requests from a plurality of clients, and transmits the captured images to the clients. In the following description, an example of a case in which four clients request photographing is given.

FIG. 29 is a block diagram schematically showing the main configuration of a remote imaging system 700 according to the third embodiment. The remote imaging system 700 includes an imaging device 10 , a client A electronic device 711 , a client B electronic device 712 , a client C electronic device 713 , and a client D electronic device 714 . In the following description, the electronic devices 711 to 714 are collectively referred to as an electronic device 710. FIG. The imaging device 10 and the electronic device 710 are connected via a communication network 720 . The communication line network 720 is, for example, a communication line such as an Internet line or a mobile or fixed telephone line.

The electronic device 710 is, for example, a computer, a television, or a portable information terminal such as a smart phone or a tablet capable of transmitting and receiving various information by connecting to the communication network 720 . Also, the electronic device 710 may be a digital camera having a communication function. The electronic devices 711 to 714 of each of the clients A to D capture information (hereinafter referred to as setting information) on various settings such as shooting conditions when taking an image, and shooting instructions via the communication network 720. Send to device 10 . Also, the electronic devices 711 to 714 of the clients A to D receive, via the communication network 720, images captured by the imaging device 10 in accordance with the imaging instructions.

The imaging apparatus 10 is obtained by adding a communication unit 101 for transmitting and receiving information via a communication network 720 to the imaging apparatus 10 of the first and second embodiments described above. Communication unit 101 connects to communication network 720 under the control of control unit 23 . The communication unit 101 is connected to the communication network 720 according to a predetermined protocol, and receives setting information and imaging instructions from the electronic device 710 via the communication network 720 . Communication unit 101 also connects to communication network 720 according to a predetermined protocol, and transmits the generated image to electronic device 710 via communication network 720 .

The control unit 23 of the imaging device 10 includes an imaging control unit 231 and an image processing unit 232 . The imaging control unit 231 sets imaging conditions and controls imaging for each unit group 32 of the imaging device 22 according to the setting information and imaging instructions received by the communication unit 101 . The image processing unit 232 uses at least one storage format from among the various storage formats described in the first and second embodiments to generate an image based on the imaging signal generated by the imaging device 22. Generate file 40 .
Setting of shooting conditions for the image sensor 22 and shooting control will be described below while exemplifying setting information and shooting instructions from clients A to D. FIG.

<Example 1>
FIG. 30 schematically shows a subject to be photographed and a background in Example 1. As shown in FIG. FIG. 30A shows an example in which an airplane in flight is photographed as the subject S1. In this case, the sky and clouds are the background S2. It is assumed that clients A to D have set imaging conditions A1 to D1 as follows for the object to be photographed shown in FIG. 30(a).
(1) Photographing condition A1: A still image is photographed at an arbitrary timing desired by the client A, with the object S1 being properly exposed.
(2) Photographing condition B1: The background S2 is properly exposed, and a still image is photographed at an arbitrary timing desired by the client B. FIG. (3) Photographing condition C1: The subject S1 and the background S2 are properly exposed by high dynamic range (hereinafter referred to as HDR), and a still image is photographed at an arbitrary timing desired by the client C. FIG.
(4) Shooting condition D1: A moving image is shot with the background S2 set to proper exposure.

When the imaging control unit 231 receives the above-described imaging conditions A1 to D1 as setting conditions, as shown in FIG. shooting conditions A1 to D1 are set for . For example, the imaging control unit 231 sets the imaging condition A1 in the upper left unit group 23 of the block 82, sets the imaging condition B1 in the lower left unit group 32, sets the imaging condition C1 in the upper right unit group 32, The photographing condition D1 is set in the unit group 32 on the lower right.

Upon receiving a photographing instruction from the electronic device 710, the imaging control unit 231 causes the imaging device 22 to perform a photographing operation and output an imaging signal. In this case, when a photographing instruction is received from the electronic device 711 of the client A, the imaging control unit 231 causes each block 82 to perform photographing under the photographing condition A1 for the upper left unit group 32, and the photographing signal output. When the image capturing instruction is received from the electronic device 712 of the client B, the image capturing control unit 231 causes each block 82 to perform image capturing under the image capturing condition B1 for the lower left unit group 32 and output an image capturing signal. . When a photographing instruction is received from the electronic device 713 of the client C, the imaging control unit 231 causes each block 82 to perform photographing under the photographing condition C1 for the upper right unit group 32 and output an imaging signal. . When a shooting instruction is received from the electronic device 714 of the client D, the imaging control unit 231 causes each block 82 to perform shooting under the shooting condition D1 for the unit group 32 on the lower right, and a predetermined frame rate. to output the imaging signal.

Images PA to PD shot under shooting conditions A1 to D1 are schematically shown in FIGS. 30(b) to 30(e), respectively. In FIGS. 30(b) to 30(e), the subject S1 and the background S2 photographed with appropriate exposure are indicated by dots, and the subject S1 whose silhouette is photographed due to underexposure is shaded. Give and show.

When the image is captured under the shooting conditions A1, as shown in FIG. 30B, the subject S1 is properly exposed to generate an image PA in which the background S2 such as the sky and clouds is overexposed. When photographing is performed under the photographing condition B1, the background S2 is properly exposed, thereby generating an image PB in which the subject S1 is underexposed and becomes a silhouette, as shown in FIG. 30(c). When photographed under the photographing condition C1, an image PC is generated in which the subject S1 and the background S2 are properly exposed, as shown in FIG. 30(d). When the image is shot under the shooting condition D1, an image PD is generated as a moving image in which the background S2 is properly exposed and the subject S1 is a silhouette, as shown in FIG. 30(e). The above images PA to PD have the same angle of view and composition as images captured using all the pixels on the imaging surface 30 of the imaging element 22, but the resolution is 1/4. becomes an image.

The image processing unit 232 uses the imaging signal output from the imaging device 22 to generate the image file 40 in, for example, the collective storage format or divisional storage format described in the second embodiment. For example, among the image files 40 generated in the divided storage format, the still image file 402 corresponding to the image PA is transmitted to the electronic device 711 of the client A via the communication unit 101 . A still image file 402 corresponding to the image PB among the generated image files 40 is transmitted to the electronic device 712 of the client B via the communication unit 101 . Among the generated image files 40 , the still image file 402 corresponding to the image PC is transmitted to the electronic device 713 of the client C via the communication unit 101 . Of the generated image files 40 , the moving image file 401 corresponding to the image PD is transmitted to the electronic device 714 of the client D via the communication section 101 . In Example 1 described above, image data corresponding to the image PD may be sequentially transmitted to the electronic device 714 of the client D via the communication unit 101 while shooting a moving image.

In the remote image capturing system 700 described in Example 1, in response to different image capturing requests (that is, setting conditions and image capturing instructions) from each of clients A to D, image capturing conditions are set and changed, and image capturing is performed according to each image capturing request. Can start and end. Accordingly, it is possible to use one imaging device 10 to generate different images shot under shooting conditions desired by each of the clients AD.

<Example 2>
In example 2, the control unit 23 controls the timing when the image file 40 is transmitted from the imaging device 10 to the electronic device 710 . A case where there is no moving image shooting instruction from each client A to D and a case where there is a moving image shooting instruction will be described below.

-If there is no video shooting instruction-
As an example of the case where there is no moving image shooting instruction, each of the clients A to D sets shooting conditions A2 to D2 as follows for the shooting target shown in FIG. 30(a).
(1) Photographing condition A2: A still image is photographed at an arbitrary timing desired by the client A, with the object S1 being properly exposed.
(2) Photographing condition B2: The background S2 is properly exposed, and a still image is photographed at an arbitrary timing desired by the client B.
(3) Photographing condition C2: The subject S1 and the background S2 are properly exposed by high dynamic range (hereinafter referred to as HDR), and a still image is photographed at an arbitrary timing desired by the client C. FIG.
(4) Photographing condition D2: Interval still image photographing (time lapse photographing) is performed in which still images are photographed at predetermined time intervals with the background S2 set to an appropriate exposure.

In this example, as in Example 1 above, each time an image capturing instruction is received from the electronic devices 711 to 714 of the clients A to D, the image capturing control unit 231 controls the image sensor 22. An imaging signal is output, and the image processing unit 232 generates an image file 40 . The control unit 23 controls the communication unit 101 to transmit the generated image file 40 to the electronic device 710 that transmitted the shooting instruction.

-When there is an instruction to shoot a video-
When there is an instruction to shoot a moving image, the control unit 23 controls the communication unit 101 to transmit the generated image file 40 to the electronic device 710 at a timing corresponding to the frame rate of the moving image to be generated.
First, as a first example when there is an instruction to shoot a moving image, it is assumed that clients A to D have set shooting conditions A3 to D3 as follows for the shooting target shown in FIG. 30(a). .
(1) Shooting condition A3: A moving image is shot at a frame rate of 120 fps.
(2) Shooting condition B3: A moving image is shot at a frame rate of 60 fps.
(3) Imaging condition C3: A still image of the subject S1 is photographed with proper exposure, low sensitivity, and low shutter speed.
(4) Photographing condition D3: Interval still image photographing (time-lapse photographing) for photographing still images at predetermined time intervals is performed.

In this case, the control unit 23 controls the communication unit 101 to transmit the moving image file 401 corresponding to the moving image with the fastest frame rate (that is, 120 fps under the shooting conditions A3) under the other shooting conditions B3 to D3. The captured and generated moving image file 401 and still image file 402 are transmitted to the electronic device 710 . That is, the communication unit 101 transmits the image file 40 to the electronic device 710 every 1/120 [s], which is the timing at which the one-frame moving image file 401 is generated under the shooting condition A3.

In this case, the one-frame moving image file 401 generated under the shooting condition B3 is generated every 1/60 [s]. to transmit the moving image file 401 under the shooting condition B3. The still image file 402 generated under the shooting condition C3 or D3 is temporarily stored in a storage medium such as the DRAM 27 after generation until the timing when the communication unit 101 transmits the image file 40 . The communication unit 101 transmits these still image files 402 to the electronic device 710 at the timing of transmitting the image file 40 . Note that when the timing at which the still image file 402 is generated coincides with the timing at which the moving image file 401 is generated under the shooting condition A3, the still image file 402 is not stored in the DRAM 27 and is processed by the communication unit 101. It is sent to electronic device 710 .

Next, as a second example when there is an instruction to shoot a moving image, it is assumed that clients A to D set shooting conditions A4 to D4 as follows for the shooting target shown in FIG. 30(a). do.
(1) Shooting condition A4: A moving image is shot at a frame rate of 60 fps.
(2) Shooting condition B4: A moving image is shot at a frame rate of 50 fps.
(3) Imaging condition C5: A still image of the subject S1 is photographed with proper exposure, low sensitivity, and low shutter speed.
(4) Photographing condition D5: Interval still image photographing (time-lapse photographing) is performed in which still images are photographed at predetermined time intervals.

In this case, the control unit 23 controls the communication unit 101 to transmit the moving image file 401 corresponding to the moving image with the fastest frame rate (that is, 60 fps under the shooting condition A4) under the other shooting conditions B3 to D3. The captured and generated moving image file 401 and still image file 402 are transmitted to the electronic device 710 . That is, the communication unit 101 transmits the image file 40 to the electronic device 710 every 1/60 [s], which is the timing at which the one-frame moving image file 401 is generated under the shooting condition A4.

In this case, the one-frame moving image file 401 generated under the shooting condition B4 is generated every 1/50 [s]. After being generated, it is temporarily stored in a storage medium such as the DRAM 27 until the timing when the communication unit 101 transmits the moving image file 401 generated under the shooting condition A4. The communication unit 101 transmits the moving image file 401 under the shooting condition B4 to the electronic device 710 at the timing of transmitting the moving image file 401 under the shooting condition A4. As in the first example, the still image file 402 generated under the shooting condition C4 or D4 is stored until the timing when the communication unit 101 transmits the moving image file 401 under the shooting condition A4 after it is generated. It is temporarily stored in a storage medium such as the DRAM 27 . The communication unit 101 transmits these still image files 402 to the electronic device 710 at the timing of transmitting the moving image file 401 under the shooting condition A4. Note that when the timing at which the still image file 402 is generated coincides with the timing at which the moving image file 401 is generated under the shooting condition A4, the still image file 402 is not stored in the DRAM 27 and is processed by the communication unit 101. It is sent to electronic device 710 . Note that in Example 2 described above, image data may be sequentially transmitted to the electronic device 714 of the client D via the communication unit 101 when moving images are being captured.

<Example 3>
In the remote image capturing system 700, if the image capturing device 22 of the image capturing device 10 is divided into the unit groups 32 according to the number of clients, the resolution of the image generated for each client will be reduced. In the imaging device 10 of the remote imaging system 700 of Example 3, the number of clients to which imaging is instructed is limited within a range in which the requested resolution desired by each client can be satisfied. For example, if the image sensor 22 has a resolution of 8K (7680×4320)
In the case of , if the requested resolution desired by the client is full high definition (FHD) resolution (1920×1080), the control unit 23 limits the number of clients to 16. Also, if the requested resolution desired by the client is 4K resolution (3840×2160), the control unit 23 limits the number of clients to four. That is, even if a new client gives an instruction to shoot while shooting is being performed at the requested resolution of 4K according to shooting instructions from four clients A to D, the control unit 23 does not allow shooting from the new client. Do not accept instructions.

In the remote image capturing system 700 described in Example 3, images are captured in response to different image capturing requests from each of the clients A to D while satisfying the resolution requested by each of the clients A to D. Conditions can be set and changed, and shooting can be started and ended.
It should be noted that the communication line speed of the communication circuit network 720 may impose a limit on the resolution that can be transmitted to the electronic device 710 .

<Example 4>
In example 4, a case will be described in which setting instructions are issued from more clients than the upper limit of the number of clients capable of capturing different images. In the following description, after the imaging condition is assigned to the unit group 32 of the imaging device 22 provided in the imaging apparatus 10 according to the setting instruction from the clients A to D, the setting instruction is further transmitted from the electronic device 715 of the client E. will be described as an example. As an example of this case, it is assumed that each client A to E sets imaging conditions A5 to E5 as follows for the object to be photographed shown in FIG. 30(a).
(1) Shooting condition A5: A moving image is shot at a frame rate of 60 fps.
(2) Photographing condition B5: A still image of the subject S1 is photographed at an arbitrary timing desired by the client B with proper exposure.
(3) Photographing condition C5: A still image is photographed at an arbitrary timing desired by the client C under automatic exposure control.
(4) Photographing condition D5: Interval still image photographing (time lapse photographing) for photographing still images at predetermined time intervals is performed on the background S2 with proper exposure.
(5) Photographing condition E5: A still image is photographed at an arbitrary timing desired by the client E under automatic exposure control.

Among the shooting conditions A5 to E5, the shooting conditions C5 and E5 are the same except for the timing of shooting a still image, that is, in terms of automatic exposure control, so they can be regarded as similar shooting conditions. In this case, the imaging control unit 231 integrates the imaging condition E5 into the imaging condition C5. The imaging control unit 231 also uses the unit group 32 used for imaging under the imaging condition C5 for imaging under the imaging condition E5. That is, the image capturing control unit 231 causes the same unit group 32 to perform the image capturing operation when receiving the image capturing instruction from the electronic device 713 of the client C and when receiving the image capturing instruction from the electronic device 715 of the client E. .

When the imaging conditions C5 and E5 are integrated, it is preferable to inform the client E that the imaging instruction from the client E will be executed using the unit group 32 used for imaging by the client C. FIG. In this case, the control unit 23 controls the communication unit 101 to transmit information regarding the integration of the imaging conditions to the electronic device 715 of the client E. FIG. When the electronic device 715 receives the information regarding the integration of the imaging conditions, for example, if a monitor is provided, the electronic device 715 displays a message on the monitor, and if provided with a speaker, outputs the message from the speaker. It is sufficient to notify that the imaging conditions will be integrated in .

When the imaging conditions C5 and E5 are integrated, when the image processing unit 232 generates the still image file 402 from the imaging signal output in response to the imaging instruction from the client C, the control unit 23 controls the communication unit 101. Then, the still image file 402 is transmitted to the client C's electronic device 713 . When the image processing unit 232 generates the still image file 402 from the imaging signal output in response to the shooting instruction from the client E, the control unit 23 controls the communication unit 101 to generate the still image file 402. Send to client E's electronic device 715 .

Note that if a setting instruction from the electronic device 713 of the client C and a setting instruction from the electronic device 715 of the client E are received before the imaging conditions are set in the unit group 23 of the imaging device 22, the imaging The control unit 231 assigns the imaging condition to the unit group 32 with priority given to the previously received setting instruction. Further, when a setting instruction is received from the electronic device 715 of the client E while the image sensor 22 is shooting according to the shooting condition C or while the still image file 402 is being generated, the communication unit After the transmission of the still image file 402 to the electronic device 713 of the client C by 101 is completed, the imaging control unit 231 integrates the imaging conditions.

In the above description, the imaging control unit 231 automatically determines the imaging condition to be integrated with the imaging condition E5 of the client E. However, the imaging condition to be integrated by the client E itself can be selected from A5 to D5. Also good. In this case, the control unit 23 controls the communication unit 101 to transmit information indicating the contents of the imaging conditions A5 to D5 to the electronic device 715 of the client E. FIG. The electronic device 715 of the client E displays information indicating the content of the received imaging conditions A5 to D5 on a monitor or the like. The client E selects one of the displayed imaging conditions A5 to D5 by operating an operation member such as an operation button or a touch panel. When the client E selects the imaging conditions, the electronic device 715 transmits the selected imaging conditions to the imaging apparatus 10 . The imaging control unit 231 integrates the received imaging conditions with the imaging conditions E5 of the client E. FIG.

In the above description, the case where the shooting conditions for shooting a still image are integrated has been described as an example, but the shooting conditions for shooting a moving image may be integrated. As an example of this case, clients A to D set imaging conditions A6 to D6 as follows for the object to be photographed shown in FIG. and
(1) Shooting condition A6: shooting a moving image at an EV value of ±0 and a frame rate of 60 fps. A shutter speed of 1/120 is allowed.
(2) Photographing condition B6: A still image of the object S1 is photographed at an arbitrary timing desired by the client B with proper exposure.
(3) Photographing condition C6: A still image is photographed at an arbitrary timing desired by the client C under automatic exposure control.
(4) Photographing condition D6: Interval still image photographing (time lapse photographing) is performed for photographing still images at predetermined time intervals with proper exposure of the background S2.
(5) Shooting condition E6: shooting a moving image at an EV value of −0.3 EV and a frame rate of 60 fps.

Of the shooting conditions A6 to E6, the shooting condition C6 and the shooting condition E6 are different exposure conditions for shooting a moving image. In this case, the imaging control unit 231 integrates the imaging condition E6 into the imaging condition C6, and uses the unit group 32 used for imaging under the imaging condition C6 also for imaging under the imaging condition E6. The imaging control unit 231 sets the frame rate when shooting a moving image to 120 fps, sets the EV value to ±0 in accordance with the imaging condition C6 of the client C, and sets the EV value to ±0 in the 2nth frame (n is a positive integer). In the 2n+1) frame, the EV value is set to -0.3 EV according to the shooting condition E6 of the client E, and a moving image is shot. As a result, a moving image shot under the shooting condition C6 and a moving image shot under the shooting condition E6 are alternately generated, that is, at a frame rate of 60 fps.

In the remote image capturing system 700 described in Example 4, even when the number of clients exceeds the upper limit, it is possible to combine similar capturing conditions among the requested capturing conditions and perform capturing.

<Example 5>
In Example 5, a case where at least one of a plurality of clients A to D requests a change of aperture will be described. In the following description, a case in which client A's imaging condition A7 includes a change in aperture will be taken as an example. In this case, the processing of the imaging control unit 231 differs depending on whether or not the prohibition of aperture change is included in the imaging conditions B7 to D7 of the other clients B to D.

If the shooting conditions B7 to D7 include prohibition of aperture change, the imaging control unit 23 does not perform shooting according to the shooting condition A7 from the client A. FIG. That is, the image capturing control unit 23 rejects the request for image capturing under the image capturing condition A7.

If prohibition of aperture change is not included in the imaging conditions B7 to D7, the imaging control unit 23 drives the aperture (not shown) according to the client A imaging condition A7. In response to shooting instructions from clients B to D, the imaging control unit 231 changes the ISO sensitivity and shutter speed according to the changed aperture so as to obtain the exposure required by the shooting conditions B7 to D7. Note that if an image with a degree of blur different from the degree of blur desired by clients B to D is obtained by changing the aperture, the degree of blur is corrected by performing image processing on the image generated by the image processing unit 232. do.
Note that when the shooting conditions B7 to D7 include moving image shooting, the imaging control unit 231 changes the aperture of the shooting condition A7 so that the shutter speed does not become extremely fast with respect to the requested frame rate. may be restricted. For example, when the frame rate is 60 fps, the imaging control section 231 may limit the aperture change so that the shutter speed is within the range of 1/60 [s] to 1/240 [s].

In the remote image capturing system 700 described in Example 5, even when a change in the aperture of the imaging device 10 is included in the plurality of capturing conditions, a plurality of images for which a plurality of different capturing conditions are set are combined into one image. An image can be captured by the imaging device 10 .

<Example 6>
In Example 6, when the resolutions included in the imaging conditions of the client are different, the number of unit blocks 32 set in each imaging condition is changed according to the resolution. As an example of this case, it is assumed that the clients A and B set imaging conditions A8 and B8 as follows for the imaging target shown in FIG. 30(a).
(1) Shooting condition A8: shoot a moving image at a frame rate of 60 fps.
(2) Shooting condition B8: A still image is shot with high resolution.

When the imaging conditions A8 and B8 are received as setting conditions, the imaging control unit 231 sets the imaging conditions A8 and B8 to the four unit groups 32 in each block 82 on the imaging surface 30 of the imaging element 22. set. For example, the imaging control section 231 sets the imaging condition A8 to one unit group 32 (for example, the upper left unit group 23) among the four unit groups 32 of the block 82. FIG. The imaging control unit 231 sets imaging conditions B8 for the other three unit groups 32 of the four unit groups 32 of the block 82 .

Note that when generating a still image using the imaging signal from the unit group 32 for which the shooting condition B8 is set, the image processing unit 232 transfers the data corresponding to the upper left unit group 32 of the block 82 to the surrounding units. It may be generated by performing interpolation processing from data corresponding to the group 32 . Further, when the timing for shooting a moving image under the shooting condition A8 matches the timing for shooting a still image under the shooting condition B8, the image processing unit 232 processes the upper left unit group of the block 82 used for shooting the moving image. 32 data may be used as data for generating a high resolution still image.

In the remote image capturing system 700 described in Example 6, the number of unit groups 32 of the image sensor 22 for which each capturing condition is set can be varied based on the capturing conditions set by the client. can be generated with one imaging device 10 .

<Example 7>
In the remote image capturing system 700 of Example 7, before actual capturing of the same image, multiple areas within the image are shared by a plurality of clients to set the capturing conditions by performing exposure adjustment, etc., and the set capturing conditions are performed. Actual photography is performed under these conditions. In the following description, an example is given in which three clients A, B, and C share the task of setting the imaging conditions for one image.

In Example 7, for the image P shown in FIG. 31(a), as shown in FIG. A corresponding area) and an area S5 (area corresponding to the background) will be described using an example of division into three. Assume that the client A sets the shooting condition A9 for the area S3 in the image P, the client B sets the shooting condition B9 for the area S4, and the client C sets the shooting condition C9 for the area S5. explain.

The imaging control unit 231 causes the imaging element 22 to perform an imaging operation without setting different imaging conditions for each unit group 32 and output an imaging signal. The image processing unit 232 uses the output imaging signal to generate three identical images for the clients A, B, and C, that is, the image P shown in FIG. are transmitted to the electronic devices 711 to 713 of the clients A to C, respectively.

Each of the electronic devices 711 to 713 displays the received image P on a monitor or the like. While confirming the image displayed on the monitor, the clients A to C individually adjust the exposure for the areas S3 to S5 that they are in charge of. In this case, the electronic device 711 extracts the area S3 from the image P using, for example, known subject area recognition processing such as face recognition and object recognition. The electronic device 712 extracts the area S4 from the image P using subject area recognition processing. The electronic device 713 extracts the region S5 from the image P using subject recognition processing.

Each of the clients A to C adjusts the exposure of the regions S3 to S5 they are in charge of, while checking the situation such as changes in brightness of the entire image. Various parameters representing the exposure of the area S3 adjusted by the client A are associated with the area S3 and transmitted as the shooting condition A by the electronic device 711 . Various parameters representing the exposure of the region S4 adjusted by the client B are associated with the region S4 and transmitted as the shooting conditions B by the electronic device 712 . Various parameters representing the exposure of the region S5 adjusted by the client C are associated with the region S5 and transmitted as the shooting conditions C by the electronic device 714 .

When the communication unit 101 receives the shooting conditions A to C, the imaging control unit 231 applies the received shooting conditions A to C to the unit group 32 at positions corresponding to the regions S3 to S5 on the imaging surface 30 of the imaging device 22. Set C. The imaging control unit 231 causes the imaging element 22 to perform an imaging operation and output an imaging signal according to the received imaging instruction. The image processing unit 232 generates the image file 40 using the output imaging signal.
In order to prevent excessive exposure adjustment for each of the areas S3 to S5 by each of the clients A to C, a limit may be set on the adjustable amount. For example, in the above example, the adjustable amount for the area S3 corresponding to the person's face may be limited to +2/3 EV of the exposure for the area S4 corresponding to the portion other than the person's face.

In the example shown in FIG. 31, the case where a plurality of clients adjust the exposure for an image of one person has been described. You may adjust the exposure of the different person who each becomes a photography object. This case will be explained below.

FIG. 32(a) shows an image P in which three persons are photographed as an example, and FIG. 32(b) shows an area S6 (an area corresponding to the person on the left side of the drawing) and an area S7 in the image P. (area corresponding to the person in the center of the drawing), area S8 (area corresponding to the person on the right side of the drawing), and background area S9. Assume that the client A sets the imaging condition A10 for the area S6 in the image P, the client B sets the imaging condition B10 for the area S7, and the client C sets the imaging condition C10 for the area S8. explain.

As in the case described above, when the electronic devices 711 to 713 of the respective clients A to C receive the same image P, they display the image of the received image file on the monitor or the like. While confirming the image displayed on the monitor, the clients A to C individually adjust the exposure for the areas S6 to S8 that they are in charge of. Also in this case, the electronic device 711 extracts the region S6 from the image P using, for example, known subject region recognition processing such as face recognition and object recognition. The electronic device 712 extracts the area S7 from the image P using subject area recognition processing. The electronic device 713 extracts the region S8 from the image P using subject recognition processing.

Each of the clients A to C adjusts the exposure of the regions S6 to S8 they are in charge of, while checking the situation such as changes in luminance of the entire image. Various parameters representing the exposure of the region S6 adjusted by the client A are associated with the region S6 and transmitted as the photographing condition A by the electronic device 711 . Various parameters representing the exposure of the area S7 adjusted by the client B are associated with the area S7 and transmitted as the shooting condition B by the electronic device 712. FIG. Various parameters representing the exposure of the area S8 adjusted by the client C are associated with the area S8 and transmitted as the shooting conditions C by the electronic device 714. FIG. After that, similarly to the case described above, the imaging control unit 231 sets the imaging conditions for the unit group 32 and the imaging device 22 performs the imaging operation, and the image processing unit 232 generates the image file 40 .

Alternatively, as shown in FIG. 32(c), the image P may be further divided into regions S61 to S81 corresponding to faces and regions S61 to S82 corresponding to non-faces. good. In this case, each of the clients A to C can adjust the exposure for the areas S61 to S81 and the areas S62 to S82 so that the areas S61 to S81, that is, the faces of the people, are exposed as desired. . By adjusting the exposure of the regions S61 to S81 corresponding to the human face, the exposure of the regions S62 to S82 corresponding to the non-human face is automatically adjusted according to the adjusted exposure of the regions S61 to S81. may be adjusted. Further, by adjusting the exposure of the areas S62 to S82 corresponding to areas other than the person's face, the exposure of the areas S61 to S81 corresponding to the person's face is automatically adjusted according to the exposure of the areas S62 to S82 after adjustment. may be adjusted.

In the example described with reference to FIG. 32, when the image generated by the imaging device 10 includes the face of a person whose exposure adjustment is desired by the clients A to C, the clients A to C and the areas S6 to S8 may be associated automatically. In this case, the electronic devices 711 to 713 on the clients A to C side have in advance information on the face of a person desired for exposure adjustment, eg, a face image. The electronic devices 711-713 detect face images on the received images using, for example, known template matching or the like.

Note that the remote image capturing system 700 described above can be used as a monitoring system having a monitoring camera, for example. In this case, the multiple clients are, for example, the security company and the landlord of the house where the surveillance cameras are installed. It is conceivable that the security company and the landlord may have different areas of generated images that they want to watch as monitoring targets. In this case, it becomes possible for the security company and the landlord to set different photographing conditions so that desired images can be obtained with respect to the range they wish to watch.

Further, the above-described image remote photographing system 700 has been described as a system in which the imaging device 10 and the electronic device 710 transmit and receive images and various information via the communication network 720, but the present invention is not limited to this example. For example, the remote image capturing system 700 may have a server device connected to the communication network 720 for transmitting and receiving images and information. In this case, images and various information are transmitted and received between the imaging device 10 and the electronic device 710 via the server device. Alternatively, the server device may generate the image file 40 using the imaging signal output by the imaging device 10 in the same manner as the image processing unit 232 .

In the remote image capturing system 700 in Example 7, different clients can adjust the exposure for each of the plurality of areas S6 to S8 of one image P, and set the capturing conditions for each of the areas S6 to S8 for capturing. . As a result, a plurality of clients can share the work required for setting the imaging conditions, which contributes to an improvement in work efficiency.

The following modifications are also within the scope of the present invention, and it is also possible to combine one or more of the modifications with the above-described embodiments.

(Modification 1)
In the first embodiment, the first image information 641 and the second image information 642 are created when the image file 40 is reproduced, but they may be recorded in the image file 40 in advance. In other words, in the second embodiment, moving images and still images recorded as separate files in separate subdirectories 94 for each large group 81 may be recorded in a single image file 40. . In this case, one frame of data recorded in the image file 40 corresponds to one large group 81 .

For example, consider a case where two moving images (a first moving image and a second moving image), which are divided and recorded in two files in the second embodiment, are recorded in a single image file 40 . At this time, data relating to the first, second, third, and so on of the first moving image are recorded in chronological order from the beginning of the data section 42, and then the first frame of the second moving image, Data relating to the second frame, the third frame, . . . can be recorded in chronological order. By doing so, the load of the reproduction process can be reduced.

In addition, as a recording method different from this, it is also possible to record data relating to each frame of the first moving image and data relating to each frame of the second moving image in chronological order of each frame. In other words, the frames of the two moving images are arranged in the chronological order of shooting, such as the first frame of the first moving image, the first frame of the second moving image, the second frame of the first moving image, and so on. may be recorded in By doing so, the load of the recording process can be reduced.

As described above, when the first image information 641, the second image information 642, etc. are recorded in the image file 40, the mask information 62b can be reduced by using the following recording method. can. In the following description, the case where the image file 40 is generated by the mixed imaging function as shown in FIG. 14 will be described as an example. Also, FIG. 26 schematically shows the structure of the image file 40 generated at this time. Data for one frame, that is, a set of one block 70 and one piece of audio information is called frame data 7 for convenience.

Frame data 7 including image information 64 corresponding to large groups 81 of 1 to 4 are individually recorded in the image file 40 in chronological order. Note that the Tv value map 65 and the like included in each frame data 7 are, of course, also in a state corresponding to each image information. 14, the image information 64 captured in the first frame (that is, the first image information 641) is recorded in the data section 42 as the first frame data 7. be done. Image information 64 captured in the second frame (that is, first image information 641 and second image information 642) are recorded as second and third frame data 7, respectively. Image information 64 captured in the third frame (that is, first image information 641, second image information 642, and third image information 643) is recorded as fourth, fifth, and sixth frame data 7, respectively. be done.

FIG. 26 schematically shows the structure of the generated image file 40. As shown in FIG. When the image information 64 is recorded as described above, the unique identification information 60 recorded in the mask portion 44 of the header portion 41 is assigned to each frame data 7 instead of the mask information 62b. Image identification information 621 corresponding to the number and next image address 622 indicating the recording position of the next frame data 7 are recorded. That is, as in the case where only the data portion 42 of the image file 40 is schematically shown in FIG. A number of "1" is recorded to indicate that. The image identification information 621 of the third frame data 7 is recorded with the number “2” indicating that it is the second image information 642 . As a result, compared to the case where the mask information 62b is recorded in the data section 42 as in the case of the embodiment, the size of the recording area can be reduced.
As the next image address 622, instead of indicating the recording position of the next frame data 7, it may indicate the recording position of the frame data 7 in which the next image having the same image identification information 621 is recorded.

When reproducing the image file 40 as described above, the control section 23 refers to the image identification information 621 and the next image address 622 of the data section 42 to search for the image information 64 to be reproduced. In FIG. 27, when searching for image information 642, the control unit 23 refers to the image identification information 621 of the first frame data 7, determines that it is not the image information 642, and sets the next image address 622. Refer to it and skip to the second frame data 7 . Since the image identification information 621 in the second frame data 7 also indicates that it is not the image information 642 , the control unit 23 refers to the next image address 622 and skips to the third frame data 7 . Since the number "2" indicating that the image information 642 is recorded is recorded in the image identification information 621 of the third frame data 7, the control section 23 extracts the image information 642 from this frame data 7. load. Therefore, when reading an image, the image information 64, Tv value map 65, Sv value map 66, Bv value map 67, and Av value information 68 that are different from the desired image can be skipped. The time required can be shortened.

(Modification 2)
In the description of the first embodiment, the image information 64 and various map information are stored in the data section 42 of the image file 40 created by the video imaging function B and the mixed imaging function. is recorded according to the sequence of This may be recorded in an arrangement different from that of the unit group 32 . This point will be described in detail below.

FIG. 19 is an explanatory diagram of Modification 2. FIG. Here, the unit groups 32 are classified into four large groups 81 as in FIG. 8B. However, the image information 64 generated by the control unit 23 thereafter does not arrange the imaging signals according to the arrangement of the unit groups 32 . Specifically, the image information 64 is generated by consolidating the imaging signals for each large group 81 and then connecting them. For example, when the image information 64 is divided into four 2×2 regions, the imaging signals from the unit groups 32 belonging to the first large group 81 are aggregated in the upper left region, and 2 The imaging signals from the unit groups 32 belonging to the th large group 81 are aggregated, the imaging signals from the unit groups 32 belonging to the third large group 81 are aggregated in the upper right area, and the imaging signals from the unit groups 32 belonging to the third large group 81 are aggregated in the lower right area. The imaging signals from the unit groups 32 belonging to the fourth large group 81 are aggregated.

As described above, when the arrangement of the imaging signals in the image information 64 is changed, the arrangement of the Tv value map 65, the Sv value map 66, the mask information 62, etc. must also be changed accordingly.

Also, the arrangement of the image information 64 may be changed by a method other than this. That is, in the image file 40, as long as the arrangement in the image information 64 corresponds to the arrangement in the information (mask information 62, etc.) related to other imaging conditions, what is the arrangement itself? may

(Modification 3)
In the moving image capturing function B or the mixed capturing function, the usage of the unit group 32 may be changed for each frame. For example, as shown in FIG. 20, in odd-numbered frames, the unit groups 32 are classified into first to fourth large groups 81, respectively, and image information including four pieces of image information 641, 642, 643, and 644 with different imaging conditions. 64 is obtained. In even-numbered frames, the unit group 32 is classified only into the fifth large group 81 so that only a single piece of image information 64 can be obtained. In other words, a plurality of images with different imaging conditions and having a relatively small number of pixels and a single image having a relatively large number of pixels may be imaged in a time division manner. Moreover, this modification 3 can also be applied to the modification 1 and the modification 2 which were mentioned above.

(Modification 4)
In the moving image capturing function B or the mixed capturing function, one unit group 32 may have multiple uses. For example, as shown in FIG. 21, the unit groups 32 may be classified into the first to fourth large groups 81, and all the unit groups 32 may be classified into the fifth large group 81 as well. In this case, when the image file 40 is reproduced (developed, etc.) according to the former classification, image information 64 including four pieces of image information 641, 642, 643, 644 is obtained, and the image file 40 is reproduced (developed, etc.) according to the latter classification. development, etc.), a single image information 64 having a larger number of pixels is obtained.

(Modification 5)
In the description of the still image capturing function B, the unit group 32 to which the number "0" is assigned on the mask information 62 is not used for capturing, and the image information 64 recorded in the data section 42 contains , indicates that information related to the unit group 32 is not included.

Also, in the mask information 62 of the header portion 41, the number "0" may represent that the image is not used for imaging. For example, in the still image capturing function B and the moving image capturing function B, when dividing the entire image capturing screen into four 2×2 unit groups 32 and assigning different uses to each of these four unit groups 32, the unit groups 32 are arranged vertically. If the number of directions (number of rows) is odd, there will be one extra row. In such a case, assuming that the remaining line is not used for imaging, the mask information 62 recorded in the header section 41 may assign a number "0" to the remaining line.

Note that the number "0" described above is only an example, and other numbers may be treated in the same way as the number "0" described above.

(Modification 6)
The structure of the image file 40 may differ from each embodiment described above. Also, the information about the imaging conditions recorded in the image file 40 may be different from the information described in the first embodiment and the like. For example, recording of some information such as the Sv value map 66 may be omitted. Conversely, additional information other than those described above may be added. Also, the recording mode may be different from the embodiment described above. For example, the Av value information 68 may be recorded as an Av value map in which the Av values for each unit group 32 are two-dimensionally arranged in the same manner as the Tv value, the Sv value, and the like.
Information different from the various information described above may be further recorded in the data section 42 . For example, distance information to a subject measured by a well-known distance measurement technique may be recorded in the data section 42 . Further, this distance information may be a so-called depth map in which the distances to the subject measured for each unit group 32 are arranged two-dimensionally. As another example, information regarding the state of the imaging optical system 21 (for example, focal length, etc.) can also be recorded. Also, considering that these pieces of information change during the shooting of moving images, these pieces of information may be recorded for each frame.

Alternatively, recording of the mask information 62 may be omitted. In this case, the control unit 23 uses the Tv value map 65, the Sv value map 66, and the Bv value map 67 at the time of reproduction to determine the purpose (purpose, role) of each unit group 32 corresponding to the mask information 62. Generate information to represent (mask equivalent information). For example, the control unit 23 refers to the Tv value map 65 and/or the Sv value map 66. The control unit 23 regards the coordinates (x, y) storing the same shutter speed as the same area on the referenced Tv value map 65 .

For example, as shown in FIG. 28A, when regions R1 and R2 having different shutter speeds exist in the Tv value map 65, the control unit 23 controls the imaging screen 50 corresponding to the regions R1 and R2. correspond to the main subject area 52 and the background area 53, respectively. For example, when the shutter speed in region 1 is faster than the shutter speed in region 2, the control unit 23 regards the region on the imaging screen 50 corresponding to region R1 as the main subject region 52, and the imaging corresponding to region R2. A region on the screen 50 is considered a background region 53 . Then, the control unit 23 generates the mask equivalent information 63 shown in FIG. 28(b). 28B, "1" is stored at the position of the unit group 32 included in the main subject 52, and "2" is stored at the position of the unit group 32 included in the background area 53, respectively.

When using the Sv value map 66, the control unit 23 considers the coordinates (x, y) in which the same ISO sensitivity is stored as the same region, and similarly generates the mask equivalent information 63. can be done. When using both the Tv value map 65 and the Sv value map 66, the control unit 23 regards the coordinates storing the same shutter speed and the coordinates storing the same ISO sensitivity as the same area. can be performed, the region can be divided in detail. Further, by referring to the Bv value map 67 in addition to the Tv value map 65 and/or the Sv value map 66, the control unit 23 can use the subject luminance distribution for region division. Accuracy can be improved.

Furthermore, the control unit 23 may refer to the Bv value map 67 to generate the mask equivalent information 63 . In this case, the control unit 23 considers the coordinates (x, y) in which the subject brightness within a predetermined range is stored as the same area, and generates the mask equivalent information 63 in the same manner as described above. When the distance information described above, that is, the depth map is recorded, the control section 23 can generate the mask equivalent information 63 using the depth map and the Bv value map 67 . In this case, the control unit 23 refers to the Bv value map 67, regards the coordinates (x, y) storing the same object distance among the regions divided into the same regions as the same region, Generate mask equivalent information 63 . For example, an area with low subject brightness can be divided into an area corresponding to the black subject itself and an area corresponding to the shadow of the subject.

As described above, even if the mask information 62 is not recorded in the image file 40, the control unit 23 can generate the mask-equivalent information based on the imaging conditions during reproduction. Therefore, it becomes unnecessary to record the mask information 62 in the image file 40, and the recording area occupied by the image file 40 can be reduced.
In the above description, the Tv value map 65, the Sv value map 66, and the Bv value map 67 are used to generate a mask corresponding to the mask information 62 generated when the still image imaging function A and the moving image imaging function A are used. Although the case of generating the equivalent information 63 has been described, the present invention is not limited to this example. For example, the control unit 23 refers to the Tv value map 65 and the Sv value map 66 so that the unit groups 32 corresponding to the coordinates storing the same shutter speed and the coordinates storing the same ISO sensitivity are the same. Assuming that the image information 64 constitutes the mask equivalent information 63 corresponding to the mask information 62 generated during the still image imaging function B, the moving image imaging function B, or the mixed imaging function, can be generated.

(Modification 7)
In each of the above-described embodiments, the imaging device, which is a single electronic device having the imaging device 22 and the control unit 23, has been described, but the present invention is not limited to such embodiments. For example, the present invention can be applied to an electronic device that controls the imaging device 22 provided outside. A form of controlling the imaging unit 1001 having the imaging element 22 from an external device will be described in detail below.

FIG. 22 is a block diagram schematically showing the configuration of an imaging system according to Modification 7. As shown in FIG. An imaging system 1000 shown in FIG. 22 comprises an imaging unit 1001 and an electronic device 1002 . The imaging unit 1001 includes the imaging optical system 21 and the imaging element 22 described in the first embodiment, and further includes a first communication section 1003 . Further, the electronic device 1002 includes the control unit 23, the liquid crystal monitor 24, the memory card 25, the operation unit 26, the DRAM 27, the flash memory 28, and the recording unit 29 described in the first embodiment. A section 1004 is provided. The first communication unit 1003 and the second communication unit 1004 are capable of two-way data communication using, for example, well-known wireless communication technology, optical communication technology, or the like. Further, the first communication unit 1003 and the second communication unit 1004 may be configured to perform two-way data communication by wired connection between the imaging unit 1001 and the electronic device 1002 using a wired cable or the like.

In the imaging system 1000 according to Modification 7, the control unit 23 controls the imaging device 22 by data communication via the second communication unit 1004 and the first communication unit 1003 . For example, by transmitting and receiving predetermined control data to and from the imaging unit 1001 , different imaging conditions are set for each unit group 32 and imaging signals are read from each unit group 32 .

As described above, in the imaging system 1000, it is the control unit 23 that controls each unit group 32. FIG. The electronic device 1002 does not have the image pickup device 22, but controls the image pickup device 22 (the image pickup unit 1001) provided outside the electronic device 1002 to perform the same control as in the first embodiment. In other words, the present invention can be applied to electronic equipment that does not have the image pickup device 22 .

(Modification 8)
In order to reduce the data amount of the image information 64, the image information 64 may be compressed and recorded by a well-known lossless compression technique. Also, the image information 64 may be recorded in the form of a difference value between adjacent pixels. For example, at the position where the pixel value (imaging signal) of a certain pixel is recorded, the difference value from the pixel on the left thereof may be recorded. Alternatively, the difference value from the average value of the pixel values of all pixels in the predetermined area may be recorded, or the difference value from the average value of all pixels may be recorded.

In the case of a moving image, the amount of data can be further reduced by recording the difference value from the pixel value at the same position in the previous frame. Alternatively, the pixel value may be recorded only when the pixel value differs from the pixel value at the same position in the previous frame, and the pixel value may not be recorded when the pixel value is the same as in the previous frame. This can also be applied to imaging conditions (Sv value, Tv value, etc.). For example, if a certain unit group 32 has the same Sv value as the previous frame, the Sv value may not be recorded.

When the image information 64 is recorded in the form described above, it is necessary to perform processing to restore the original pixel values from those forms at the time of reproduction (at the time of development).

(Modification 9)
When the frame rate is different between the large groups 81 during the moving image imaging function B or the mixed imaging function, the control unit 23 records each frame based on the fastest frame rate, but the present invention is not limited to this example. For example, the control unit 23 may record each frame at a frame rate corresponding to the lowest common multiple of each frame rate. As shown in FIG. 14, when the first large group 811 captures a moving image at 60 fps and the second large group 812 captures a moving image at 50 fps, the frame rate is 300 frames/second, which is the least common multiple of 60 fps and 50 fps. record. That is, the image information 64 based on the imaging signal from the large group 811 set at 60 fps is recorded every five frames, and the image information 64 based on the imaging signal from the second group 812 set at 50 fps is recorded every six frames. Record it.

(Modification 10)
In each of the above-described embodiments, an example in which the present invention is applied to a lens-integrated camera has been described, but the present invention can also be applied to, for example, an interchangeable-lens camera. Moreover, the present invention is applicable not only to cameras, but also to camera-equipped electronic devices such as PCs, mobile phones, smart phones, and tablets.

As long as the features of the present invention are not impaired, the present invention is not limited to the above embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention. .

DESCRIPTION OF SYMBOLS 10... Imaging device, 21... Imaging optical system, 22... Imaging element, 23... Control part, 24... Liquid crystal monitor, 25... Memory card, 26... Operation part, 27... DRAM, 28... Flash memory, 29... Recording part, 101... Communication unit, 231... Imaging control unit, 232... Image processing unit,
700...Remote image capturing system, 710...Electronic device

Claims (14)

first data having image data of a first subject imaged in a first imaging area among imaging areas of an imaging element and a second subject imaged in a second imaging area among the imaging areas; data relating to a first imaging condition set when imaging the first subject in one imaging region and data relating to a second imaging condition set when imaging the second subject in the second imaging region; a control unit for recording second data having
a reproduction unit for reproducing, on a display unit, an image of the image data included in the first data recorded in the recording unit, based on the second data recorded in the recording unit ;
playback device. The playback device according to claim 1,
The reproducing device, wherein the control unit records the first data and the second data in association with each other in the recording unit. In the playback device according to claim 1 or claim 2,
The control unit records information representing the use of the first imaging area as data related to the first imaging condition, and records information representing the use of the second imaging area as data related to the second imaging condition. playback device. In the playback device according to any one of claims 1 to 3,
The control unit records, as data related to the first imaging condition, information representing a shutter speed set when the first subject is imaged in the first imaging area, and records the shutter speed in the second imaging area. 2. A reproducing device for recording information representing a shutter speed set when an object is imaged as data relating to the second imaging condition. In the playback device according to any one of claims 1 to 4,
The control unit records, as data relating to the first imaging condition, information representing the ISO sensitivity set when the first subject is imaged in the first imaging area, and records the ISO sensitivity in the second imaging area. 2. A reproducing device for recording information representing an ISO sensitivity set when an object is imaged as data relating to the second imaging condition. In the playback device according to any one of claims 1 to 5,
The control unit records information representing luminance of the first subject imaged in the first imaging area as data regarding the first imaging condition, and records the second subject imaged in the second imaging area. a reproducing device for recording information representing the luminance of the image as data relating to the second imaging condition. In the playback device according to any one of claims 1 to 6,
The reproducing device, wherein the image data is still image data. In the playback device according to any one of claims 1 to 6,
The reproducing device, wherein the image data is moving image data. In the playback device according to any one of claims 1 to 8,
A plurality of pixels are arranged in the imaging area,
At least one pixel of the plurality of pixels is arranged in the first imaging region,
The playback device in which at least one of the plurality of pixels is arranged in the second imaging area. In the playback device according to claim 9,
The playback device, wherein the first imaging area and the second imaging area are arranged side by side in the row direction in the imaging area. In the playback device according to claim 9,
The playback device, wherein the first imaging area and the second imaging area are arranged side by side in a column direction in the imaging area. An electronic device comprising the playback device according to any one of claims 1 to 11. In the electronic device according to claim 12,
An electronic device comprising the imaging device. In the electronic device according to claim 13,
The imaging element is an electronic device configured with a plurality of semiconductor substrates.

Images