JP2018191248A - Imaging device, imaging method, and program - Google Patents

Abstract

In an apparatus including a plurality of imaging devices, the plurality of imaging devices can be effectively used.
An imaging unit, a signal processing unit that processes a signal from the imaging unit, a generation unit that generates a clock signal, and a first mode that operates in synchronization with another imaging device or a first that operates asynchronously. A control unit that controls each unit in the second mode. In the first mode, a clock signal supplied from another imaging device is supplied to the signal processing unit, and the generation unit is stopped. The present technology can be applied to an imaging device including a plurality of imaging devices.
[Selection] Figure 2

Description

  The present technology relates to an imaging device, an imaging method, and a program. For example, the present technology relates to an imaging device, an imaging method, and a program suitable for use when imaging is performed with a plurality of imaging devices.

  For example, it has been proposed to perform stereoscopic shooting using a plurality of imaging devices capable of imaging alone. When stereoscopic imaging is performed with a plurality of imaging devices, the optical conditions that change depending on the state (position) of the subject such as focus, zoom, and aperture of these imaging devices are driven simultaneously so as to match.

  For example, Patent Document 1 discloses a technique for synchronizing two imaging devices by controlling a position signal of a lens serving as a master as a position command signal on the slave side.

Japanese Patent No. 3278667

  In a system in which a plurality of imaging devices are synchronized and imaged, it is desired to use the plurality of imaging devices more effectively. For example, it is desired to be able to use a plurality of imaging devices more effectively by synchronizing a plurality of imaging devices to perform imaging or each imaging device independently imaging. Yes.

  This technique is made in view of such a situation, and makes it possible to use a plurality of imaging devices more effectively.

  A first imaging device according to an aspect of the present technology operates in synchronization with an imaging unit, a signal processing unit that processes a signal from the imaging unit, a generation unit that generates a clock signal, and another imaging device. And a control unit that controls each unit in the first mode or the second mode that operates asynchronously.

  A first imaging method according to an aspect of the present technology is an imaging method of an imaging device including an imaging unit, which processes a signal from the imaging unit, generates a clock signal, and operates in synchronization with another imaging device. The method includes a step of controlling each unit in a first mode or a second mode that operates asynchronously.

  A first program according to an aspect of the present technology is a first mode in which an imaging device including an imaging unit processes a signal from the imaging unit, generates a clock signal, and operates in synchronization with another imaging device. Or the process including the step which controls each part is performed in the 2nd mode which operate | moves asynchronously.

  A second imaging device according to an aspect of the present technology includes a plurality of imaging devices including an imaging unit, a signal processing unit that processes a signal from the imaging unit, and a generation unit that generates a clock signal. Among the imaging devices, the imaging device includes a first imaging device operating in the master mode and a second imaging device operating in the slave mode, and the second imaging device is the first imaging device. The master mode includes a pause mode, and the slave mode includes a synchronous mode that operates in synchronization with the first imaging device and an asynchronous mode that operates asynchronously. .

  A second imaging method according to an aspect of the present technology includes a plurality of imaging devices including an imaging unit, a signal processing unit that processes a signal from the imaging unit, and a generation unit that generates a clock signal. Among imaging devices, in the imaging method of an imaging device comprising a first imaging device operating in a master mode and a second imaging device operating in a slave mode, the second imaging device The master mode operates in an operation mode or a pause mode, and the slave mode operates in synchronization with the first imaging device by receiving a clock signal generated by the first imaging device. Or a step of operating in an asynchronous mode that operates asynchronously.

  A second program according to an aspect of the present technology includes a plurality of imaging devices including an imaging unit, a signal processing unit that processes a signal from the imaging unit, and a generation unit that generates a clock signal, and the plurality of imaging units Among the devices, the second imaging device includes the first imaging device that includes the first imaging device that operates in the master mode and the second imaging device that operates in the slave mode. The master mode operates in an operation mode or a sleep mode, and the slave mode operates in a synchronous mode or asynchronously that operates in synchronization with the first imaging device. The process including the step that operates in the asynchronous mode is executed.

  In the first imaging device, the imaging method, and the program according to an aspect of the present technology, an imaging unit is provided, a signal from the imaging unit is processed, and a clock signal is generated. Each unit is controlled in a first mode that operates in synchronization with another imaging device or in a second mode that operates asynchronously.

  A second imaging device according to an aspect of the present technology includes a plurality of imaging devices, each of the plurality of imaging devices includes an imaging unit, processes a signal from the imaging unit, and generates a clock signal. . Among the plurality of imaging devices, a first imaging device operating in the master mode and a second imaging device operating in the slave mode are provided. The second imaging device is supplied with the clock signal generated by the first imaging device, the master mode includes a pause mode, and the slave mode operates in synchronization with the first imaging device. It includes a synchronous mode and an asynchronous mode that operates asynchronously.

  Note that the imaging device may be an independent device, or may be an internal block constituting one device.

  The program can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.

  According to one aspect of the present technology, a plurality of imaging devices can be used more effectively.

  Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure showing the composition of the 1 embodiment of the imaging device to which this art is applied. It is a figure which shows the structural example of a sensor. It is a figure which shows the structural example of the sensor at the time of master mode. It is a figure which shows the structural example of the sensor at the time of slave mode. It is a figure for demonstrating the synchronization of sensors. It is a figure for demonstrating the operation example of a sensor. It is a figure which shows the other structural example of the sensor at the time of master mode. It is a figure which shows the other structural example of the sensor at the time of slave mode. It is a figure for demonstrating another operation example. It is a figure for demonstrating another operation example. It is a figure for demonstrating another operation example. It is a figure for demonstrating another operation example. It is a figure for demonstrating another operation example. It is a figure for demonstrating another operation example. It is a figure which shows the example of an image imaged with a sensor. It is a figure for demonstrating the specific example of operation | movement of a sensor. It is a figure which shows the example of an image imaged with a sensor. It is a figure for demonstrating the specific example of operation | movement of a sensor. It is a figure for demonstrating a recording medium. It is a block diagram which shows an example of a schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of a vehicle exterior information detection part and an imaging part.

  Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described.

<Configuration of imaging device>
Since the present technology can be applied to an imaging apparatus, in the following description, the case where the present technology is applied to an imaging apparatus will be described as an example.

  FIG. 1 is a diagram illustrating a configuration of an embodiment of an imaging apparatus to which the present technology is applied. The imaging device 10 illustrated in FIG. 1 includes a sensor 11-1, a sensor 11-2, and a control unit 12.

  A sensor 11-1 and a sensor 11-2 (hereinafter, when the sensors 11-1 and 11-2 do not need to be individually distinguished are simply referred to as a sensor 11) are a CCD (Charge Coupled Device) or a CMOS (Complementary An imaging device such as a metal-oxide semiconductor sensor can be used.

  The control unit 12 can be, for example, an ISP (Image Signal Processor). The control unit 12 processes a signal from the sensor 11 and outputs the signal to a subsequent processing unit (not shown) or controls the sensor 11.

  The sensor 11-1 and the sensor 11-2 will be described later in detail, and there are a mode in which they operate in synchronization and a mode in which they operate individually. In the mode that operates in synchronization, there is a master-slave relationship between the sensor 11-1 and the sensor 11-2, one operating as a master (main) and the other operating as a slave (slave).

  Here, since the image pickup apparatus 10 will be described as an example, the description will be continued by taking the case where the sensor 11 is an image pickup element (image sensor) such as a CCD or a CMOS sensor as an example. The present technology can be applied to other sensors. For example, a sensor such as a pressure sensor, a temperature sensor, an acceleration sensor, or an illuminance sensor may be used.

  Further, the sensor 11-1 and the sensor 11-2 are described as being the same type of sensor, in this case, an image sensor, but may be different types of sensors. According to the present technology, as will be described below, the sensor 11-1 and the sensor 11-2 can operate synchronously or individually, and therefore, with a plurality of sensors. Information can be acquired at the same timing, and information can be acquired at different timings.

  The imaging apparatus 10 shown in FIG. 1 will be described as having two sensors 11 of the same type, that is, a sensor 11-1 and a sensor 11-2 configured by an imaging element or the like. 11 is also applicable.

<Sensor configuration>
FIG. 2 is a diagram illustrating a configuration example of the sensor 11.

  The sensor 11 includes an imaging unit 31, a signal processing unit 32, a clock generation unit 33, and a control unit 34. The control unit 34 includes a master mode control unit 41 and a slave mode control unit 42.

  The sensor 11 shown in FIG. 2 shows a configuration corresponding to both modes when operating as a master (master mode) and when operating as a slave (slave mode).

  When the sensor 11 is a sensor that operates only as a master, the control unit 34 can be configured to include only the master mode control unit 41. When the sensor 11 is a sensor that operates only as a slave, the control unit 34 can be configured to include only the slave mode control unit 42.

  The imaging unit 31 includes an imaging element and a lens, receives reflected light from the subject, and outputs a signal corresponding to the amount of received light to the signal processing unit 32. The signal processing unit 32 performs processing such as defect correction, noise reduction, and high dynamic range synthesis processing (HDR) on the input signal and outputs the signal to the control unit 12 (FIG. 1). Note that the function of processing the signal output from the imaging unit 31 may be distributed between the signal processing unit 32 and the control unit 12 (FIG. 1), or may be provided in only one of them.

  The clock generation unit 33 generates a clock signal and supplies it to the signal processing unit 32. The clock signal generated by the clock generation unit 33 is also supplied to the slave-side sensor 11 when operating in the master mode. When operating in the slave mode, the generation of the clock signal by the clock generation unit 33 is stopped, and the clock signal from the sensor 11 operating in the master mode is supplied to the signal processing unit 32.

  Note that the clock signal generated by the clock generation unit 33 is a synchronization signal (XVS), and for example, the description will be continued assuming that it is a vertical synchronization signal, and the clock signal will also be described as a vertical synchronization signal as appropriate.

  The control unit 34 operates in a master mode in which control is performed by the master mode control unit 41 or is controlled by the slave mode control unit 42 based on an external signal (control signal from the control unit 12 (FIG. 1)). Whether to operate in the slave mode is set, and each unit in the sensor 11 is controlled in the set mode.

  The master mode control unit 41 has an operation mode 51 and a pause mode 52. The operation mode 51 is a mode when operating in the master mode. The pause mode 52 is a mode in which the operation is paused. In the pause mode 52, power consumption is reduced because imaging in the imaging unit 31, signal processing in the signal processing unit 32, generation of a clock signal in the clock generation unit 33, and the like are stopped.

  The slave mode control unit 42 has a synchronous mode 61 and an asynchronous mode 62. The synchronization mode 61 is a mode in which a clock signal is supplied from the master-side sensor 11 and operates in synchronization with the master-side sensor 11. The asynchronous mode 62 is a mode that operates based on the clock signal generated by its own clock generation unit 33 without receiving the supply of the clock signal from the master-side sensor 11, and operates asynchronously with the master-side sensor 11. This is the mode to use.

<Operation in synchronous mode>
The operation of the image pickup apparatus 10 shown in FIG. 1 including the sensor 11 shown in FIG. 2 will be described.

  First, the operation in the synchronous mode will be described. The synchronous mode is a mode in which the sensor 11 set as the master and the sensor 11 set as the slave operate in synchronization. Here, the sensor 11 set as the master is described as the sensor 11M, for example, the sensor 11-1 in FIG. 1, and the sensor 11 set as the slave is described as the sensor 11S. For example, the sensor in FIG. The description will be continued assuming that it is 11-2.

  FIG. 3 is a diagram showing a portion that is operating when the sensor 11M operates in the master mode. Since the sensor 11M operates in the master mode, the master mode control unit 41 is operating. Further, when the sensor 11M operates in the synchronous mode, the master mode control unit 41 is set to a mode that operates in the operation mode 51.

  The master-side sensor 11M supplies a clock signal to the slave-side sensor 11S, so that the clock generation unit 33M is also operating to generate a clock signal when the sensor 11M operates, and the generated clock Output the signal to the outside. Further, the signal processing unit 32M is in a state where it is not supplied with a clock signal from the outside (a state where it is blocked).

  FIG. 4 is a diagram illustrating a portion that is operating when the sensor 11S operates in the slave mode. Since the sensor 11S operates in the slave mode, the slave mode control unit 42 is operating. Further, when the sensor 11S operates in the synchronous mode, the slave mode control unit 42 is set to a mode that operates in the synchronous mode 61.

  Further, the slave-side sensor 11S receives the supply of the clock signal from the master-side sensor 11M and operates based on the clock signal, so that the clock generation unit 33S is in a stopped state. The signal processing unit 32S is in a state of receiving a supply of a clock signal from the outside (sensor 11M), and operates based on the supplied clock signal.

  FIG. 5 shows a clock signal (vertical synchronization signal, described as XVS in FIG. 3) generated by the sensor 11M set as the master, data output from the sensor 11M, and a sensor set from the sensor 11M to the slave. It is a figure which shows the relationship between the clock signal (vertical synchronizing signal) supplied to 11S, internal XVS of the sensor 11S, and the data which the sensor 11S outputs.

  At time t1, a clock signal (XVS) is generated by the clock generation unit 33M of the sensor 11M. An image of one frame is output before the next vertical synchronization signal with a vertical synchronization signal (XVS) as a trigger. The clock generation unit 33M generates such a vertical synchronization signal. The vertical synchronization signal generated at time t1 is described as a vertical synchronization signal 1M.

  At time t2, the vertical synchronization signal 2M is generated by the clock generation unit 33M of the sensor 11M. The frame 1M is output from the sensor 11S between time t1 and time t2 (between the vertical synchronization signal 1M and the vertical synchronization signal 2M).

  At time t1, the vertical synchronization signal 1M generated by the clock generation unit 33M is also supplied from the sensor 11M to the sensor 11S. The signal processing unit 32S of the sensor 11S is supplied with the vertical synchronization signal 1M at time t1. The sensor 11S starts outputting the frame S1 upon receiving the supply of the vertical synchronization signal 1M at time t1.

  The sensor 11S receives the vertical synchronization signal 2M from the sensor 11M at time t2. The sensor 11S starts to output the frame 2S upon receiving the supply of the vertical synchronization signal 2M at time t2.

  That is, in this case, at time t1, the vertical synchronization signal 1M generated by the master-side sensor 11M is supplied to the inside of the sensor 11M and also to the slave-side sensor 11S. Therefore, at time t1, the sensor 11M and the sensor 11S start imaging and output the frame 1M and the frame 1S, respectively.

  That is, at time t1, the sensor 11M and the sensor 11S perform synchronized imaging. Similarly, at time t2, the sensor 11M and the sensor 11S output a frame 2M and a frame 2S, respectively, according to the vertical synchronization signal 2M generated at the time t2. Such synchronized imaging is sequentially performed.

  For example, a parallax image can be obtained by performing synchronized imaging with the sensor 11M and the sensor 11S (sensor 11-1 and sensor 11-2). For example, a parallax image can be obtained by using the sensor 11-1 as a sensor that captures an image for the right eye and the sensor 11-2 as a sensor that captures an image for the left eye, and performing synchronized imaging.

  By applying the present technology, as described below, it is possible to perform shooting other than synchronized shooting.

  For example, one of the sensor 11M and the sensor 11S can be stopped, the other can be operated, and imaging can be performed with the sensor 11 that is operating. By applying the present technology, this is possible. However, in the sensor 11 to which the present technology is not applied, the operation of the master-side sensor 11M cannot be stopped for the following reason.

  As described with reference to FIGS. 3 to 5, the vertical synchronization signal M generated by the sensor 11 </ b> M is supplied to the sensor 11 </ b> M and the sensor 11 </ b> S so that synchronization is performed. When the sensor 11M is stopped in such a state, the vertical synchronization signal M is not generated by the sensor 11M, and the vertical synchronization signal M is not supplied from the sensor 11M to the sensor 11S. Therefore, the sensor 11S cannot be operated.

  However, as described below, according to the present technology, one of the sensor 11M and the sensor 11S can be stopped, the other can be operated, and imaging can be performed with the sensor 11 that is operating. Thus, the mode in which the sensor 11 is stopped is referred to as a pause mode.

  In addition, by applying the present technology, the sensor 11M and the sensor 11S are individually operated, so that, for example, imaging with a long exposure time is performed on one side and imaging with a short exposure time is performed on the other side. Imaging can be performed.

  As described with reference to FIGS. 3 to 5, the vertical synchronization signal M generated by the sensor 11M is supplied to the sensor 11M and the sensor 11S, so that the sensor 11M and the sensor 11S perform imaging. Since the sensor 11M and the sensor 11S perform processing based on the same signal, it is difficult to perform shooting with different exposure times, but according to the present technology, it is possible.

  As described above, the case where imaging is performed under different conditions between the sensor 11M and the sensor 11S is referred to as an asynchronous mode.

<About sleep mode>
<First operation example>
A description will be given of the sleep mode. The pause mode is a state in which the sensor 11M as a master is in a pause mode and the sensor 11S as a slave is in an asynchronous mode.

  FIG. 6 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  Before the time t11, the master-side sensor 11M operates in the operation mode 51, and the slave-side sensor 11S operates in the synchronous mode 61.

  Before the time t11, the sensor 11M is in a state as shown in FIG. 3 and is controlled by the master mode control unit 41, and the master mode control unit 41 is operating in the operation mode 51. It is.

  Further, at a time before time t11, the sensor 11S is controlled by the slave mode control unit 42, and the slave mode control unit 42 is operating in the synchronous mode 61.

  At time t11, the control unit 12 instructs the control unit 34S of the sensor 11S to shift from the synchronous mode 61 to the asynchronous mode 62. For example, when the control unit 34 is set to execute the instruction after two clock signals after receiving the instruction from the control unit 12 (external), the sensor 11S operates in the asynchronous mode after two clock signals from time t11. Shift to 62.

  At time t12, which is a time point after time t11, the control unit 12 (FIG. 1) instructs the control unit 34M of the sensor 11M to shift from the operation mode 51 to the pause mode 52.

  When the sensor 11M shifts from the operation mode 51 to the pause mode 52, the sensor 11M shifts to a state where no clock signal is supplied to the sensor 11S. If the sensor 11S on the slave side shifts to the asynchronous mode 62, in other words, when the clock signal is not supplied from the sensor 11M in the synchronous mode 61, the sensor 11S becomes inoperable. There is a possibility that.

  Therefore, in order to prevent the sensor 11S from being inoperable in this way, for example, the mode of the sensor 11M is shifted from the operation mode 51 to the sleep mode 52 at a time after the sensor 11S has shifted to the asynchronous mode 62. To. Here, it is assumed that such timing is time t12.

  Alternatively, at time t11, the control unit 34M of the sensor 11M may be instructed to shift from the operation mode 51 to the pause mode 52. At the time t11, the sensor 11M and the sensor 11S operate in synchronization, so that when the mode switching is instructed at the same time, the mode switching is executed at the same timing. Therefore, the mode switching may be instructed to the sensor 11M and the sensor 11S at the same time.

  Such a mode switching instruction timing can be set in consideration of the characteristics of the sensor 11 and the like.

  At time t12, the control unit 12 (FIG. 1) instructs the control unit 34M of the sensor 11M to shift from the operation mode 51 to the pause mode 52. By issuing such an instruction, at time t12, the master mode control unit 41 of the control unit 34M of the sensor 11M shifts to the sleep mode 52 and starts operation in the sleep mode 52.

  FIG. 7 is a diagram illustrating a portion that is operating when the sensor 11M operates in the sleep mode 52. In FIG. Since the sensor 11M operates in the master mode, the master mode control unit 41 is operating, and the sensor 11M is set to operate in the sleep mode 52 of the master mode.

  Further, when the master-side sensor 11M operates in the pause mode 52, it is not necessary to supply a clock signal to the inside or supply a clock signal to the slave-side sensor 11S. Has been stopped. The imaging unit 31M and the signal processing unit 32M are also stopped because they can stop the operation.

  As described above, in the sleep mode 52, each unit in the sensor 11M is in a stopped state, so that the power consumed by the sensor 11M can be reduced. That is, by realizing such a sleep mode 52, it is possible to realize low power consumption.

  FIG. 8 is a diagram showing a portion that is operating when the sensor 11S operates in the slave mode and operates in the asynchronous mode. Since the sensor 11S operates in the slave mode, the slave mode control unit 42 is operating. Further, when the sensor 11S operates in the asynchronous mode, the slave mode control unit 42 is set to a mode in which the asynchronous mode 62 operates.

  Further, when operating in the asynchronous mode 62, the slave-side sensor 11S does not receive the clock signal supplied from the master-side sensor 11M, and generates a clock signal in its own clock generation unit 33S. The sensor 11S operates based on the clock signal generated by the clock generation unit 33S.

  In addition, the signal processing unit 32S is changed from a state in which an external clock signal is supplied to a state in which the clock signal generated by the clock generation unit 33S is supplied, and operates based on the supplied clock signal.

  At time t13, the control unit 41M of the sensor 11M is instructed to switch from the pause mode 52 to the operation mode 51. By giving such an instruction, the sensor 11M is switched from a state where the pause mode 52 as shown in FIG. 7 is valid to a state where the operation mode 51 as shown in FIG. 3 is valid.

  In addition, when the operation mode 51 is in a valid state, the clock generation unit 33M starts generating the clock signal, and the clock signal can be supplied to the slave-side sensor 11S.

  At time t14 when the clock signal is output from the sensor 11M, the control unit 41S of the sensor 11S is instructed to switch from the asynchronous mode 62 to the synchronous mode 61. . By giving such an instruction, the sensor 11S is switched from a state where the asynchronous mode 62 as shown in FIG. 8 is valid to a state where the synchronous mode 61 as shown in FIG. 4 is valid.

  In addition, when the synchronization mode 61 is enabled, input of a clock signal supplied from the sensor 11M is started.

  As described above, in the imaging apparatus 10 including the plurality of sensors 11, the plurality of sensors 11 can be driven synchronously or can be driven asynchronously. In addition, when the plurality of sensors 11 are driven asynchronously, it is possible to provide sensors 11 that are not driven. Further, the power consumption of the sensor 11 that is not driven can be reduced, and the power consumption of the imaging device 10 can also be reduced.

  In the example shown in FIG. 6, the slave-side sensor 11S continues to be driven in the synchronous mode 61 and the asynchronous mode 62, but the slave-side sensor 11S is controlled by the master mode control unit 41. By setting the state to operate in the pause mode 52, it is possible to enter a pause state (a state in which it is not driven). Therefore, it is possible to stop the driving of the plurality of sensors 11 as well as the single sensor 11 among the plurality of sensors 11.

<Second operation example>
Next, an operation example (second operation example) in the asynchronous mode will be described.

  FIG. 9 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  Of the operations of the sensor 11M and the sensor 11S illustrated in FIG. 9, the description of the same portions as the operations of the sensor 11M and the sensor 11S illustrated in FIG. 6 is omitted. The operation of the slave-side sensor 11S shifts from the synchronous mode 61 to the asynchronous mode 62 and from the asynchronous mode 62 to the synchronous mode 61, similarly to the operation of the sensor 11S shown in FIG.

  At time t21, the control unit 41 is instructed to switch from the synchronous mode 61 to the asynchronous mode 62 to the control unit 41S of the sensor 11S. After receiving such an instruction, the synchronous mode 61 is switched to the asynchronous mode 62 after two clock signals. At this time, the frequency (frame rate) of the clock signal after shifting to the asynchronous mode 62 is also instructed.

  At time t22, the control unit 12 instructs the control unit 41M of the sensor 11M to change the frequency of the clock signal. For example, here, the frequency at the time before the instruction is the frequency 11MA, and the frequency at the later time is the frequency 11MB.

  When the sensor 11M receives an instruction to change the frequency of the clock signal from the frequency 11MA to the frequency 11MB at time t22, the sensor 11M changes the frequency of the clock signal to the frequency 11MB after receiving two clock signals from the time t22. .

  On the other hand, when switching from the synchronous mode 61 to the asynchronous mode 62 is instructed at time t21, the sensor 11S receives the two clock signals from time t21 and then generates a clock signal by its own clock generation unit 33S. The frequency of the generated clock signal is a frequency 11SB.

  The sensor 11M continues to operate in the operation mode 51, but the frequency of the clock signal is changed, and a clock signal having a frequency of 11 MB after the change is generated. The sensor 11S transitions from the synchronous mode 61 to the asynchronous mode 62, and enters a state of generating a clock signal having a frequency of 11SB from the state in which the supply of the clock signal is received.

  The frequency 11MB and the frequency 11SB can be different frequencies. Therefore, the sensor 11M and the sensor 11S can be operated at different frequencies. In other words, the sensor 11S set as the slave can be operated at a frame rate that does not depend on the sensor 11M set as the master.

  At time t23, the control unit 41M of the sensor 11M instructs the control unit 12 to change the frequency of the clock signal from the frequency 11MB to the frequency 11MC. At time t23, the control unit 41 is instructed to switch from the asynchronous mode 62 to the synchronous mode 61 to the control unit 41S of the sensor 11S.

  Upon receiving an instruction to change the frequency of the clock signal from the frequency 11 MB to the frequency 11MC at time t23, the sensor 11M changes the frequency of the clock signal to the frequency 11MC after receiving the two clock signals from time t22. .

  On the other hand, when switching from the asynchronous mode 62 to the synchronous mode 61 is instructed at the time t23, the sensor 11S shifts to a state in which the clock signal is input from the sensor 11M after receiving the two clock signals from the time t23. The operation is synchronized with the sensor 11M.

  In this way, it is possible to dynamically switch between a state in which the plurality of sensors 11 are synchronized and an asynchronous state.

<Third operation example>
Next, a third operation example will be described.

  FIG. 10 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  When the sensor 11M and the sensor 11S have the configuration of the sensor 11 shown in FIG. 2, for example, the sensor 11 operating in the slave mode can be operated by switching to the master mode. As a third operation example, an operation when the sensor 11 operating in the slave mode is operated by switching to the master mode will be described.

  At a time before time t31, the sensor 11S operates in the slave mode. That is, as shown in FIG. 4, the slave mode control unit 42 operates in a state where the synchronization mode 61 is valid.

  At time t31, the control unit 41 of the sensor 11S is instructed to switch from the slave mode to the master mode. After receiving such an instruction and receiving the 2-clock signal, the sensor 11S switches from the slave mode to the master mode. By switching to the master mode, as shown in FIG. 3, the operation mode 51 of the master mode control unit 41 is in a valid state.

  At time t31, the frequency of the clock signal after shifting to the master mode is also designated for the sensor 11S, and the sensor 11S starts generating the clock signal at the designated frequency.

  On the other hand, at time t32, the control unit 41 of the sensor 11M is instructed to change the frequency of the clock signal. After receiving such an instruction and receiving the two clock signal, the sensor 11M switches the frequency of the clock signal to the instructed frequency.

  Before and after time t32, the sensor 11M operates in the master mode (maintains the operation in the master mode). On the other hand, the sensor 11S operates in the master mode after time t31. In this case, after time t32, the two sensors 11 are both operating in the master mode.

  When the sensor 11 is in the master mode, the clock signal is supplied to the outside, but the clock signal from the outside is not input. Therefore, even if the two sensors 11 are operating in the master mode, the external clock signal is not input, so that the two sensors 11 can operate without interfering with each other.

  Thus, it is possible to operate the sensor 11S operating in the slave mode in the master mode. It is also possible to operate the sensor 11M operating in the master mode in the slave mode.

  Referring to FIG. 10, at time t <b> 33, the control unit 41 instructs the control unit 41 </ b> M of the sensor 11 </ b> M to switch from the master mode to the slave mode.

  After receiving such an instruction and receiving the 2-clock signal, the sensor 11M switches from the master mode to the slave mode. By switching to the slave mode, the synchronization mode 61 of the slave mode control unit 42 is enabled as shown in FIG. When the sensor 11M starts operating in the slave mode synchronous mode 61, the clock signal supplied from the sensor 11S is input, and the sensor 11M operates based on the input clock signal.

  Thus, the master mode and the slave mode can be changed dynamically. Further, in a system including a plurality of sensors 11, each sensor 11 can be operated in a desired mode of either a master mode or a slave mode. It is also possible to operate a plurality of sensors 11 in the master mode.

<Fourth operation example>
Next, a fourth operation example will be described.

  FIG. 11 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  In the operation described with reference to FIG. 10, the sensor 11M shifts from the master mode to the slave mode. However, the sensor 11M may continue to operate in the master mode. Further, the frequency (frame rate) of the clock signal can be changed. Such an operation example will be described as a fourth operation example.

  At a time before time t41, the sensor 11S is operating in the slave mode. That is, as shown in FIG. 4, the slave mode control unit 42 operates in a state where the synchronization mode 61 is valid.

  At time t41, the control unit 41 instructs the sensor 11S to switch from the slave mode to the master mode. After receiving such an instruction and receiving the 2-clock signal, the sensor 11S switches from the slave mode to the master mode. By switching to the master mode, as shown in FIG. 3, the operation mode 51 of the master mode control unit 41 is in a valid state.

  At time t41, the frequency of the clock signal after shifting to the master mode is also designated for the sensor 11S, and the sensor 11S starts generating the clock signal at the designated frequency.

  On the other hand, at time t42, the control unit 41 of the sensor 11M is instructed to change the frequency of the clock signal. After receiving such an instruction and receiving the 2-clock signal, the sensor 11M switches to the instructed frequency.

  At time t43, the control unit 12M of the sensor 11M is instructed to change the frequency of the clock signal. At time t43, the control unit 12S of the sensor 11S is instructed to change the frequency of the clock signal. After receiving such an instruction and receiving the 2-clock signal, the sensor 11M and the sensor 11S each change the frequency of the clock signal.

  In the operation example illustrated in FIG. 11, the sensor 11M continues to operate in a state where the operation mode 51 of the master mode control unit 41 is enabled. Further, the sensor 11S shifts from the state in which the synchronization mode 61 of the slave mode control unit 42 is enabled to the state in which the operation mode 51 of the master mode control unit 41 is enabled, and operates in the master mode. continuing.

  Thus, the master mode and the slave mode can be changed dynamically. Further, in a system including a plurality of sensors 11, each sensor 11 can be operated in a desired mode of either a master mode or a slave mode. It is also possible to operate a plurality of sensors 11 in the master mode.

<Fifth operation example>
Next, a fifth operation example will be described.

  FIG. 12 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  As a fifth operation example, a description will be given of an operation in which a plurality of sensors 11 are operating in synchronization, but the frame rates of the sensors 11 are different.

  In FIG. 12, the sensor 11M continues to operate in a state where the operation mode 51 of the master mode control unit 41 is enabled. Further, the sensor 11S continues to operate in a state where the synchronization mode 61 of the slave mode control unit 42 is enabled.

  Since the sensor 11S operates in the synchronous mode 61 of the slave mode control unit 42, the sensor 11S operates based on the clock signal supplied from the sensor 11M (operates at the same frame rate as the sensor 11M). FIG. 12 shows a case where the frame rate is changed only on the sensor 11S side.

  In other words, at time t51, the control unit 41S of the sensor 11S is instructed by the control unit 12 to change the frequency of the clock signal (change of the frame rate). The sensor 11S changes the frequency by thinning out the clock signal supplied from the sensor 11M. The example shown in FIG. 12 shows an example in which the frame rate is reduced to ¼.

  That is, the sensor 11S applies the clock signal (vertical synchronization signal) four times from the sensor 1M, applies it once, and discards the remaining three times, thereby reducing the frame rate to ¼.

  In this way, different operations can be performed by controlling the number of input clocks while synchronizing.

  In the example shown in FIG. 12, at time t12, the control unit 41S of the sensor 11S is instructed by the control unit 12 to change the frequency of the clock signal (change of the frame rate). That is, an instruction to use the clock signal of the sensor 11M as it is is issued. As a result, the sensor 11S performs the process without thinning out in the synchronous mode 61.

  In this way, it is possible to allow each sensor 11 to perform processing at a desired frame rate as in the asynchronous mode by changing the method of obtaining the synchronization signal while synchronizing.

  Further, by reducing the frame rate, a low consumption operation can be realized.

<Sixth operation example>
Next, a sixth operation example will be described.

  FIG. 13 shows the relationship between the clock signal (master XVS) generated by the sensor 11M on the master side, the mode of the sensor 11M, the clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, and the mode of the sensor 11S. It is a figure for demonstrating.

  In the sixth operation example, as in the fifth operation example, a plurality of sensors 11 are operated in synchronism, but a case where the plurality of sensors 11 operate at different frame rates will be described.

  In FIG. 13, the sensor 11M continues to operate in a state where the operation mode 51 of the master mode control unit 41 is enabled, but the frame rate is instructed at time t61 and time t62. In the example shown in FIG. 13, the sensor 11S is instructed to reduce the frame rate to ¼ at time t61, and the instruction to return the frame rate to the original is issued at time t62. Yes.

  On the other hand, the sensor 11S is instructed to operate in a state of being synchronized with the sensor 11M and maintaining the frame rate. In order to maintain the frame rate, at time t61, the control unit 12 instructs the sensor 11S to interpolate a clock signal in order to maintain the frame rate.

  That is, in this case, at time t61, an instruction to reduce the frame rate to ¼ is issued to the sensor 11M, and the sensor 11S is synchronized with the sensor 11M but not to decrease the frame rate. An instruction to interpolate the vertical synchronization signal is issued.

  From time t61, after two clock signals, the sensor 11S synchronizes with the vertical signal of the sensor 11M once every four times, but the remaining three clock signals other than the clock signal used for the synchronization are: The sensor 11S itself generates.

  At the slave XVS in FIG. 13, a clock signal indicated by a dotted line represents a signal generated by the clock generator 33S by the sensor 11S. As shown in FIG. 13, among the four clock signals of the sensor 11S, three clock signals (in this case, XVS) are signals generated by the clock generation unit 33S, and one clock signal is transmitted from the sensor 11M. This is a signal to be supplied.

  Thus, the sensor 11M and the sensor 11S can be operated asynchronously while being synchronized. In addition, when operating asynchronously while synchronizing, signals required other than the synchronized signal can be interpolated to perform different operations.

  At time t63, an instruction is issued to the sensor 11S to stop interpolation (generation) of the clock signal and complete synchronization with the sensor 11M (described as complete synchronization). As a result, after two click signals from time t63, the sensor 11S is completely synchronized with the sensor 11M.

  In this way, it is possible to allow each sensor 11 to perform processing at a desired frame rate as in the asynchronous mode by changing the method of obtaining the synchronization signal while synchronizing.

  Further, by reducing the frame rate, a low consumption operation can be realized.

<Seventh operation example>
Next, a seventh operation example will be described.

  Refer to FIG. 13 again. At time t62, an instruction to restore the clock signal is issued to the sensor 11M. At time t63, the sensor 11S is instructed to be completely synchronized with the sensor 11M. This time t62 and time t63 are different times.

  From time t62, the sensor 11M returns to the original frequency after two clock signals from the clock signal generated by the sensor 11M, and starts synchronization with the sensor 11S from that point. In addition, from time t63, the sensor 11S starts complete synchronization with the sensor 11M after two clock signals after the clock signal interpolated by the sensor 11S and the clock signal supplied from the sensor 11M are combined.

  Since the frame rates (frequency of the clock signal) of the sensor 11M and the sensor 11S before complete synchronization are different, in order to start complete synchronization at the same time, as shown in FIG. 13, time t62 and time t63 are set. It is necessary to set different times, in other words, different timings for giving instructions to the sensor 11M and the sensor 11S.

  In addition, the timing for issuing a complete synchronization instruction to the sensor 11S needs to be output at an appropriate timing in consideration of the clock signal generated by the sensor 11S itself and the timing at which the clock signal from the sensor 11M is supplied. There is.

  As a seventh operation example, an operation example will be described in which complete synchronization is instructed at the same timing.

  FIG. 14 shows a clock signal (master XVS) generated by the sensor 11M on the master side, a mode of the sensor 11M, a clock signal (slave XVS) supplied or generated to the sensor 11S on the slave side, a mode of the sensor 11S, and communication. It is a figure for demonstrating the relationship of a reflection mask.

  The seventh operation example is the same as the sixth operation except that a communication reflection mask is added to the sixth operation example (FIG. 13). Omitted.

  The communication reflection mask performs a masking process so that the clock signal is not counted in the sensor 11S when the clock signal is generated in the sensor 11S.

  By performing the communication reflection mask, it is possible to perform a count in which only the signal from the sensor 11M is reflected as the clock signal. Therefore, at time t62, an instruction is issued to the sensor 11M and the sensor 11S, and the timing at which the instruction is executed is two clock signals after the clock signal generated by the sensor 11M in both the sensor 11M and the sensor 11S. It becomes.

  In this way, by adding the communication reflection mask process, it is possible to issue an instruction to the sensor 11M and the sensor 11S at the same timing and start the instructed operation at the same timing.

  Similarly to the sixth operation example, in the seventh operation example, the individual sensors 11 in the asynchronous mode are changed to the desired frames by changing the method of obtaining the synchronization signal while synchronizing. It can also be processed at a rate.

  Further, by reducing the frame rate, a low consumption operation can be realized.

<Specific example>
As described above, in the imaging device 10 including the plurality of sensors 11, by applying the present technology, the plurality of sensors 11 can be operated in synchronization or can be operated asynchronously. In addition, it is possible to perform processing at a desired frequency (for example, a frame rate) with each sensor 11 while operating the plurality of sensors 11 in synchronization.

  Here, description will be continued with a specific operation example of the imaging apparatus 10 including the sensor 11-1 and the sensor 11-2 (FIG. 1) performing the above-described operation.

  The sensor 11-1 is a wide sensor, and the sensor 11-2 is a telesensor. The wide sensor is a sensor that captures an image on the wide end side, and captures a relatively wide range of images. The telesensor is a sensor that captures an image on the tele end side, and captures a relatively narrow range of images.

  FIG. 15 shows an example of an image captured when the wide sensor and the telesensor are imaged at the same time. The wide sensor (sensor 11-1) captures an image as shown in FIG. 15A, for example. Referring to FIG. 15A, when imaging is performed by the sensor 11-1, which is a wide sensor, three persons and a background (sky, ground, sun, etc.) are imaged.

  The telesensor (sensor 11-2) captures an image as shown in FIG. 15B, for example. Referring to FIG. 15B, when the image is taken by the sensor 11-2 that is a telesensor, two people are imaged greatly.

  An example of the operation of the imaging apparatus 10 including the wide sensor and the telesensor will be described with reference to FIG. The operation illustrated in FIG. 16 is an operation in which imaging is performed by gradually enlarging a subject, in other words, imaging is performed when moving from the wide end to the tele end side.

  First, an image captured by the sensor 11-1 (wide sensor) is output to a display unit (not shown). While the image from the sensor 11-1 is displayed on the display unit (between time t81 and time t83), the sensor 11-2 (telesensor) is set to the pause mode 52. Assume that zooming at a magnification of 2 is instructed at time t82.

  In this case, the sensor 11-1 operates in the slave mode, and the sensor 11-2 operates in the master mode. The sensor 11-1 is a state where the asynchronous mode 62 of the slave mode control unit 42 is enabled, and the sensor 11-2 is a state where the pause mode 52 of the master mode control unit 41 is enabled. Such a state can be executed by applying an operation corresponding to the first operation example described above, for example.

  Or it can also be realized by operating both the sensor 11-1 and the sensor 11-2 in the master mode. The sensor 11-1 is a state in which the operation mode 51 of the master mode control unit 41 is enabled, and the sensor 11-2 is a state in which the pause mode 52 of the master mode control unit 41 is enabled. Such a state can be executed, for example, by applying the third operation example or the fourth operation example described above.

  In an image captured by a wide sensor, for example, in FIG. 15A, an image with a magnification of 1 × is 1500 × 500 (pix), and an image with a magnification of 2 × is 750 × 250 (pix). Further, an image captured at 750 × 250 (pix) of the wide sensor corresponds to an image of 1500 × 500 (pix) of the telesensor as shown in FIG. 15B.

  Further, in an image captured by the telesensor, when an image with a magnification of 1 × is 1500 × 500 (pix), an image with a magnification of 2 × is 750 × 250 (pix). Therefore, when an image with a magnification of 1 by a wide sensor (image captured at 1500 × 500 (pix)) is used as a reference, the magnification of an image captured at 750 × 250 (pix) of a telesensor is 4 ×. It becomes an image.

  Referring to FIG. 16, when a magnification of 2 is instructed at time t <b> 82, the pause mode 52 of the sensor 11-2 is canceled and the operation mode 51 is shifted to. From time t83, the image captured by the sensor 11-1 and the image captured by the sensor 11-2 are combined, and the combined image is displayed on the display unit.

  The sensor 11-1 (wide sensor) captures an image of 750 × 250 (pix), and the sensor 11-2 (telesensor) captures an image of 1500 × 500 (pix). An image obtained by combining images with different resolutions is displayed on the display unit.

  When the sensor 11-1 operates in the asynchronous mode 62 in the slave mode and the sensor 11-2 operates in the sleep mode 52 in the master mode at a time point before the time t83, the sensor 11- 1 shifts to the synchronization mode 61, and the sensor 11-2 shifts to the operation mode 51.

  Alternatively, when the sensor 11-1 operates in the master mode operation mode 51 and the sensor 11-2 operates in the master mode pause mode 52, the sensor 11-1 is in the master mode at time t83. The operation in the operation mode 51 is maintained, and the sensor 11-2 shifts to the operation mode 51 in the master mode.

  Alternatively, when the sensor 11-1 operates in the master mode operation mode 51 and the sensor 11-2 operates in the master mode pause mode 52, the sensor 11-1 is in the master mode at time t83. The operation in the operation mode 51 is maintained, and the sensor 11-2 shifts to the synchronization mode 61 in the slave mode.

  At time t84, the magnification is further instructed to be four times. At time t85, the sensor 11-1 shifts from the operation mode 51 to the pause mode 52. After time t85, an image captured and output by the sensor 11-2 is displayed on the display unit.

  When the sensor 11-1 operates in the synchronous mode 61 in the slave mode and the sensor 11-2 operates in the operation mode 51 in the master mode at the time before the time t85, the sensor 11- 1 shifts to the sleep mode 52 of the master mode, and the sensor 11-2 maintains the operation mode 51.

  Alternatively, when both the sensor 11-1 and the sensor 11-2 are operating in the master mode operation mode 51, the sensor 11-1 shifts to the master mode pause mode 52 at time t85, and the sensor 11 -2 maintains the operation mode 51 of the master mode.

  Alternatively, the sensor 11-1 may shift to the sleep mode 52 in the master mode, and the sensor 11-2 may shift to the asynchronous mode 62 in the slave mode.

  As described above, the present technology can be applied when the sensor 11-1 and the sensor 11-2 capture images having different angles of view. Further, when switching from the image captured by the sensor 11-1 to the image captured by the sensor 11-2, a process of outputting an image synthesized in the middle by switching the mode of each sensor 11 is performed. It can be easily included.

  As described above, when switching from the image captured by the sensor 11-1 to the image captured by the sensor 11-2, an image obtained by combining the images from the sensor 11-1 and the sensor 11-2 is displayed. By this, it becomes possible to make it difficult to see the step when switching from the image from the wide sensor to the image from the telesensor.

  If the composite image is not included when switching from the image from the wide sensor to the image from the telesensor, the image from the wide sensor is suddenly changed at a certain point in time. Such an abrupt image change may cause discomfort to the user.

  According to this technology, when switching from an image from a wide sensor to an image from a telesensor, it is possible to prevent the image from changing suddenly and to make the user feel uncomfortable. Can be prevented.

  In the specific operation example illustrated in FIG. 16, there is a period in which the sensor 11-1 or the sensor 11-2 is operating in the sleep mode 52. While operating in the sleep mode 52, power consumption can be suppressed, so that power consumption can be reduced.

  Another specific example will be described with reference to FIGS. 17 and 18. The specific examples shown in FIGS. 17 and 18 are different from the specific examples shown in FIGS. 16 and 18 only in the role of the sensor 11 and are the same with respect to mode transitions, and therefore the description thereof will be omitted as appropriate. To do.

  The sensor 11-1 is a high resolution sensor, and the sensor 11-2 is a low resolution sensor. The high resolution sensor is configured by small pixels, and is a sensor that captures an image that looks dark but has high resolution. The low resolution sensor is a sensor that is composed of big pixels and captures an image that looks bright but has a low resolution.

  FIG. 17 shows an example of an image captured when the high resolution sensor and the low resolution sensor capture images at the same time. The high resolution sensor (sensor 11-1) captures an image as shown in FIG. 17A, for example. Referring to FIG. 17A, when imaging is performed by the sensor 11-1, which is a high resolution sensor, an image of 1500 × 500 (pix) is captured.

  The low resolution sensor (sensor 11-2) captures an image as shown in FIG. 17B, for example. Referring to FIG. 17B, when imaging is performed by the sensor 11-2 which is a low resolution sensor, an image of 750 × 250 (pix) is captured.

  The high resolution sensor and the low resolution sensor capture images having the same angle of view, but the number of pixels (resolution) is different. Therefore, as described above, the high-resolution sensor (sensor 11-1) captures a dark image, while the high-resolution sensor (sensor 11-1) captures a dark image, and the low-resolution sensor (sensor 11-2) has a low resolution but a bright image. It is comprised so that it can image.

  An example of the operation in the imaging apparatus 10 including the high resolution sensor and the low resolution sensor will be described with reference to FIG. In the operation shown in FIG. 18, in a dark scene, the gain and exposure time are gradually increased from a sensor with high resolution but low sensitivity (high resolution sensor). When the upper limit is reached, the resolution is low but the sensitivity is high. It is an operation example when switching to a sensor (low resolution sensor).

  From time t91 to time t92, the high resolution sensor (sensor 11-1) is set to the operation mode 51, performs imaging, and an image captured by the sensor 11-1 is displayed on a display unit (not shown). It is in a state. From time t91 to time t92, the high resolution sensor (sensor 11-1) is in a state of performing shooting with gradually increasing gain and exposure time.

  From time t91 to time t92, the low-resolution sensor (sensor 11-2) is set to the pause mode 52 and is in a state where no imaging operation is performed.

  At time t92, the low resolution sensor (sensor 11-2) transits to the operation mode 51. Between time t92 and time t93, the sensor 11-1 and the sensor 11-2 operate, and the image captured by the sensor 11-1 and the image captured by the sensor 11-2 are combined and combined. The image is displayed on the display unit.

  At time t93, the high resolution sensor (sensor 11-1) transitions to the pause mode 52. After time t93, an image captured by the sensor 11-2 is displayed on the display unit.

  Thus, the present technology can be applied when the sensor 11-1 and the sensor 11-2 capture images with different resolutions. Further, when switching from the image captured by the sensor 11-1 to the image captured by the sensor 11-2, a process of outputting an image synthesized in the middle by switching the mode of each sensor 11 is performed. It can be easily included.

  As described above, when switching from the image captured by the sensor 11-1 to the image captured by the sensor 11-2, an image obtained by combining the images from the sensor 11-1 and the sensor 11-2 is displayed. Thus, it is possible to make it difficult to see a change in brightness when switching from an image from a high resolution sensor (dark image) to an image from a low resolution sensor (bright image).

  If the composite image is not included when switching from the image from the high resolution sensor to the image from the low resolution sensor, the image from the high resolution sensor is suddenly changed to the image from the low resolution sensor at a certain point in time. Will be replaced. Such a sudden image change (brightness change) may cause discomfort to the user.

  According to the present technology, it is possible to prevent a sudden change of an image by inserting a composite image when switching from an image from a high resolution sensor to an image from a low resolution sensor, and the user feels uncomfortable. This can be prevented.

  According to the present technology, in an imaging device including a plurality of imaging devices (sensors), the plurality of imaging devices can be operated synchronously or individually asynchronously. Therefore, the necessary processing can be performed by the imaging device as necessary, and a plurality of imaging devices can be used effectively.

  In addition, it is possible to drive only the imaging device necessary for performing the necessary processing. In other words, the imaging device unnecessary for performing the processing can be stopped without being driven. It becomes possible to reduce electric power.

<About recording media>
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  FIG. 19 is a block diagram illustrating an example of a hardware configuration of a computer that executes the series of processes described above according to a program. In a computer, a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003 are connected to each other by a bus 1004. An input / output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

  The input unit 1006 includes a keyboard, a mouse, a microphone, and the like. The output unit 1007 includes a display, a speaker, and the like. The storage unit 1008 includes a hard disk, a nonvolatile memory, and the like. The communication unit 1009 includes a network interface. The drive 1010 drives a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 1001 loads the program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 1001) can be provided by being recorded on the removable medium 1011 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 1008 via the input / output interface 1005 by attaching the removable medium 1011 to the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. In addition, the program can be installed in advance in the ROM 1002 or the storage unit 1008.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

<Application examples to mobile objects>
The technology according to the present disclosure (present technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device that is mounted on any type of mobile body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, and a robot. May be.

  FIG. 20 is a block diagram illustrating a schematic configuration example of a vehicle control system that is an example of a mobile control system to which the technology according to the present disclosure can be applied.

  The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example illustrated in FIG. 20, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, a vehicle exterior information detection unit 12030, a vehicle interior information detection unit 12040, and an integrated control unit 12050. Further, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, a sound image output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.

  The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 includes a driving force generator for generating a driving force of a vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism that adjusts and a braking device that generates a braking force of the vehicle.

  The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a blinker, or a fog lamp. In this case, the body control unit 12020 can be input with radio waves transmitted from a portable device that substitutes for a key or signals from various switches. The body system control unit 12020 receives input of these radio waves or signals, and controls a door lock device, a power window device, a lamp, and the like of the vehicle.

  The vehicle outside information detection unit 12030 detects information outside the vehicle on which the vehicle control system 12000 is mounted. For example, the imaging unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The vehicle outside information detection unit 12030 may perform an object detection process or a distance detection process such as a person, a car, an obstacle, a sign, or a character on a road surface based on the received image.

  The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of received light. The imaging unit 12031 can output an electrical signal as an image, or can output it as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared rays.

  The vehicle interior information detection unit 12040 detects vehicle interior information. For example, a driver state detection unit 12041 that detects a driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the vehicle interior information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated or it may be determined whether the driver is asleep.

  The microcomputer 12051 calculates a control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside / outside the vehicle acquired by the vehicle outside information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit A control command can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, following traveling based on inter-vehicle distance, vehicle speed maintaining traveling, vehicle collision warning, or vehicle lane departure warning. It is possible to perform cooperative control for the purpose.

  Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform cooperative control for the purpose of automatic driving that autonomously travels without depending on the operation.

  Further, the microcomputer 12051 can output a control command to the body system control unit 12030 based on information outside the vehicle acquired by the vehicle outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamp according to the position of the preceding vehicle or the oncoming vehicle detected by the outside information detection unit 12030, and performs cooperative control for the purpose of anti-glare, such as switching from a high beam to a low beam. It can be carried out.

  The sound image output unit 12052 transmits an output signal of at least one of sound and image to an output device capable of visually or audibly notifying information to a vehicle occupant or the outside of the vehicle. In the example of FIG. 20, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

  FIG. 21 is a diagram illustrating an example of the installation position of the imaging unit 12031.

  In FIG. 21, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

  The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as a front nose, a side mirror, a rear bumper, a back door, and an upper part of a windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided in the front nose and the imaging unit 12105 provided in the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The imaging units 12102 and 12103 provided in the side mirror mainly acquire an image of the side of the vehicle 12100. The imaging unit 12104 provided in the rear bumper or the back door mainly acquires an image behind the vehicle 12100. The imaging unit 12105 provided on the upper part of the windshield in the passenger compartment is mainly used for detecting a preceding vehicle or a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

  FIG. 21 shows an example of the imaging range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively, and the imaging range 12114 The imaging range of the imaging part 12104 provided in the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, an overhead image when the vehicle 12100 is viewed from above is obtained.

  At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

  For example, the microcomputer 12051, based on the distance information obtained from the imaging units 12101 to 12104, the distance to each three-dimensional object in the imaging range 12111 to 12114 and the temporal change in this distance (relative speed with respect to the vehicle 12100). In particular, it is possible to extract, as a preceding vehicle, a solid object that travels at a predetermined speed (for example, 0 km / h or more) in the same direction as the vehicle 12100, particularly the closest three-dimensional object on the traveling path of the vehicle 12100. it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in advance before the preceding vehicle, and can perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. Thus, cooperative control for the purpose of autonomous driving or the like autonomously traveling without depending on the operation of the driver can be performed.

  For example, the microcomputer 12051 converts the three-dimensional object data related to the three-dimensional object to other three-dimensional objects such as a two-wheeled vehicle, a normal vehicle, a large vehicle, a pedestrian, and a utility pole based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 is connected via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration or avoidance steering via the drive system control unit 12010, driving assistance for collision avoidance can be performed.

  At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the captured images of the imaging units 12101 to 12104. Such pedestrian recognition is, for example, whether or not the user is a pedestrian by performing a pattern matching process on a sequence of feature points indicating the outline of an object and a procedure for extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras. It is carried out by the procedure for determining. When the microcomputer 12051 determines that there is a pedestrian in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 has a rectangular contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to be superimposed and displayed. Moreover, the audio | voice image output part 12052 may control the display part 12062 so that the icon etc. which show a pedestrian may be displayed on a desired position.

  In this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  In addition, the effect described in this specification is an illustration to the last, and is not limited, Moreover, there may exist another effect.

  The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

In addition, this technique can also take the following structures.
(1)
An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A generator for generating a clock signal;
An imaging apparatus comprising: a control unit that controls each unit in a first mode that operates in synchronization with another imaging apparatus or in a second mode that operates asynchronously.
(2)
In the case of the first mode, a clock signal supplied from the other imaging device is supplied to the signal processing unit,
The imaging device according to (1), wherein the generation unit is in a stopped state.
(3)
In the case of the second mode, the generation unit generates the clock signal,
The imaging apparatus according to (1) or (2), wherein the signal processing unit processes the signal based on the clock signal generated by the generation unit.
(4)
The imaging apparatus according to any one of (1) to (3), further including a third mode for supplying the other imaging apparatus with the clock signal generated by the generation unit.
(5)
The imaging apparatus according to any one of (1) to (4), further including a fourth mode in which at least one of the imaging unit, the signal processing unit, and the generation unit is stopped.
(6)
In the second mode, a clock signal having a frequency different from that of the clock signal of the other imaging device is generated. The imaging device according to any one of (1) to (5)
The imaging apparatus according to any one of (1) to (6), wherein in the first mode, the clock signal is interpolated so that the clock signal has a predetermined frame rate.
(8)
The imaging device according to (7), wherein the interpolated clock signal is masked.
(9)
In an imaging method of an imaging device including an imaging unit,
Processing the signal from the imaging unit;
Generate a clock signal,
An imaging method including a step of controlling each unit in a first mode that operates in synchronization with another imaging device or in a second mode that operates asynchronously.
(10)
In an imaging device provided with an imaging unit,
Processing the signal from the imaging unit;
Generate a clock signal,
A program for executing processing including a step of controlling each unit in a first mode that operates in synchronization with another imaging apparatus or a second mode that operates in an asynchronous manner.
(11)
An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
A second imaging device operating in slave mode,
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode includes a sleep mode,
The slave mode includes an imaging mode that includes a synchronous mode that operates in synchronization with the first imaging device and an asynchronous mode that operates asynchronously.
(12)
The imaging device according to (11), wherein the second imaging device is operating in the asynchronous mode when the first imaging device is operating in the pause mode.
(13)
When the second imaging device is operating in the asynchronous mode, the first imaging device and the second imaging device are operating based on different clock signals (11) or (12) The imaging device described in 1.
(14)
The first imaging device and the second imaging device have the master mode and the slave mode, respectively.
The imaging apparatus according to any one of (11) to (13), wherein both the first imaging apparatus and the second imaging apparatus operate in a master mode.
(15)
The second imaging device operates in the synchronous mode, and an image at a predetermined frame rate is obtained by interpolating the clock signal generated by the generation unit to the clock signal supplied from the first imaging device. The imaging device according to any one of (11) to (14).
(16)
The imaging device according to (15), wherein the interpolated clock signal is masked.
(17)
The first imaging device images the wide end side,
The imaging device according to any one of (11) to (16), wherein the second imaging device images the tele end side.
(18)
The first imaging device captures a high-resolution image,
The imaging device according to any one of (11) to (16), wherein the second imaging device captures a low-resolution image.
(19)
An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
In an imaging method of an imaging device comprising: a second imaging device operating in slave mode;
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode operates in an operation mode or a sleep mode,
The slave mode is an imaging method including a step of operating in a synchronous mode operating in synchronization with the first imaging device or an asynchronous mode operating asynchronously.
(20)
An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
An imaging device comprising: a second imaging device operating in slave mode;
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode operates in an operation mode or a sleep mode,
The slave mode is a program for executing processing including a step of operating in a synchronous mode that operates in synchronization with the first imaging device or an asynchronous mode that operates asynchronously.

  DESCRIPTION OF SYMBOLS 10 Imaging device, 11 Sensor, 12 Control part, 31 Imaging part, 32 Signal processing part, 33 Clock generation part, 34 Control part, 41 Master mode control part, 42 Slave mode control part, 51 Operation mode, 52 Pause mode, 61 Synchronous mode, 62 Asynchronous mode

Claims (20)

An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A generator for generating a clock signal;
An imaging apparatus comprising: a control unit that controls each unit in a first mode that operates in synchronization with another imaging apparatus or in a second mode that operates asynchronously. In the case of the first mode, a clock signal supplied from the other imaging device is supplied to the signal processing unit,
The imaging device according to claim 1, wherein the generation unit is in a stopped state. In the case of the second mode, the generation unit generates the clock signal,
The imaging device according to claim 1, wherein the signal processing unit processes the signal based on the clock signal generated by the generation unit. The imaging apparatus according to claim 1, further comprising a third mode for supplying the other imaging apparatus with the clock signal generated by the generation unit. The imaging device according to claim 1, further comprising a fourth mode in which at least one of the imaging unit, the signal processing unit, and the generation unit is stopped. The imaging apparatus according to claim 1, wherein in the second mode, a clock signal having a frequency different from that of the clock signal of the other imaging apparatus is generated. The imaging apparatus according to claim 1, wherein in the first mode, the clock signal is interpolated so that the clock signal has a predetermined frame rate. The imaging apparatus according to claim 7, wherein the interpolated clock signal is masked. In an imaging method of an imaging device including an imaging unit,
Processing the signal from the imaging unit;
Generate a clock signal,
An imaging method including a step of controlling each unit in a first mode that operates in synchronization with another imaging device or in a second mode that operates asynchronously. In an imaging device provided with an imaging unit,
Processing the signal from the imaging unit;
Generate a clock signal,
A program for executing processing including a step of controlling each unit in a first mode that operates in synchronization with another imaging apparatus or a second mode that operates in an asynchronous manner. An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
A second imaging device operating in slave mode,
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode includes a sleep mode,
The slave mode includes an imaging mode that includes a synchronous mode that operates in synchronization with the first imaging device and an asynchronous mode that operates asynchronously. The imaging apparatus according to claim 11, wherein when the first imaging apparatus is operating in the pause mode, the second imaging apparatus is operating in the asynchronous mode. The imaging device according to claim 11, wherein when the second imaging device is operating in the asynchronous mode, the first imaging device and the second imaging device are operated based on different clock signals. . The first imaging device and the second imaging device have the master mode and the slave mode, respectively.
The imaging device according to claim 11, wherein both the first imaging device and the second imaging device operate in a master mode. The second imaging device operates in the synchronous mode, and performs imaging at a predetermined frame rate by interpolating the clock signal generated by the generation unit to the clock signal supplied from the first imaging device. The imaging device according to claim 11. The imaging apparatus according to claim 15, wherein the interpolated clock signal is masked. The first imaging device images the wide end side,
The imaging device according to claim 11, wherein the second imaging device images a tele end side. The first imaging device captures a high-resolution image,
The imaging device according to claim 11, wherein the second imaging device images a low-resolution image. An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
In an imaging method of an imaging device comprising: a second imaging device operating in slave mode;
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode operates in an operation mode or a sleep mode,
The slave mode is an imaging method including a step of operating in a synchronous mode operating in synchronization with the first imaging device or an asynchronous mode operating asynchronously. An imaging unit;
A signal processing unit for processing a signal from the imaging unit;
A plurality of imaging devices including a generation unit that generates a clock signal,
A first imaging device operating in a master mode among the plurality of imaging devices;
An imaging device comprising: a second imaging device operating in slave mode;
The second imaging device is supplied with a clock signal generated by the first imaging device,
The master mode operates in an operation mode or a sleep mode,
The slave mode is a program for executing processing including a step of operating in a synchronous mode that operates in synchronization with the first imaging device or an asynchronous mode that operates asynchronously.

Images