[go: up one dir, main page]

WO2024048069A1 - Dispositif de traitement d'informations, procédé de traitement d'informations et programme - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et programme Download PDF

Info

Publication number
WO2024048069A1
WO2024048069A1 PCT/JP2023/024971 JP2023024971W WO2024048069A1 WO 2024048069 A1 WO2024048069 A1 WO 2024048069A1 JP 2023024971 W JP2023024971 W JP 2023024971W WO 2024048069 A1 WO2024048069 A1 WO 2024048069A1
Authority
WO
WIPO (PCT)
Prior art keywords
blur correction
information processing
amount
shutter speed
correction amount
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/024971
Other languages
English (en)
Japanese (ja)
Inventor
壮平 岡田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2024543833A priority Critical patent/JPWO2024048069A1/ja
Publication of WO2024048069A1 publication Critical patent/WO2024048069A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations

Definitions

  • the present technology relates to an information processing device, an information processing method, and a program.
  • a first image stabilization unit performs image stabilization processing on the image data
  • a second image stabilization unit uses the image stabilization process output from the first image stabilization unit.
  • the image corrected by the electronic blur correction function will be an image with motion blur when the imaging device is moving, and will be an image without motion blur when the imaging device is stationary, resulting in an output result.
  • the larger the amount of movement of the imaging device the more unnatural the image becomes.
  • the present technology has been developed in view of these points, and aims to provide an information processing device, an information processing method, and a program that can suppress the sense of discomfort that occurs in captured images during electronic blur correction.
  • a first technique is to use one of a value related to blur correction and a shutter speed in an imaging device that is equipped with an electronic blur correction function that corrects blur in image data and a shutter speed control function.
  • This is an information processing device including an adjustment section that adjusts.
  • the second technology is an information processing method for adjusting either a value related to blur correction or a shutter speed in an imaging device equipped with an electronic blur correction function for correcting blur in image data and a shutter speed control function. be.
  • a third technology provides an information processing method for adjusting either a value related to blur correction or a shutter speed in an imaging device equipped with an electronic blur correction function for correcting blur in image data and a shutter speed control function. It is a program that is executed by a computer.
  • FIG. 2 is a diagram showing a first example for explaining the problem of the present technology. It is a figure which shows the 2nd example for demonstrating the problem of this technique. It is a figure which shows the 3rd example for demonstrating the problem of this technique. It is a figure which shows the 4th example for demonstrating the problem of this technique.
  • FIG. 1 is a block diagram showing the configurations of an imaging device 10 and an information processing device 300 in a first embodiment. 1 is an external view showing the configuration of a lens head 100.
  • FIG. FIG. 3 is an explanatory diagram of a blur correction method. 3 is a flowchart showing processing of the information processing device 300. It is a graph showing adjustment data in a 1st embodiment.
  • FIG. 1 is a block diagram showing the configurations of an imaging device 10 and an information processing device 300 in a first embodiment. 1 is an external view showing the configuration of a lens head 100.
  • FIG. FIG. 3 is an explanatory diagram of a blur correction method. 3 is a flowchart showing
  • FIG. 3 is an explanatory diagram when an image is captured by an imaging device 10 that performs processing by an information processing device 300.
  • FIG. FIG. 2 is a block diagram showing the configurations of an imaging device 10 and an information processing device 300 in a second embodiment. It is a graph which shows the adjustment data in 2nd Embodiment.
  • FIG. 3 is a block diagram showing the configurations of an imaging device 10 and an information processing device 300 in a third embodiment. It is a graph which shows the adjustment data in 3rd Embodiment. It is a block diagram showing the composition of imaging device 10 and information processing device 300 in a modification. It is a graph which shows the modification of the data for correction.
  • FIG. 1 is a first example for explaining the problem of imaging a stationary subject P with an imaging device CAM without functioning a blur correction function (or without a blur correction function).
  • Motion blur refers to the movement of the subject P or the imaging device CAM, which causes the subject P in the captured image to blur in a flowing manner, or to cause an afterimage.
  • FIG. 1B when imaging is performed while the imaging device CAM is panning clockwise from the state of FIG. 1A, the blur caused by the movement of the imaging device CAM is not corrected and compared to the state of FIG. 1A.
  • the subject P is located on the left side in the captured image, and the composition has changed. Furthermore, even if the subject P is stationary, motion blur of the subject P occurs in the captured image due to the movement of the imaging device CAM.
  • the subject P when the imaging device CAM is panned clockwise by 90 degrees, the subject P will be located on the left side of the imaging device CAM in the imaging direction of the imaging device CAM (the direction in which the lens faces).
  • the imaging direction of the imaging device CAM the direction in which the lens faces.
  • the composition of the captured image will change if the image capture device CAM moves, and furthermore, motion blur will occur in the subject P due to the movement of the image capture device CAM. Therefore, there is no sense of discomfort in the captured image.
  • FIG. 2 is a second example for explaining the problem of imaging a stationary subject P with the imaging device CAM using blur correction. Note that the imaging conditions such as the positions of the imaging device CAM and the subject P are the same as in the first example.
  • FIG. 2B when imaging is performed while the imaging device CAM is panning clockwise from the state of FIG. 2A, motion blur of the subject P occurs in the captured image due to the movement of the imaging device CAM.
  • changes in the composition are suppressed by performing blur correction to cancel the movement of the imaging device CAM, and the subject P is located on the right side in the captured image, as in FIG. 2A.
  • the length of the arrow b in FIG. 2B indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image due to blur correction.
  • the subject P when the imaging device CAM is panned clockwise by 90 degrees, the subject P will be located on the left side of the imaging device CAM in the imaging direction of the imaging device CAM (the direction in which the lens faces). If you capture an image after panning the image capturing device CAM clockwise to 90 degrees, shake correction is applied to cancel the movement of the image capturing device CAM, suppressing changes in the composition, and subject P in the captured image. is located on the right side as in FIG. 2A.
  • the length of the arrow c in FIG. 2C indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image due to blur correction.
  • FIG. 3 is a third example for explaining the problem in which blur correction is performed to eliminate the discomfort caused by the blur correction shown in FIG. 2, and the shutter is released at a faster shutter speed to capture an image.
  • the imaging conditions such as the positions of the imaging device 10 and the subject P are the same as in the first and second examples.
  • FIG. 3B if an image is captured while the imaging device CAM is panning clockwise from the state shown in FIG.
  • the motion blur of the subject P that occurs due to this can be suppressed.
  • shake correction is performed to cancel the movement of the imaging device CAM, thereby suppressing changes in the composition, and the subject P is located on the right side in the captured image, as in FIG. 3A.
  • the length of the arrow b in FIG. 3B indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image due to blur correction.
  • the subject P when the imaging device CAM is panned clockwise by 90 degrees, the subject P will be located on the left side of the imaging device CAM in the imaging direction of the imaging device CAM (the direction in which the lens faces). If you capture an image after panning the image capturing device CAM clockwise to 90 degrees, shake correction is applied to cancel the movement of the image capturing device CAM, suppressing changes in the composition, and subject P in the captured image. is located on the right side as in FIG. 3A.
  • the length of the arrow c in FIG. 3C indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image due to blur correction.
  • the brightness of the captured image can be corrected by adjusting the ND (Neutral Density), iris, gain, etc.
  • ND Neutral Density
  • iris iris
  • gain etc.
  • these image processes affect not only the brightness but also the image quality, there is a problem in that increasing the gain increases noise and deteriorates the image quality, as shown in FIG.
  • the image quality desired by the user is not achieved.
  • adjusting the ND or iris is not effective in adjusting the image quality, so it is necessary to increase the gain, but there is also the problem that increasing the gain increases noise in the captured image.
  • the imaging device 10 includes a lens head 100, a camera body 200, and an information processing device 300.
  • the lens head 100 includes a lens 101 including an optical lens for condensing light from a subject onto an image sensor and a focus lens for adjusting the focus, and a lens information storage section that holds information regarding the lens 101. 102, a lens drive driver (not shown) for moving the lens 101, and the like.
  • An optical image of the subject obtained through the lens 101 is formed on an image sensor in the imaging unit 201.
  • the lens head 100 is constructed as a separate member from the camera body 200 and is attached to the camera body 200. Therefore, as shown in FIG. 6, it is possible to replace the lens head 100 attached to the camera body 200 with another lens head 100B.
  • the lens head 100 is fixed to the camera body 200 by, for example, screws. Since the lens head 100 is replaceable, it can be traded alone as a product. If the lens head 100 is to be traded, it may be traded as a set with a coaxial cable and a board.
  • the lens information storage unit 102 is composed of, for example, an EPROM (Erasable Programmable Read Only Memory).
  • the lens information storage unit 102 stores in advance various information regarding the lens 101, such as a lens center deviation value which is a unique value of each lens, lens shading correction data which is a fixed value of the lens type, distortion correction data, and lens type information. ing.
  • the camera body 200 Since information regarding lenses is stored in the lens information storage section 102, the camera body 200 reads information regarding the lenses from the lens information storage section 102 after replacing the lens head 100 or when starting up the imaging device 10. Then, the camera body 200 recognizes the attached lens 101 based on the information, automatically changes to an operation mode suitable for the lens 101, and performs imaging.
  • FIG. 6 shows an example in which the lens 101 and the lens 101C attached to the lens head 100 are interchangeable, and furthermore, the lens 101B and the lens 101C attached to the lens head 100B are interchangeable.
  • Various lenses such as a wide-angle lens, a narrow-angle lens, and a fisheye lens can be attached to the lens head 100.
  • a user can rewrite the information stored in the lens information storage section 102 by connecting the lens head 100 to a device such as a personal computer. Therefore, when the user replaces the lens 101, the user connects the lens head 100 to a personal computer or the like and rewrites the information in the lens information storage section 102 with information about the newly attached lens. Thereby, the camera body 200 can recognize a newly attached lens, automatically change to an operation mode suitable for the lens, and perform imaging.
  • the camera body 200 includes an imaging section 201, a sensor section 202, an exposure control section 203, a blur correction amount generation section 204, a blur correction amount adjustment section 205, a blur correction processing section 206, a development processing section 207, an output processing section 208, and a communication section. 209.
  • An information processing device 300 is configured by an exposure control section 203, a blur correction amount generation section 204, a blur correction amount adjustment section 205, and a blur correction processing section 206.
  • the imaging unit 201 includes an imaging element and a shutter mechanism that photoelectrically converts incident light from a subject obtained through the lens 101 into an amount of charge to generate image data.
  • the imaging unit 201 outputs the generated image data to the exposure control unit 203 and the blur correction processing unit 206.
  • As the image sensor a CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor), or the like is used.
  • the shutter mechanism drives the shutter based on the shutter speed acquired from the exposure control unit 203 and adjusts the exposure time.
  • the shutter mechanism may be of any type, such as a mechanical shutter type or an electronic shutter type.
  • the sensor unit 202 is a variety of sensors that detect the angular velocity and acceleration of the movement of the imaging device 10. Examples of the sensor include an IMU (Inertial Measurement Unit), an inertial sensor (an acceleration sensor in two or three axis directions, an angular velocity sensor, a gyro sensor), and the like. A plurality of sensors may be used together as the sensor unit 202.
  • the sensor unit 202 outputs sensing results to the blur correction amount generation unit 204. Note that the sensor unit 202 may not be included in the camera body 200 but may be configured as an external device and transmit sensing results to the camera body 200 by wired or wireless communication.
  • the exposure control unit 203 sets the shutter speed to adjust the exposure time based on user input and functions provided in the imaging device 10 in advance. In this way, the camera body 200 has a shutter speed control function.
  • the exposure control unit 203 outputs the set shutter speed to the imaging unit 201 and the blur correction amount adjustment unit 205.
  • the blur correction amount generation unit 204 uses the sensing results obtained from the sensor unit 202 to generate a blur correction amount based on a general blur correction algorithm.
  • the blur correction amount generation unit 204 outputs the blur correction amount to the blur correction amount adjustment unit 205.
  • the amount of blur correction is the amount of movement of the cutting position of image data in electronic blur correction that corrects blur by cutting out image data of a predetermined size from the image data output from the imaging unit 201 according to the degree of blur. It is.
  • the blur correction amount corresponds to the value related to blur correction in the claims.
  • the blur correction amount adjustment unit 205 holds adjustment data prepared in advance, and adjusts the blur correction amount based on the adjustment data and the shutter speed obtained from the exposure control unit 203.
  • the blur correction amount adjustment unit 205 outputs the adjusted blur correction amount (referred to as an adjusted blur correction amount) to the blur correction processing unit 206.
  • the blur correction processing unit 206 performs blur correction processing using an electronic blur correction method that cuts out an image of a predetermined size from the image data based on the adjusted blur correction amount. As shown in FIG. 7, the blur correction processing unit 206 performs lens distortion removal and rotation processing on the input image data and rotation vector mesh, respectively, and pastes the image data onto the rotation vector mesh, for example, 4K ( Blurring is corrected by cutting out full HD (horizontal 1920 x vertical 1080) image data from image data (horizontal 3840 x vertical 2160). Note that the blur correction process may be performed using a general method, such as a method using sensing results from a gyro sensor or IMU as the sensor unit 202, or a method in which alignment with past frames is performed by image processing.
  • a general method such as a method using sensing results from a gyro sensor or IMU as the sensor unit 202, or a method in which alignment with past frames is performed by image processing.
  • the development processing unit 207 performs predetermined image processing, such as gain correction processing, white balance processing, color adjustment processing, Debayer processing, and aspect conversion processing, on the image data subjected to blur correction processing by the blur correction processing unit 206. etc.
  • the output processing unit 208 performs output processing such as compression encoding and output format conversion on the image data output from the development processing unit 207.
  • output processing such as compression encoding and output format conversion on the image data output from the development processing unit 207.
  • MPEG Motion Picture Experts Group
  • JPEG Joint Photographic Experts Group
  • the output format include MDMI (High-Definition Multimedia Interface) and SDI (Serial Digital Interface).
  • the communication unit 209 is a communication module that performs communication with other external devices, the Internet, and the like.
  • the communication method may be either wired or wireless, and specifically, cellular communication, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), Wi-Fi, Bluetooth (registered trademark), NFC (Near Field Communication), Ethernet (registered trademark), HDMI (registered trademark) (High-Definition Multimedia Interface), USB (Universal Serial Bus), etc.
  • the image data output from the output processing unit 208 is transmitted as a captured image to an external device such as a personal computer via the communication unit 209 and a network.
  • the imaging device 10 can also transmit captured images to an external device in real time. Note that the imaging device 10 does not need to include the communication unit 209.
  • the imaging device 10 may include a storage medium, a display section, a control section, an input section, and the like.
  • the storage medium stores captured images, and is composed of a drive device for a portable recording medium such as a magnetic tape or an optical disk, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.
  • the storage medium may be an external device connected to imaging device 10 via a wired or wireless connection.
  • the image data output from the output processing unit 208 may be stored as a captured image in a storage medium.
  • the display unit displays captured images, through images, GUI (Graphical User Interface), and the like.
  • the display unit is realized by, for example, a monitor display composed of an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an organic EL (Electro Luminescence) panel, or an EVF (Electronic View Finder).
  • the display unit may be an external device connected to the imaging device 10 through a wired or wireless connection.
  • the control unit is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like.
  • the CPU controls the entire imaging device 10 and each part by executing various processes and issuing commands according to programs stored in the ROM.
  • the input unit is for the user to input various instructions and the like to the imaging device 10.
  • a control signal corresponding to the input is generated, and the imaging device 10 performs various processes corresponding to the control signal.
  • Input units include a release button for issuing a release instruction, physical buttons for various operations, a touch panel, a touch screen integrated with a display unit, and the like.
  • the imaging device 10 and the information processing device 300 are configured as described above.
  • the information processing device 300 may be configured as a dedicated device using hardware having that function, or the imaging device 10 may have the function as the information processing device 300 by executing a program, and the information processing method may be realized.
  • the program may be installed in the imaging device 10 in advance, or may be downloaded or distributed on a storage medium, and installed by the user in the imaging device 10.
  • the information processing device 300 may include the imaging unit 201.
  • the blur correction amount adjustment unit 205 adjusts the blur correction amount based on the shutter speed.
  • the blur correction amount generation unit 204 generates a target amount of blur correction based on the sensing results received from the sensor unit 202.
  • the blur correction amount generation unit 204 outputs the target amount of the blur correction amount to the blur correction amount adjustment unit 205.
  • the target amount of blur correction amount is a value of 100% (maximum amount) of the blur correction amount calculated based on the sensing results.
  • step S12 the exposure control unit 203 sets the shutter speed.
  • the exposure control section 203 outputs the shutter speed to the blur correction amount adjustment section 205.
  • step S11 and step S12 may be performed in the reverse order, or may be performed simultaneously or almost simultaneously.
  • the blur correction amount adjustment unit 205 calculates the minimum amount of the blur correction amount by multiplying the target amount of the blur correction amount by a predetermined magnification (0x to 1x).
  • the predetermined magnification may be set in advance, for example, at the time of designing or manufacturing the imaging device 10 or the information processing device 300, or may be set by the user.
  • the blur correction amount adjustment unit 205 adjusts the blur correction amount according to the shutter speed based on the blur correction amount (target amount and minimum amount), shutter speed, and adjustment data shown in FIG. 9 prepared in advance. adjust.
  • the blur correction amount adjustment unit 205 outputs the adjusted blur correction amount to the blur correction processing unit 206.
  • the blur correction amount when the shutter speed is equal to or higher than the first speed, the blur correction amount is increased to reach the target amount than when the shutter speed is less than the first speed. Adjust to. Further, when the shutter speed is less than or equal to a second speed that is smaller than the first speed, the blur correction amount is adjusted to be smaller than when the shutter speed is larger than the second speed so that it becomes the minimum amount. Furthermore, when the shutter speed is higher than the second speed and lower than the first speed, the blur correction amount is adjusted so as to approach the target amount as the shutter speed becomes faster. Note that the characteristics of the adjustment data shown in FIG. 9 may change not linearly but exponentially or quadratically, and may be adjusted according to the use case.
  • the shutter speed is increased, and in order to weakly suppress the motion blur of the subject, the shutter speed is slowed. Further, in order to strongly suppress changes in the composition, the adjusted blur correction amount is increased, and in order to weakly suppress changes in the composition, the adjusted blur correction amount is decreased.
  • the first speed of the shutter speed is "1/250" and the second speed is "1/200", but this is just an example, and the present technology is limited to these values. It's not a thing.
  • the blur correction amount adjustment unit 205 adjusts the blur correction amount according to the shutter speed set by the exposure control unit 203, and determines the adjusted blur correction amount.
  • step S14 the blur correction processing unit 206 performs blur correction processing on the image data based on the adjusted blur correction amount.
  • the image data that has been subjected to blur correction processing is processed by a development processing section 207 and an output processing section 208, and is output as a captured image.
  • the processing of the information processing device 300 in the first embodiment is performed as described above.
  • the composition does not change due to blur correction, the captured image will feel unnatural.
  • the adjusted blur correction amount approaches the minimum amount and becomes small, and changes in the composition are also suppressed weakly. Therefore, the sense of discomfort in the captured image can be suppressed.
  • imaging is performed without moving the imaging device 10 in a state where the subject P is located on the right side of the imaging device 10 in the imaging direction of the imaging device 10 (the direction in which the lens faces). Then, the subject P is located on the right side in the captured image. This is similar to FIG. 2A.
  • the subject P since blur correction is performed to cancel the movement of the imaging device 10, the subject P, whose composition has changed due to the movement of the imaging device 10 and has moved to the left, is corrected so that it moves to the right. Change is suppressed.
  • the adjusted blur correction amount approaches the minimum amount and becomes smaller, so the change in composition is weakly suppressed, and the position of the subject P after blur correction in FIG. 10B is located on the left side compared to FIG. 2B. become.
  • the length of the arrow b' in FIG. 10B indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image by blur correction.
  • the amount of movement of the subject P indicated by the length of the arrow b' is shorter than the amount of movement of the subject P due to suppressed changes in the composition in FIG. 2B, which is indicated by the arrow b. That is, in FIG. 10B, changes in the composition are suppressed weaker than in FIG. 2B.
  • the subject P when the imaging device 10 is panned clockwise by 90 degrees, the subject P will be located on the left side of the imaging device 10 in the imaging direction of the imaging device 10 (the direction in which the lens faces).
  • blur correction is performed to cancel the movement of the imaging device 10.
  • the shutter speed is slowed down, the adjusted blur correction amount approaches the minimum amount and becomes smaller, so the change in composition is weakly suppressed, and the position of the subject P after blur correction in FIG. 10C is on the left side compared to FIG. 2C. become.
  • the length of the arrow c' in FIG. 10C indicates the amount of movement of the position of the subject P due to suppressing changes in the composition of the captured image by blur correction.
  • the amount of movement of the subject P indicated by the length of the arrow c' is shorter than the amount of movement of the subject P due to suppressed changes in the composition in FIG. 2C, which is indicated by the arrow c. That is, in FIG. 10C, changes in the composition are suppressed weaker than in FIG. 2C.
  • the adjusted blur correction amount approaches the minimum amount and becomes small, so changes in the composition are also suppressed. Therefore, the sense of discomfort in the captured image can be suppressed.
  • the adjusted blur correction amount will approach the target amount and increase, which will also strongly suppress changes in the composition. Therefore, in this case as well, the sense of discomfort in the captured image can be suppressed.
  • the shutter speed is brought closer to the second speed.
  • the adjusted blur correction amount approaches the minimum amount. Further, when the shutter speed is lower than the second speed, the adjusted blur correction amount becomes the minimum value.
  • the present technology is useful when the movement or vibration of the imaging device 10 is large because it can correct blur caused by movement of the imaging device 10 and suppress changes in composition while suppressing the sense of discomfort in the captured image.
  • the present technology is useful when mounted on a moving motorcycle to capture images of the front and rear of the moving motorcycle, the driver's face, etc.
  • the moving object may be other than a motorcycle, such as a bicycle, a car, a drone, a ship, or a person.
  • the imaging device 10 is not limited to one that is mounted on a moving object, but may be a digital camera, a single-lens reflex camera, a camcorder, a professional camera, a professional imaging device, etc. that is held and used by a person.
  • Second embodiment> [2-1. Configuration of imaging device 10 and information processing device 300 and processing by information processing device 300] Next, a second embodiment of the present technology will be described with reference to FIG. 11. Regarding the configurations of the imaging device 10 and the information processing device 300, only the processing units that perform processing different from those in the first embodiment will be described.
  • the blur correction amount generation unit 204 uses the sensing results obtained from the sensor unit 202 to generate a blur correction amount based on a general blur correction algorithm.
  • the blur correction amount generation unit 204 outputs the blur correction amount to the exposure adjustment unit 210 and the blur correction processing unit 206.
  • the exposure adjustment unit 210 determines the adjusted shutter speed by adjusting the shutter speed based on the blur correction amount obtained from the blur correction amount generation unit 204 and the adjustment data shown in FIG. 12 prepared in advance.
  • the exposure adjustment section 210 outputs the adjusted shutter speed to the imaging section 201.
  • the imaging unit 201 performs shutter control based on the adjusted shutter speed.
  • the second embodiment is characterized in that the exposure adjustment section 210 adjusts the shutter speed based on the amount of blur correction in this way.
  • the lower limit value of the shutter speed (that is, the maximum exposure time) is determined according to the amount of blur correction, thereby placing constraints on the exposure control circuit.
  • the lower limit value of the shutter speed is calculated from the blur correction amount based on the adjustment data shown in FIG.
  • the shutter speed is adjusted using this lower limit value as the shutter speed limit value.
  • the limit on the vertical axis (lower limit of shutter speed) is determined by the capability value of the imaging device 10. In the case of videos, the speed is usually determined by the frame rate.
  • the lower limit value is basically a value determined in advance by the camera designer, but may be determined by the user.
  • the shutter speed is made slower than when the amount of blur correction is less than the first threshold to reach the lower limit value. Adjust as follows. Further, when the amount of blur correction is less than or equal to a second threshold value which is smaller than the first threshold value, the shutter speed is adjusted to be faster than when the amount of blur correction is greater than the second threshold value so that it is unlimited. Further, when the amount of blur correction is greater than or equal to the second threshold and less than or equal to the first threshold, the shutter speed is adjusted so as to approach the lower limit as the amount of blur correction increases. Note that the characteristics of the adjustment data shown in FIG. 12 may change not linearly but exponentially or quadratically, and may be adjusted according to the use case.
  • the blur correction processing unit 206 performs blur correction processing using an electronic blur correction method that cuts out an image of a predetermined size from the image data based on the blur correction amount obtained from the blur correction amount generation unit 204.
  • the blur correction method is the same as in the first embodiment.
  • the image data that has been subjected to blur correction processing is processed by a development processing section 207 and an output processing section 208, and is output as a captured image.
  • imaging device 10 and the information processing device 300 are the same as in the first embodiment.
  • the processing of the information processing device 300 in the second embodiment is performed as described above.
  • the second embodiment for example, when the amount of blur correction is made small in order to weakly suppress changes in composition, the shutter speed is slowed and motion blur is suppressed weakly. Therefore, similar to the first embodiment, it is possible to suppress the sense of discomfort in the captured image.
  • the shutter speed will become faster and motion blur will be strongly suppressed. Therefore, similar to the first embodiment, it is possible to suppress the sense of discomfort in the captured image. In this way, prioritize the amount of image stabilization, and if a large amount of image stabilization is required, increase the shutter speed and shorten the exposure time, and if a small amount of image stabilization is required, slow the shutter speed and take the necessary amount of exposure. It is also possible to set the time to be reserved. In this case, when the imaging device 10 moves, the exposure time changes and the brightness changes, so it is desirable to adjust the captured image to a constant brightness using ND, iris, gain, etc.
  • the exposure control unit 203 sets the shutter speed to adjust the exposure time based on user input and functions provided in the imaging device 10 in advance.
  • the exposure control unit 203 outputs the set shutter speed to the imaging unit 201 and the blur correction setting value adjustment unit 211.
  • the blur correction setting value adjustment unit 211 allows the blur correction amount generation unit 204 to generate the blur correction amount based on the shutter speed acquired from the exposure control unit 203 and adjustment data shown in FIG. 14 prepared in advance. Adjust the blur correction settings used for calculations.
  • the blur correction setting value corresponds to the value related to blur correction in the claims. Examples of the blur correction setting value include a correction tracking speed and a cutoff frequency. However, the blur correction setting value is not limited to these, and may be any value as long as it is used in calculations for generating the blur correction amount.
  • the third embodiment is characterized in that the blur correction setting value adjustment section 211 adjusts the blur correction setting value based on the shutter speed in this way.
  • the blur correction setting value when the shutter speed is equal to or higher than the first speed, the blur correction setting value is increased and adjusted to the target amount. Furthermore, when the shutter speed is equal to or lower than the second speed, which is smaller than the first speed, the blur correction setting value is decreased and adjusted to the minimum amount. Furthermore, when the shutter speed is equal to or higher than the second speed, the camera shake correction setting value is adjusted so as to approach the target amount as the shutter speed becomes faster. Note that the characteristics of the adjustment data shown in FIG. 14 may change not linearly but exponentially or quadratically, and may be adjusted according to the use case.
  • the blur correction amount generation unit 204 uses the sensing result obtained from the sensor unit 202 and the blur correction setting value obtained from the blur correction setting value adjustment unit 211 to generate a blur correction amount based on a general blur correction algorithm. generate. When the blur correction setting value is adjusted to a low value, the correction amount generated by the blur correction amount generation unit 204 also becomes small, and when the blur correction setting value is adjusted to a high value, the blur correction amount also becomes large. . The blur correction amount generation unit 204 outputs the blur correction amount to the blur correction processing unit 206.
  • the blur correction processing unit 206 performs blur correction processing using an electronic blur correction method that cuts out an image of a predetermined size from the image data based on the amount of blur correction.
  • the blur correction method is the same as in the first embodiment.
  • the image data that has been subjected to blur correction processing is processed by a development processing section 207 and an output processing section 208, and is output as a captured image.
  • the other configurations of the imaging device 10 and the information processing device 300 are the same as in the first embodiment.
  • the processing of the information processing device 300 in the third embodiment is performed as described above.
  • the present technology can be applied not only to the amount of camera shake correction itself but also to adjustment of various setting values for generating the amount of camera shake correction.
  • By adjusting the blur correction setting value based on the shutter speed and determining the adjusted blur correction amount based on the blur correction setting value it is possible to suppress the sense of discomfort in the captured image as in the first embodiment. .
  • the imaging device 10 has the function of the information processing device 300, and all processing is performed within the imaging device 10. Not limited.
  • the processing by the information processing device 300 may be divided between the imaging device 10 and the external device 20.
  • the imaging device 10 includes an exposure control section 203 and a blur correction amount generation section 204, as shown in FIG. 15A.
  • the external device 20 includes a blur correction amount adjustment section 205 and a blur correction processing section 206.
  • Examples of the external device 20 include a server, a personal computer, a smartphone, a tablet terminal, and a camera.
  • the imaging device 10 superimposes the shutter speed set by the exposure control unit 203 and the blur correction amount generated by the blur correction amount generation unit 204 on image data as metadata, and transmits the metadata to the external device 20. Setting the shutter speed and generating the blur correction amount are the same as in the first embodiment.
  • the external device 20 that has received the image data through the communication unit 21 outputs the image data to the blur correction processing unit 206. Further, the shutter speed and the blur correction amount as metadata superimposed on the image data are output to the blur correction amount adjustment unit 205.
  • the blur correction amount adjustment unit 205 adjusts the blur correction amount in the same manner as in the first embodiment, and outputs the adjusted blur correction amount to the blur correction processing unit 206. Then, the blur correction processing unit 206 performs blur correction processing on the image data based on the adjusted blur correction amount.
  • the image data subjected to the blur correction process is stored in, for example, the storage medium 22 as a captured image. Note that the captured image may be transmitted from the external device 20 to another device.
  • the blur correction amount generation unit 204 is configured to be included in the external device 20, and the imaging device 10 superimposes the sensing result acquired by the sensor unit 202 on image data as metadata and transmits it to the external device 20. You may also do so. In this case, the blur correction amount generation unit 204 in the external device 20 generates the blur correction amount.
  • the blur correction processing unit 206 performs blur correction processing by cutting out a predetermined range based on the center of the image data as a fail-safe measure. I do. Thereby, even if there is an abnormality in the sensing result, the output of the captured image will not be interrupted, and for example, it is possible to avoid problems with broadcasting using the captured image.
  • Cutting out a predetermined range based on the center of the image data can be realized, for example, by generating the blur correction amount as 0 in the blur correction amount generation unit 204.
  • the blur correction amount By setting the blur correction amount to 0, even if the blur correction amount is adjusted by the blur correction amount adjustment unit 205, the result is 0 even if 0 is multiplied by the magnification (0 to 1 times).As a result, the center is used as the reference. It will be cut out.
  • the determination as to whether or not to execute failsafe may be made by the blur correction amount generation unit 204 that receives the sensing results from the sensor unit 202, or may be made by a control unit included in the imaging device 10.
  • a processing unit dedicated to the determination may perform the determination.
  • the adjustment data for adjusting the blur correction amount is not limited to that shown in FIG. 9, and may have a characteristic having hysteresis, for example, as shown in FIG. 16.
  • a plurality of adjustment data may be prepared for each imaging environment (indoor, outdoor, etc.), and the adjustment data to be used may be selected depending on the imaging environment.
  • a plurality of adjustment data may be prepared for each type of subject P, and the adjustment data to be used may be selected depending on the subject P.
  • adjustment data preselected by the user from a plurality of adjustment data may be used, or the control unit (not shown) of the imaging device 10 may use the image data transmitted from the imaging unit 201 to create an imaged scene ( Alternatively, the adjustment data may be selected based on the recognition result.
  • the lens head 100 including the lens 101 is described as being replaceable with respect to the camera body 200, but the present technology relates to an imaging device in which the lens 101 and the camera body 200 are integrally configured. is also applicable.
  • this technology can be applied to any device that has a camera function, such as a smartphone, personal computer, tablet terminal, portable game machine, or wearable device.
  • the present technology is not limited to still images, but can also be applied to frame images that make up a moving image.
  • the present technology can also have the following configuration.
  • An information processing device that includes an adjustment unit that adjusts either a value related to blur correction or the shutter speed in an imaging device that has an electronic blur correction function that corrects blur in image data and a shutter speed control function.
  • the value related to blur correction is a blur correction amount indicating a movement amount of a cutting position of image data in electronic blur correction.
  • the adjustment section is a blur correction amount adjustment section that adjusts the blur correction amount based on the shutter speed.
  • the information processing device determines the minimum amount of the blur correction amount based on the target amount of the blur correction amount.
  • the blur correction amount adjustment unit adjusts the blur correction amount to the minimum amount when the shutter speed is less than or equal to a second speed that is a smaller value than the first speed (5) or (5) 6)
  • the information processing device according to item 6).
  • the blur correction amount adjustment unit adjusts the blur correction amount from the minimum amount to approach the target amount as the shutter speed approaches the second speed from the first speed (5) to (7). ).
  • the information processing device is an exposure adjustment section that adjusts the shutter speed based on the blur correction amount.
  • the value related to blur correction is a blur correction setting value used to calculate the blur correction amount.
  • the blur correction setting value is a tracking speed of blur correction.
  • the blur correction setting value is a cutoff frequency.
  • the adjustment section is a blur correction setting value adjustment section that adjusts the blur correction setting value based on the shutter speed.
  • the information processing device according to any one of (10) to (13), including a blur correction amount generation unit that generates a blur correction amount based on the blur correction setting value.
  • the information processing device according to any one of (1) to (14), including a blur correction processing section that performs blur correction processing on the image data.
  • the information processing device according to any one of (1) to (15), including an imaging unit that generates the image data.
  • the blur correction processing unit performs blur correction processing by cutting out a predetermined range based on the center of the image data. .
  • Imaging device 201 Imaging section 204... Shake correction amount generation section 205... Shake correction amount adjustment section 206... Shake correction processing section 210... Exposure adjustment section 211... Blur correction setting value adjustment unit 300...information processing device

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

Ce dispositif de traitement d'informations comprend une unité d'ajustement pour ajuster une vitesse d'obturateur et une valeur relative à une correction de flou dans un dispositif d'imagerie qui comprend une fonction électronique de correction de flou pour corriger un flou dans des données d'image, et une fonction de commande de vitesse d'obturateur.
PCT/JP2023/024971 2022-08-30 2023-07-05 Dispositif de traitement d'informations, procédé de traitement d'informations et programme Ceased WO2024048069A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024543833A JPWO2024048069A1 (fr) 2022-08-30 2023-07-05

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022136500 2022-08-30
JP2022-136500 2022-08-30

Publications (1)

Publication Number Publication Date
WO2024048069A1 true WO2024048069A1 (fr) 2024-03-07

Family

ID=90099425

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/024971 Ceased WO2024048069A1 (fr) 2022-08-30 2023-07-05 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Country Status (2)

Country Link
JP (1) JPWO2024048069A1 (fr)
WO (1) WO2024048069A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011050048A (ja) * 2009-07-27 2011-03-10 Panasonic Corp 撮像装置
JP2017181717A (ja) * 2016-03-30 2017-10-05 リコーイメージング株式会社 防振制御装置
JP2018072540A (ja) * 2016-10-28 2018-05-10 キヤノン株式会社 画像処理装置、その画像処理方法および光学機器

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011050048A (ja) * 2009-07-27 2011-03-10 Panasonic Corp 撮像装置
JP2017181717A (ja) * 2016-03-30 2017-10-05 リコーイメージング株式会社 防振制御装置
JP2018072540A (ja) * 2016-10-28 2018-05-10 キヤノン株式会社 画像処理装置、その画像処理方法および光学機器

Also Published As

Publication number Publication date
JPWO2024048069A1 (fr) 2024-03-07

Similar Documents

Publication Publication Date Title
CN110035222B (zh) 半导体装置和电子设备
CN110035223B (zh) 半导体设备和电子设备
KR101528860B1 (ko) 디지털 촬영 장치의 흔들림 보정 방법 및 장치
CN109951638B (zh) 摄像头防抖系统、方法、电子设备和计算机可读存储介质
US20090167911A1 (en) Imaging device and its shutter drive mode selection method
US9111129B2 (en) Subject detecting method and apparatus, and digital photographing apparatus
JP2017058660A (ja) 像振れ補正装置、傾き補正装置、像振れ補正装置の制御方法、傾き補正装置の制御方法
JP6652300B2 (ja) 画像処理装置、撮像装置および制御方法
EP3836540B1 (fr) Appareil de traitement d'image et appareil de capture d'image
CN104349019A (zh) 图像处理装置、图像处理方法和程序
JP6721084B2 (ja) ズーム制御装置、ズーム制御方法およびプログラム
JP6529533B2 (ja) 撮像装置、撮像装置の制御方法、及び、プログラム
CN106303249A (zh) 视频防抖方法和设备
JP5393877B2 (ja) 撮像装置および集積回路
JP7324284B2 (ja) 撮像デバイス、手ぶれ補正デバイス、撮像方法及び手ぶれ補正方法
WO2017104102A1 (fr) Dispositif d'imagerie
TW202044819A (zh) 嚴格同步光學影像穩定化中之滾動補償及模糊減少
JP7131541B2 (ja) 画像処理装置、画像処理方法および画像処理プログラム
WO2024048069A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et programme
JP6053422B2 (ja) 撮像装置および撮像方法
JP6289070B2 (ja) 撮像装置およびその制御方法
CN113808173A (zh) 图像校正方法、装置、电子设备、存储介质
US20250310644A1 (en) Dynamic selection of optical-image stabilization modes
JP6257715B2 (ja) 表示装置および表示装置の作動方法
JP2017204666A (ja) 撮像装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23859839

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024543833

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23859839

Country of ref document: EP

Kind code of ref document: A1