WO2024090075A1 - Information processing device, information processing method, and program - Google Patents

Abstract

[Problem] To more efficiently suppress vibration caused to a camera mounted in a mobile object. [Solution] This information processing device comprises: an acquisition unit which acquires vibration information that relates to vibration caused to a camera mounted in a mobile object and is based on an image obtained by image capturing by the camera; and a generation unit which, on the basis of the vibration information acquired by the acquisition unit, generates control information to be used for suppressing the vibration.

Description

Information processing device, information processing method, and program

This disclosure relates to an information processing device, an information processing method, and a program.

In recent years, technology has been developed to attach a camera to a moving object such as an automobile, an autonomous mobile cart, or a drone, and use the moving object for photography purposes. For example, the technology described in Patent Document 1 discloses a technology in which a sensor detects the shaking of a camera mounted on a moving object, and a control value based on the detected camera shaking is fed back to a motor to control the camera shaking.

JP 2019-092017 A

However, in Patent Document 1, for example, when the acceleration or frequency of the vibrations generated in the moving body becomes greater than a certain level, it can become difficult to suppress camera shake. A passive vibration-proof mechanism consisting of a wire or spring-damper system can be used as a means of suppressing high-frequency vibrations, but while this is effective in preventing vibration in the high-frequency range, there are regions in the low-frequency range where vibration worsens due to resonance. Also, in Patent Document 1, the shake is detected by a sensor, but due to the influence of the camera's image stabilization and the distance from the subject, the tendency of the shake detected by the sensor and the actual shake of the image may not match.

Therefore, this disclosure proposes a new and improved information processing device, information processing method, and program that can more efficiently suppress vibrations that occur in a camera mounted on a moving object.

According to the present disclosure, there is provided an information processing device including an acquisition unit that acquires vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera, and a generation unit that generates control information used to suppress the vibrations based on the vibration information acquired by the acquisition unit.

The present disclosure also provides an information processing method executed by a computer, which includes acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera, and generating control information used to suppress the vibrations based on the acquired vibration information.

Furthermore, according to the present disclosure, a program is provided that causes a computer to realize an acquisition function for acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera, and a generation function for generating control information used to suppress the vibrations based on the vibration information acquired by the acquisition function.

FIG. 1 is an explanatory diagram illustrating an example of an information processing system according to the present disclosure. FIG. 2 is an explanatory diagram for explaining an example of a functional configuration of a vehicle according to the present disclosure. FIG. 2 is an explanatory diagram for explaining an overview of acquiring vibration information; 10 is an explanatory diagram for explaining an example of a graph showing the relationship between the magnitude of the amplitude obtained by the analysis unit and the frequency; FIG. FIG. 11 is an explanatory diagram for explaining an example of control information instructing a travel route. FIG. 2 is an explanatory diagram for explaining an example of an operation process of the information processing device according to the embodiment. FIG. 2 is an explanatory diagram for explaining a specific example of the shape of a wheel of a vehicle. FIG. 2 is an explanatory diagram for explaining a specific example of the shape of a wheel of a vehicle. FIG. 2 is an explanatory diagram for explaining a specific example of the shape of a wheel of a vehicle. FIG. 2 is an explanatory diagram for explaining a specific example of the shape of a road surface in an environment in which a vehicle moves. FIG. 2 is an explanatory diagram for explaining a specific example of the shape of a road surface in an environment in which a vehicle moves. FIG. 2 is a block diagram showing a hardware configuration of an information processing device.

Below, a preferred embodiment of the present disclosure will be described in detail with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals to avoid redundant description.

The "Mode for Carrying Out the Invention" will be described in the following item order.
1. Overview 1.1. Overview of information processing system 1.2. Example of functional configuration of vehicle 2. Examples 2.1. Overview 2.2. Example of vibration analysis 2.3. First example 2.4. Second example 2.5. Third example 3. Example of operation processing 4. Modification 5. Example of hardware configuration 6. Supplementary information

＜＜1. Overview＞＞
As an embodiment of the present disclosure, a mechanism for more efficiently suppressing vibrations occurring in a camera mounted on a moving object will be described.
<1.1. Overview of information processing system>

FIG. 1 is an explanatory diagram for explaining an example of an information processing system according to the present disclosure. The information processing system according to the present disclosure includes a vehicle 1.

(Vehicle 1)
The vehicle 1 according to the present disclosure is an example of a moving object that moves while capturing an image of a certain environment. For example, when the vehicle 1 is used for the purpose of monitoring the inside of a factory, the vehicle 1 moves while capturing an image of the inside of the factory. Furthermore, the vehicle 1 may move while capturing an image of the inside of the factory in advance to obtain frequency response information described later, prior to actual operation of monitoring the inside of the factory.

Vehicle 1 is an unmanned mobile body that moves autonomously. For example, vehicle 1 may move autonomously based on input information such as the positions of two points, a start point and an end point. In addition, the input information used for the autonomous control of vehicle 1 may include information regarding the travel route.

The unmanned moving body is not limited to the vehicle 1. The unmanned moving body may be, for example, a ship or an air vehicle such as a drone.

The vehicle 1 according to the present disclosure also includes a camera unit 5, a vehicle body unit 10, and a wheel unit 50. An example of the functional configuration of the vehicle 1 will be described below with reference to FIG. 2.

<1.2. Example of functional configuration of vehicle 1>
FIG. 2 is an explanatory diagram for explaining an example of a functional configuration of the vehicle 1 according to the present disclosure.

(Camera unit 5)
The camera unit 5 according to the present disclosure captures an image of the environment in which the vehicle 1 moves. Vibrations generated in the wheel unit 50 are transmitted from the vehicle body unit 10 to the camera unit 5.

As shown in FIG. 2, the camera unit 5 according to the present disclosure includes a video capture unit 6 and a correction unit 7.

The video capture unit 6 according to the present disclosure has a video acquisition function for acquiring video by shooting. In addition, the video capture unit 6 outputs the video acquired by shooting to the information processing device 110 described below.

The correction unit 7 according to the present disclosure has a correction function for correcting blur in the video captured by the video capture unit 6. The correction function may be realized by executing a known correction process. For example, the known correction process includes a process related to image stabilization that is included in commercially available cameras.

(Vehicle body section 10)
The vehicle body section 10 according to the present disclosure is a vehicle body portion of the vehicle 1, and transmits vibrations input from the wheel section 50 to the camera section 5. As shown in FIG. 2 , the vehicle body section 10 includes an information processing device 110, a semi-active vibration isolation mechanism 120, a self-position recognition section 130, and a battery 140.

The information processing device 110 according to the present disclosure acquires vibration information relating to vibrations occurring in the camera unit 5 based on the image captured by the camera unit 5, and generates control information used to suppress the vibrations based on the vibration information.

As shown in FIG. 2, the information processing device 110 includes an analysis unit 111 and a generation unit 113.

The analysis unit 111 according to the present disclosure is an example of an acquisition unit, and analyzes the video input from the camera unit 5 to acquire vibration information related to the vibrations occurring in the camera unit 5. Here, the vibration information includes various information such as information related to the magnitude of the vibration and information related to the frequency of the vibration. Furthermore, the magnitude of the vibration may be, for example, the magnitude of the amplitude. Specific examples of the analysis of the video will be described later.

The generation unit 113 according to the present disclosure generates vibration information to be used for suppressing vibrations based on the vibration information obtained by the analysis unit 111.

The generating unit 113 generates control information to be used for suppressing vibrations based on the fact that the magnitude of vibrations does not satisfy a predetermined criterion. For example, when the magnitude of the amplitude is equal to or greater than a predetermined value, the generating unit 113 may determine that the magnitude of vibrations does not satisfy a predetermined criterion and generate control information to be used for suppressing vibrations. Specific examples of control information to be used for suppressing vibrations will be described later.

The semi-active vibration isolation mechanism 120 according to the present disclosure is an anti-vibration mechanism whose characteristics, such as elasticity or resistance, can be changed. By changing the characteristics of the semi-active vibration isolation mechanism 120, the resonance region is also changed. For example, vibrations generated in the wheel unit 50 are transmitted to the camera unit 5 via the semi-active vibration isolation mechanism 120. Here, the magnitude of the vibrations transmitted to the camera unit 5 changes according to the characteristics of the semi-active vibration isolation mechanism 120.

The semi-active vibration isolation mechanism 120 may change its characteristics based on control information generated by the generation unit 113, for example.

As shown in FIG. 2, the semi-active vibration isolation mechanism 120 includes a variable spring 121 and a variable damper 123.

The variable spring 121 according to the present disclosure is a spring with a variable elastic force. By changing the elastic force of the variable spring 121, the frequency corresponding to the resonant frequency can be shifted. For example, the elastic force of the variable spring 121 may be changed based on control information generated by the generating unit 113. Furthermore, the variable spring 121 is not particularly limited, and may be, for example, an air spring.

The variable damper 123 according to the present disclosure is a damper with variable resistance. By changing the resistance of the variable damper 123, the magnitude of the amplitude corresponding to the resonant frequency can be reduced. For example, the resistance of the variable spring may be changed based on control information generated by the generating unit 113. Furthermore, the variable damper 123 is not particularly limited, and may be, for example, a magnetic fluid damper.

The self-location recognition unit 130 according to the present disclosure recognizes the location of the vehicle 1. For example, the function of the self-location recognition unit 130 is realized by a sensor capable of acquiring information related to the self-location of the vehicle 1.

For example, the self-location recognition unit 130 may recognize the location of the vehicle 1 using a positioning sensor (LiDAR), or may recognize the location of the vehicle 1 using a Global Navigation Satellite System (GNSS).

The battery 140 according to the present disclosure is capable of storing (i.e., charging) and releasing (i.e., discharging) electric power using chemical reactions, and supplies the stored electric power as a power source for each component of the vehicle 1. The battery 140 may be any type of battery, such as a lithium-ion secondary battery, a nickel-cadmium storage battery, or a lead storage battery.

(Wheel portion 50)
The wheel unit 50 according to the present disclosure includes wheels that move the vehicle 1 by contacting the road surface, and is a source of vibrations that occur in the camera unit 5. As shown in FIG. 2 , the wheel unit 50 includes a motor 510 and a wheel encoder 520.

The motor 510 according to the present disclosure drives the wheels of the vehicle 1 using power input from the battery 140. For example, the motor 510 drives the wheels of the vehicle 1 based on the control information generated by the generation unit 113.

The wheel encoder 520 according to the present disclosure acquires wheel speed information including the vehicle speed pulse of the wheels of the vehicle 1. The wheel encoder 520 outputs the acquired wheel speed information to the information processing device 110.

Above, an example of the functional configuration of the vehicle 1 according to the present disclosure has been described. Next, an example of an information processing system according to the present disclosure will be described.

<<2. Example>>
2.1 Overview
In recent years, there has been an increase in cases where a moving object is equipped with a camera and used for photography purposes. However, vibrations that occur when the moving object is moving can have an adverse effect on the images captured by the camera, such as blurring.

Various methods are used to reduce such image blur. For example, one method of reducing image blur is to use image stabilization, a function of a camera. However, image stabilization is designed assuming human hand shake, and it is difficult to suppress high-frequency vibrations, for example. Furthermore, the performance related to blur suppression may differ for each camera mounted on a moving object. For this reason, the manager of the moving object may need to verify the performance related to blur suppression for each camera mounted on the moving object.

Another method of reducing image blur is to install a passive vibration isolation mechanism on a moving object. There are several types of passive vibration isolation mechanisms, such as suspension, spring-damper systems, or wire systems. However, no matter the type, there is a resonance range and an isolation range, so it is difficult to uniformly suppress all vibration frequencies. In addition, the resonance frequency varies depending on the weight of the camera and the performance of the image stabilization, so every time the camera is changed, work such as replacing and adjusting vibration isolation parts such as springs and wires may be required.

In addition, the vibration input to the vehicle changes depending on various information such as the road surface on which the moving object moves, the type of wheels of the moving object, the speed of the moving object, or the attitude of the vehicle and camera. Therefore, the vibration control performance required to reduce blurring of the image changes depending on the driving environment of the moving object and the characteristics of the moving object. Therefore, depending on the vibration control performance required, the administrator may need to perform tasks such as replacing or adjusting vibration-proof parts.

　In the methods for reducing image blur as described above, when the parts of the moving object (camera, wheels, etc.) or the environment in which the moving object moves are changed, the person in charge of the moving object needs to verify the vibration isolation performance each time, which may require work such as replacing or adjusting the vibration isolation parts as necessary.

The information processing device 110 according to the present disclosure acquires vibration information generated in the camera unit 5 mounted on the vehicle 1 based on the image captured by the camera unit 5, and generates control information used to suppress vibration based on the vibration information. This makes it possible to reduce the amount of work required for vibration isolation as described above, and the information processing device 110 may be able to more efficiently suppress vibration generated in the camera unit 5.

First, a specific example of a method in which the analysis unit 111 according to the present disclosure acquires vibration information will be described with reference to Figures 3 and 4.

<2.2. Vibration analysis example>
Fig. 3 is an explanatory diagram for explaining an overview of acquiring vibration information. Fig. 3 is an example of an image captured by the camera unit 5 equipped in the vehicle 1. The image shown in Fig. 3 is an image in which there is a wall at the end of the passageway in which the vehicle 1 moves, and a screen is hung on the wall.

First, the vehicle 1 moves through a certain environment while capturing images. The analysis unit 111 then extracts an edge E from the image using a known method. For example, in the example shown in FIG. 3, the analysis unit 111 extracts an edge E that exists in the lower left corner of the screen from the image using a known method.

The method for extracting the edge E is not particularly limited, but may be a method using various filters such as a Sobel filter or a Laplacian filter.

Then, the analysis unit 111 observes the movement of the extracted edge E and obtains vibration information including amplitude and frequency. For example, the analysis unit 111 may obtain vibration information by analyzing the vibration components of the movement of the edge E using FFT (Fast Fourier Transform). Furthermore, the analysis unit 111 obtains the relationship between the magnitude of the amplitude in the environment and the frequency based on the analysis results.

FIG. 4 is an explanatory diagram for explaining an example of a graph showing the relationship between the magnitude of the amplitude obtained by the analysis unit 111 and the frequency. In the graph shown in FIG. 4, the horizontal axis represents frequency, and the vertical axis represents amplitude.

For example, the analysis unit 111 obtains a graph showing the relationship between the amplitude and frequency, as shown in FIG. 4, based on the analysis results. Then, the analysis unit 111 identifies the frequency that is dominant in the vibration (for example, the resonant frequency).

For example, in the graph shown in Figure 4, a resonant frequency RF exists at approximately 9 Hz. The amplitude corresponding to the resonant frequency RF is larger than the amplitudes corresponding to other frequencies, as shown in Figure 4.

In other words, when the frequency of the vibrations generated in the camera unit 5 is approximately 9 Hz (resonant frequency RF), the shaking of the image captured by the camera unit 5 will be greater than when the vibration frequency is a frequency that is not the resonant frequency RF (for example, 5 Hz or 15 Hz). In other words, in order to reduce shaking of the image, it is desirable for the generation unit 113 to generate control information that is away from the frequency band of the resonant frequency RF.

The acceptable range of image blur may vary depending on the intended use of the image. For example, if the intended use of the image is to provide it to a manager who monitors the inside of a factory, the acceptable range of image blur may be set to the level of vibration that does not cause the manager to feel uncomfortable watching the image.

As a more detailed example, the amplitude at which a human can recognize an object contained in an image may be set as the acceptable range. If the amplitude set as the acceptable range is set to a predetermined value, the generation unit 113 may generate control information that suppresses vibrations when the amplitude exceeds the predetermined value, assuming that the human feels uncomfortable with the image. Note that the predetermined value may be obtained experimentally or may be set manually by the user.

A specific example in which the analysis unit 111 according to the present disclosure acquires vibration information has been described above. Next, a specific example in which the generation unit 113 according to the present disclosure generates control information used to suppress vibration based on the vibration information will be described.

In the following explanation, three different examples of control information used to suppress vibrations will be described. The control information according to the first example is control information that changes the speed of the vehicle 1, the control information according to the second example is control information that changes the characteristics of the semi-active vibration isolation mechanism 120, and the control information according to the third example is control information that indicates the travel path of the vehicle 1. The generation unit 113 according to the present disclosure may execute any one of the processes described in the first to third examples, or may execute a combination of processes of multiple examples.

In the following explanation, the vehicle 1 moves through the environment prior to actual operation, and the analysis unit 111 acquires the relationship between the magnitude of the amplitude and the frequency described above (in the following explanation, this may be referred to as frequency response information). In addition, the generation unit 113 mainly explains an example in which it generates various types of control information based on the frequency response information acquired in advance and vibration information acquired in real time (amplitude, frequency, etc.).

2.3. First embodiment
In general, as the wheel speed of the vehicle 1 increases, the frequency of vibration generated in the wheel unit 50 may increase. That is, as the wheel speed of the vehicle 1 changes, the magnitude of vibration transmitted from the wheel unit 50 to the camera unit 5 via the vehicle body unit 10 may also vary.

The generation unit 113 may generate control information to change the speed of the vehicle 1 when the magnitude of the amplitude is equal to or greater than a predetermined value (i.e., when shaking outside the acceptable range occurs).

For example, the generation unit 113 may generate control information for changing the speed of the vehicle 1 based on the frequency response information. When frequency response information such as that shown in FIG. 4 is obtained, the magnitude of the amplitude generated in the camera unit 5 is maximum at a resonant frequency RF of approximately 9 Hz.

In other words, if the frequency of the vibration occurring in the camera unit 5 is 8.5 Hz, increasing the speed of the vehicle 1 will cause the frequency to approach the resonant frequency RF shown in Figure 4 and the amplitude to become larger.

On the other hand, as the vibration frequency decreases from 8.5 Hz to 8 Hz, it moves away from the resonant frequency RF shown in Figure 4 and the amplitude decreases.

The generation unit 113 may therefore generate control information to reduce the speed of the vehicle 1 so that the frequency of the vibrations generated in the camera unit 5 moves away from the resonant frequency RF.

The generating unit 113 may also generate control information to reduce the speed of the wheels of the vehicle 1 so that the magnitude of the amplitude is equal to or less than a predetermined value. More specifically, when the predetermined value is the value of the magnitude of the amplitude corresponding to 8.5 Hz, the generating unit 113 may generate control information to reduce the speed of the wheels of the vehicle 1.

Also, as shown in FIG. 4, even if the frequency of the vibration generated in the camera unit 5 exceeds the resonant frequency RF (approximately 9 Hz) and becomes even larger, the amplitude may become smaller. Therefore, the generation unit 113 may generate control information to increase the speed of the wheels of the vehicle 1 so that the magnitude of the amplitude becomes equal to or less than a predetermined value.

Then, the information processing device 110 outputs control information for changing the speed of the vehicle 1 to the motor 510 of the wheel unit 50. Furthermore, the motor 510 may drive the wheels and change the speed of the vehicle 1 based on the input control information.

According to the first embodiment described above, by changing the speed of the vehicle 1, the magnitude of the amplitude generated in the camera unit 5 is changed, which may make it possible to reduce blurring of the image.

Note that the control information according to the first embodiment is not limited to the above-mentioned example. For example, the generation unit 113 may generate control information for changing the speed of the vehicle 1 so that the speed is outside a predetermined range from the resonant frequency RF. Here, outside the predetermined range includes a range in which the magnitude of the amplitude is less than a predetermined value.

2.4. Second Example
As described above, the semi-active vibration isolation mechanism 120 is a mechanism capable of changing characteristics such as elasticity or resistance. Furthermore, since vibrations generated in the wheel unit 50 are transmitted to the camera unit 5 via the semi-active vibration isolation mechanism 120, the magnitude of the vibrations transmitted to the camera unit 5 changes depending on the characteristics of the semi-active vibration isolation mechanism 120.

The generation unit 113 may generate control information that changes the characteristics of the semi-active vibration isolation mechanism 120 when the magnitude of the amplitude generated in the camera unit 5 is equal to or greater than a predetermined value.

The variable spring 121 has a variable elastic force. For example, by changing the elastic force of the variable spring 121, the frequency corresponding to the resonant frequency shifts. More specifically, if the resonant frequency RF is approximately 9 Hz as shown in FIG. 4, it is possible to increase the frequency corresponding to the resonant frequency RF to 11 Hz by increasing the elastic force of the variable spring 121, and it is possible to decrease the frequency corresponding to the resonant frequency RF to 8 Hz by decreasing the elastic force of the variable spring 121.

The generation unit 113 may generate control information for changing the elastic force of the variable spring 121 when the magnitude of the amplitude generated in the camera unit 5 is equal to or greater than a predetermined value. For example, the generation unit 113 may generate control information for changing the elastic force of the variable spring 121 based on frequency response information so that the magnitude of the amplitude becomes less than a predetermined value.

The variable damper 123 also has a variable resistance. For example, the magnitude of the amplitude corresponding to the resonant frequency changes as the resistance of the variable damper 123 changes. More specifically, it is possible to reduce the amplitude corresponding to the resonant frequency RF by increasing the resistance of the variable damper 123.

The generation unit 113 may generate control information that changes the resistance force of the variable damper 123 when the magnitude of the amplitude generated in the camera unit 5 is equal to or greater than a predetermined value. For example, the generation unit 113 may generate control information that increases the resistance force of the variable damper 123 based on frequency response information when the magnitude of the amplitude generated in the camera unit 5 is equal to or greater than a predetermined value.

According to the second embodiment described above, by changing the characteristics of the semi-active vibration isolation mechanism 120, it becomes possible to modify the frequency characteristics from vibration input to image output, which may make it possible to reduce image blur.

2.5. Third Example
The environment in which the vehicle 1 moves may include a mixture of areas where vibrations are large (for example, areas with many bumps and grooves) and areas where vibrations are small (for example, areas with few bumps and grooves).

In such a case, it may be possible to reduce image blur by having the vehicle 1 move away from areas where the vibration is strong. Therefore, the information processing device 110 may obtain map information in which the magnitude of vibration is set for each area.

FIG. 5 is an explanatory diagram for explaining an example of control information that indicates a travel route. For example, the information processing device 110 acquires map information M including nine 3×3 areas as shown in FIG. 5 for the environment in which the vehicle 1 travels. For example, the map information M includes an area A1 (area highlighted with dots) where vibrations are strong, and an area A2 (area without dots) where vibrations are weak.

For example, when vehicle 1 moves from point A to point B, route R1 that goes straight from point A to point B passes through area A1 where vibration is large, which may result in significant shaking of the image.

On the other hand, route R2, in which vehicle 1 takes a detour from point A to point B, does not pass through area A1 where vibrations are large, so image blur may be smaller than route R1.

The generation unit 113 may generate control information that indicates the travel route of the vehicle 1 based on the relationship between each of the multiple areas included in the map information of the environment in which the vehicle 1 travels and the magnitude of the vibration.

For example, the self-position recognition unit 130 may acquire position information regarding the self-position of the vehicle 1, and output the position information to the information processing device 110. The generation unit 113 may then link the position information with the map information M to generate control information that indicates a travel route that avoids areas where large vibrations occur.

More specifically, the generator 113 may generate control information that indicates a movement path along which the magnitude of the amplitude is less than a predetermined value.

Note that the map information M as shown in FIG. 5 may be prepared in advance before the vehicle 1 is actually operated in the environment, or may be acquired in parallel while the vehicle 1 is actually operated.

According to the third embodiment described above, by moving the vehicle 1 along a route that causes less vibration, it may be possible to further reduce blurring of the image.

<<3. Operation processing example>>
6 is an explanatory diagram for explaining an example of the operation process of the information processing device 110 according to this embodiment. First, the generation unit 113 causes the vehicle 1 to run in an environment (S101).

Then, the analysis unit 111 acquires vibration information by performing image analysis on the image captured by the camera unit 5 (S105).

Then, the generation unit 113 determines whether the magnitude of the amplitude included in the vibration information is less than a predetermined value (S109). If the magnitude of the amplitude is less than the predetermined value (S109: Yes), the process returns to S101 again, and if the magnitude of the amplitude is equal to or greater than the predetermined value (S109: No), the process proceeds to S117.

If the magnitude of the amplitude is equal to or greater than the predetermined value (S109: No), the information processing device 110 acquires wheel speed information from the wheel encoder 520 (S117).

Then, the generation unit 113 determines whether or not the current moving speed is required (S121). If the current moving speed is not required (S121: No), the process proceeds to S125. If the current moving speed is required (S121: Yes), the process proceeds to S129.

If the current travel speed is not necessary (S121: No), the generation unit 113 generates control information to change the speed of the vehicle 1 (S125).

If the current moving speed is required (S121: Yes), the generation unit 113 determines whether or not there is a moving path that reduces the amplitude (S129). If there is a moving path that reduces the amplitude (S129: Yes), the process proceeds to S133, and if there is no moving path that reduces the amplitude (S129: No), the process proceeds to S137.

If there is a travel route that reduces the magnitude of the amplitude (S129: Yes), the generation unit 113 generates control information that indicates the travel route of the vehicle 1 (S133).

If there is no movement path that reduces the magnitude of the amplitude (S129: No), the generator 113 generates control information that changes the characteristics of the semi-active vibration isolation mechanism 120 (S137).

Then, the generation unit 113 controls the operation of the vehicle 1 based on the generated control information (S141), and the information processing device 110 according to the present disclosure ends the operation process. Note that if the control information generated by the generation unit 113 is control information for changing the speed of the vehicle 1 or control information for instructing the travel route of the vehicle 1, the generation unit 113 may control the operation of the vehicle 1 by outputting the control information to the wheel unit 50 (more specifically, the motor 510). Also, if the control information generated by the generation unit 113 is control information for changing the characteristics of the semi-active vibration isolation mechanism 120, the generation unit 113 may control the operation of the vehicle 1 by outputting the control information to the semi-active vibration isolation mechanism 120.

In the above-mentioned operation processing example, a determination regarding the generation of control information is made in the order of priority of the first embodiment (changing the speed), the third embodiment (instruction of the movement path), and the second embodiment (changing the characteristics of the semi-active vibration isolation mechanism 120), but the order of priority is not limited to the example shown in FIG. 6. For example, the second embodiment or the third embodiment may have the highest priority.

<<4. Modifications>>
The characteristics of the vibrations generated in the camera unit 5 may vary depending on the state of the vehicle 1 or the environment in which the vehicle 1 moves. Such vibration characteristics may be predictable in advance by using model information and environmental information of the vehicle 1. For example, the period and characteristics of the vibrations may be predictable depending on the shape of the wheels or the shape of the road surface.

FIGS. 7A, 7B, and 7C are explanatory diagrams for explaining specific examples of the shape of the wheels of the vehicle 1. For example, the wheels of the vehicle 1 may be Mecanum wheels as shown in FIG. 7A. For example, when the vehicle 1 moves straight or sideways on a flat road surface (i.e. a road surface with few bumps and grooves) using the Mecanum wheels, strong vibrations are generated at a frequency that is a multiple of the product of the rotational speed of the wheels and the number of barrels per wheel. Furthermore, when movement involves the rotation of the Mecanum wheels, periodic vibrations are also generated according to the difference in wheel speed.

The generation unit 113 may therefore utilize vibration characteristics related to such wheel information (for example, when the wheel type is a Mecanum wheel) to generate control information to be used to suppress vibration.

Furthermore, although an example in which the type of wheel is a Mecanum wheel has been described, the type of wheel is not limited to a Mecanum wheel. For example, the type of wheel may be an omni wheel as shown in FIG. 7B, or a normal wheel as shown in FIG. 7C.

The generating unit 113 may then generate control information to be used to suppress vibrations based on wheel information, such as the position and number of bumps and recesses, the material of the wheel, or the attitude of the wheel.

The generating unit 113 may also generate an estimation model by learning in advance the relationship between the wheel information and the vibrations that occur in the wheels. The generating unit 113 may then generate control information used to suppress vibrations based on the wheel information of the vehicle 1 and the estimation model.

FIGS. 8A and 8B are explanatory diagrams for explaining specific examples of the shape of the road surface in the environment in which the vehicle 1 moves. There may be unevenness in the road surface, just as there are unevenness in the positions of the wheels. For example, the vibrations generated in the wheels may differ between the road surface shown in FIG. 8A and the road surface shown in FIG. 8B due to differences in the positions and spacing of grooves and protrusions, the materials, and other road surface conditions.

For example, the generation unit 113 may learn in advance the relationship between road surface information and vibrations that occur in the wheels, and generate an estimation model. The generation unit 113 may then generate control information used to suppress vibrations based on the information about the road surface on which the vehicle 1 travels and the estimation model.

In addition, the characteristics of the vibrations generated in the camera unit 5 may differ depending on the material and weight of the camera unit 5 mounted on the vehicle 1. For example, the resonant frequency may be estimated from information about the camera unit 5, such as the material and weight of the camera unit 5, and the spring constant of the vibration isolation mechanism. Therefore, by using the information about the camera unit 5, it may be possible to make various settings to suppress vibrations before the vehicle 1 is actually put into operation.

For example, the generation unit 113 may estimate the resonant frequency based on camera information, and set the characteristics of the semi-active vibration isolation mechanism 120 or the speed of the vehicle 1 so that they are outside a predetermined range from the estimated resonant frequency.

According to the modified example described above, by using wheel information, road surface information, or camera unit 5 information (such as material or weight), it may be possible to predict the characteristics of vibrations occurring in the camera unit 5. As a result, it may be possible to omit the process in which the analysis unit 111 acquires frequency response information based on images obtained by the vehicle 1 moving around the environment while taking pictures before actual operation.

However, if the error between the actually obtained vibration characteristics and the predicted vibration characteristics exceeds an allowable value when the vehicle 1 is actually operated, the analysis unit 111 may acquire frequency response information. In this case, the generation unit 113 may generate control information based on the newly acquired frequency response information.

<<5. Hardware configuration example>>
The above-described embodiment of the present disclosure has been described. The above-described information processing is realized by cooperation between software and hardware of the information processing device 110 described below.

FIG. 9 is a block diagram showing the hardware configuration of the information processing device 110. The information processing device 110 includes a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, a RAM (Random Access Memory) 1003, and a host bus 1004. The information processing device 110 also includes a bridge 1005, an external bus 1006, an interface 1007, an input device 1008, an output device 1010, a storage device (HDD) 1011, a drive 1012, and a communication device 1015.

The CPU 1001 functions as an arithmetic processing device and control device, and controls the overall operation of the information processing device 110 in accordance with various programs. The CPU 1001 may also be a microprocessor. The ROM 1002 stores programs and arithmetic parameters used by the CPU 1001. The RAM 1003 temporarily stores programs used in the execution of the CPU 1001 and parameters that change appropriately during the execution. These are interconnected by a host bus 1004 that is composed of a CPU bus and the like. The functions of the analysis unit 111 and generation unit 113 described with reference to FIG. 2 can be realized by the cooperation of the CPU 1001, ROM 1002, and RAM 1003 with software.

The host bus 1004 is connected to an external bus 1006, such as a PCI (Peripheral Component Interconnect/Interface) bus, via a bridge 1005. Note that it is not necessary to configure the host bus 1004, bridge 1005, and external bus 1006 separately, and these functions may be implemented on a single bus.

The input device 1008 is composed of input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 1001. A user of the information processing device 110 can input various data and instruct the information processing device 110 to perform processing operations by operating the input device 1008.

The output device 1010 includes, for example, a display device such as a liquid crystal display device, an OLED device, and a lamp. Furthermore, the output device 1010 includes an audio output device such as a speaker and headphones. The output device 1010 outputs, for example, played content. Specifically, the display device displays various information such as played video data as text or images. On the other hand, the audio output device converts played audio data, etc. into audio and outputs it.

The storage device 1011 is a device for storing data. The storage device 1011 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 1011 is configured, for example, with a HDD (Hard Disk Drive). This storage device 1011 drives the hard disk and stores the programs executed by the CPU 1001 and various data.

The drive 1012 is a reader/writer for storage media, and is built into the information processing device 110 or is externally attached. The drive 1012 reads information recorded on a removable storage medium 1018, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 1003. The drive 1012 can also write information to the removable storage medium 15.

The communication device 1015 is, for example, a communication interface configured with a communication device for connecting to the network 16. The communication device 1015 may be a wireless LAN compatible communication device, an LTE (Long Term Evolution) compatible communication device, or a wired communication device that performs wired communication.

＜＜6. Supplementary Information＞＞
Although the preferred embodiment of the present disclosure has been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

For example, the display device may display information related to the control information generated by the generation unit 113. More specifically, if the control information generated by the generation unit 113 includes an instruction to reduce the speed, the display device provided in the vehicle 1 may display information instructing the driver to reduce the speed of the vehicle 1. Then, the driver of the vehicle 1 may control the operation of the vehicle 1 based on the information displayed by the display device.

Furthermore, the camera unit 5, the vehicle body unit 10, and the wheel unit 50 do not necessarily have to include all of the components shown in FIG. 2. For example, the camera unit 5 does not have to include the correction unit 7.

In addition, although the present specification has mainly described an example in which the generation unit 113 generates control information used to suppress vibrations using frequency response information acquired before actual operation, the generation unit 113 does not necessarily have to acquire frequency response information in advance. For example, the generation unit 113 may generate control information used to suppress vibrations while acquiring the relationship between the magnitude of amplitude and frequency during actual operation. Furthermore, if frequency response information has not been acquired in advance, the generation unit 113 may continue to generate control information to reduce the speed of the vehicle 1 until the magnitude of the amplitude becomes less than a predetermined value.

Furthermore, in a situation where the vehicle 1 comes into contact with an obstacle on the road surface, the magnitude of the amplitude of the vibration generated in the camera unit 5 may exceed a predetermined value. In such a case, the generation unit 113 may regard the temporary increase in amplitude as noise and may not generate control information used to suppress the vibration. Furthermore, the generation unit 113 may generate control information used to suppress the vibration when the time during which the magnitude of the amplitude exceeds the predetermined value continues for a certain period of time or more. However, the determination of noise is not limited to the method based on time as described above. For example, the analysis unit 111 may detect an obstacle (e.g., a large stone, etc.) on the road surface using a known method. Then, the generation unit 113 may determine that the vibration generated in the vehicle 1 is noise when it is estimated that the wheels of the vehicle 1 have come into contact with the detected obstacle.

The generating unit 113 may also generate control information to change the attitude of the vehicle 1 when the magnitude of the amplitude is equal to or greater than a predetermined value, as control information to be used to suppress vibration. For example, if the connecting part between the vehicle body part 10 and the camera part 5 is movable, the generating unit 113 may generate control information to change the attitude of the connecting part.

In addition, although an example in which the vehicle 1 is equipped with the information processing device 110 has been described in this specification, the information processing device 110 according to the present disclosure may be disposed in a location separate from the vehicle 1. Furthermore, the information processing device 110 may be mounted on the camera unit 5 or the wheel unit 50, rather than on the body unit 10 of the vehicle 1.

Furthermore, the steps in the processing of the information processing device 110 in this specification do not necessarily have to be processed chronologically in the order described in the flowchart. For example, the steps in the processing of the information processing system may be processed in an order different from the order described in the flowchart or in parallel.

It is also possible to create a computer program that causes the hardware, such as the CPU, ROM, and RAM, built into the information processing device 110 to perform functions equivalent to those of the components of the information processing device 110 described above. A non-transitory storage medium that stores the computer program is also provided.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects that are apparent to a person skilled in the art from the description in this specification, in addition to or in place of the above effects.

Note that the following configurations also fall within the technical scope of the present disclosure.
(1)
an acquisition unit that acquires vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
a generation unit that generates control information used to suppress the vibration based on the vibration information obtained by the acquisition unit;
An information processing device comprising:
(2)
The generation unit is
generating the control information based on the magnitude of the vibration not satisfying a predetermined criterion;
The information processing device according to (1).
(3)
The magnitude of the vibration includes the magnitude of the amplitude,
The vibration magnitude not satisfying a predetermined standard includes the amplitude being equal to or greater than a predetermined value.
The information processing device according to (2).
(4)
The generation unit is
generating control information for changing a speed of the moving object when the magnitude of the amplitude is equal to or greater than a predetermined value;
The information processing device according to (3).
(5)
The generation unit is
generating control information for changing a speed of the moving object based on the relationship between the magnitude of the amplitude and the frequency;
The information processing device according to (4).
(6)
The relationship between the magnitude of the amplitude and the frequency includes a resonant frequency at which the magnitude of the amplitude is equal to or greater than a predetermined value.
The information processing device according to (5).
(7)
The generation unit is
generating control information for changing the speed of the moving body so that the speed is outside a predetermined range from the resonant frequency;
The information processing device according to (6) above.
(8)
The moving body has a semi-active vibration isolation mechanism,
The generation unit is
generating the control information for changing a characteristic of the semi-active vibration isolation mechanism when the magnitude of the amplitude is equal to or greater than a predetermined value;
The information processing device according to any one of (3) to (7).
(9)
The semi-active vibration isolation mechanism includes a variable spring.
The generation unit is
generating the control information for changing the elastic force of the variable spring;
The information processing device according to (8).
(10)
The semi-active vibration isolation mechanism includes a variable damper.
The generation unit is
generating the control information for changing the resistance force of the variable damper;
The information processing device according to (8) or (9).
(11)
The generation unit is
generating control information for instructing a moving path of the moving body based on a relationship between each of a plurality of areas included in map information of an environment in which the moving body moves and the magnitude of the vibration;
The information processing device according to any one of (3) to (10).
(12)
The generation unit is
generating control information for instructing a moving route of the moving object based on the position information of the moving object;
The information processing device according to (11) above.
(13)
The generation unit is
generating control information instructing a movement path in which the magnitude of the amplitude is less than a predetermined value;
The information processing device according to (12).
(14)
The moving object includes a vehicle.
14. The information processing device according to claim 13.
(15)
The generation unit is
generating control information used to suppress the vibration based on information about the wheels of the vehicle;
The information processing device according to (14) above.
(16)
The generation unit is
generating control information used to suppress the vibration based on information about a road surface on which the vehicle is traveling;
The information processing device according to (14) or (15).
(17)
The generation unit is
generating control information used to suppress the vibration based on information from a camera mounted on the moving object;
The information processing device according to any one of (14) to (16).
(18)
Mounted on the moving body,
The information processing device according to (1).
(19)
acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
generating control information used to suppress the vibration based on the acquired vibration information;
2. An information processing method implemented by a computer, comprising:
(20)
On the computer,
an acquisition function for acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
a generating function that generates control information used to suppress the vibration based on the vibration information obtained by the acquiring function;
A program to achieve this.

Claims (20)

an acquisition unit that acquires vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
a generation unit that generates control information used to suppress the vibration based on the vibration information obtained by the acquisition unit;
An information processing device comprising: The generation unit is
generating the control information based on the magnitude of the vibration not satisfying a predetermined criterion;
The information processing device according to claim 1 . The magnitude of the vibration includes the magnitude of the amplitude,
The vibration magnitude not satisfying a predetermined standard includes the amplitude being equal to or greater than a predetermined value.
The information processing device according to claim 2 . The generation unit is
generating control information for changing a speed of the moving object when the magnitude of the amplitude is equal to or greater than a predetermined value;
The information processing device according to claim 3 . The generation unit is
generating control information for changing a speed of the moving object based on the relationship between the magnitude of the amplitude and the frequency;
The information processing device according to claim 4. The relationship between the magnitude of the amplitude and the frequency includes a resonant frequency at which the magnitude of the amplitude is equal to or greater than a predetermined value.
The information processing device according to claim 5 . The generation unit is
generating control information for changing the speed of the moving body so that the speed is outside a predetermined range from the resonant frequency;
The information processing device according to claim 6. The moving body has a semi-active vibration isolation mechanism,
The generation unit is
generating the control information for changing a characteristic of the semi-active vibration isolation mechanism when the magnitude of the amplitude is equal to or greater than a predetermined value;
The information processing device according to claim 3 . The semi-active vibration isolation mechanism includes a variable spring.
The generation unit is
generating the control information for changing the elastic force of the variable spring;
The information processing device according to claim 8. The semi-active vibration isolation mechanism includes a variable damper.
The generation unit is
generating the control information for changing the resistance force of the variable damper;
The information processing device according to claim 8. The generation unit is
generating control information for instructing a moving path of the moving body based on a relationship between each of a plurality of areas included in map information of an environment in which the moving body moves and the magnitude of the vibration;
The information processing device according to claim 3 . The generation unit is
generating control information for instructing a moving route of the moving object based on the position information of the moving object;
The information processing device according to claim 11. The generation unit is
generating control information instructing a movement path in which the magnitude of the amplitude is less than a predetermined value;
The information processing device according to claim 12. The moving object includes a vehicle.
The information processing device according to claim 3 . The generation unit is
generating control information used to suppress the vibration based on information about the wheels of the vehicle;
The information processing device according to claim 14. The generation unit is
generating control information used to suppress the vibration based on information about a road surface on which the vehicle is traveling;
The information processing device according to claim 15. The generation unit is
generating control information used to suppress the vibration based on information from a camera mounted on the moving object;
The information processing device according to claim 16. Mounted on the moving body,
The information processing device according to claim 1 . acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
generating control information used to suppress the vibration based on the acquired vibration information;
2. An information processing method implemented by a computer, comprising: On the computer,
an acquisition function for acquiring vibration information related to vibrations occurring in a camera mounted on a moving object based on an image captured by the camera;
a generating function that generates control information used to suppress the vibration based on the vibration information obtained by the acquiring function;
A program to achieve this.

Images