WO2024090389A1 - 冷却実行装置、冷却実行方法、冷却実行プログラム、及び車両 - Google Patents
冷却実行装置、冷却実行方法、冷却実行プログラム、及び車両 Download PDFInfo
- Publication number
- WO2024090389A1 WO2024090389A1 PCT/JP2023/038226 JP2023038226W WO2024090389A1 WO 2024090389 A1 WO2024090389 A1 WO 2024090389A1 JP 2023038226 W JP2023038226 W JP 2023038226W WO 2024090389 A1 WO2024090389 A1 WO 2024090389A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- cooling
- information
- information processing
- processing device
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W20/00—Control systems specially adapted for hybrid vehicles
- B60W20/10—Controlling the power contribution of each of the prime movers to meet required power demand
- B60W20/13—Controlling the power contribution of each of the prime movers to meet required power demand in order to stay within battery power input or output limits; in order to prevent overcharging or battery depletion
- B60W20/14—Controlling the power contribution of each of the prime movers to meet required power demand in order to stay within battery power input or output limits; in order to prevent overcharging or battery depletion in conjunction with braking regeneration
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W20/00—Control systems specially adapted for hybrid vehicles
- B60W20/50—Control strategies for responding to system failures, e.g. for fault diagnosis, failsafe operation or limp mode
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/04—Monitoring the functioning of the control system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
- B60W60/001—Planning or execution of driving tasks
- B60W60/0025—Planning or execution of driving tasks specially adapted for specific operations
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/20—Cooling means
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/20—Cooling means
- G06F1/206—Cooling means comprising thermal management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
- H01L23/00—Details of semiconductor or other solid state devices
- H01L23/34—Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements
- H01L23/46—Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids
- H01L23/473—Arrangements for cooling, heating, ventilating or temperature compensation ; Temperature sensing arrangements involving the transfer of heat by flowing fluids by flowing liquids
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
- H05K7/00—Constructional details common to different types of electric apparatus
- H05K7/20—Modifications to facilitate cooling, ventilating, or heating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
Definitions
- the present invention relates to a cooling execution device, a cooling execution method, a cooling execution program, and a vehicle.
- JP Patent Publication No. 2022-035198 describes a vehicle with an autonomous driving function.
- the system includes an acquisition unit that acquires detection results of an object related to the operation of a control device mounted on the vehicle that controls the automatic driving of the vehicle, and an execution unit that performs cooling in the control device based on the detection results.
- the acquisition unit may extract a point indicating the location of the object from one frame of an image of the object as the detection result, and acquire motion information indicating the movement of the point indicating the location of the object along a predetermined coordinate axis at a frame rate of 100 frames/second or more.
- cooling execution devices may further include a prediction unit that predicts the operation of the control device using the detection results, and the prediction unit may predict the operation of the control device using a learning model generated by machine learning using the detection results and the operating status of the control device at the time the detection results were obtained as learning data.
- the prediction unit may further predict temperature changes in each of a plurality of parts in the control device, and the execution unit may control the cooling of the parts in the control device.
- any of the cooling execution devices may include an acquisition unit that outputs point information that captures the object as a point and identification information that identifies the object from images of the object captured by multiple cameras facing in corresponding directions, and acquires a detection result of the object by an information processing device that associates the point information and the identification information, and an execution unit that executes cooling for the information processing device based on the detection result acquired by the acquisition unit.
- the execution unit may stop cooling the information processing device when a predetermined condition is met.
- the execution unit may stop cooling the information processing device when the predetermined condition is satisfied, that is, the object detected by the information processing device is no longer detected.
- the execution unit may stop cooling the information processing device when the predetermined condition is satisfied that the object detected by the information processing device is moving outside the detection range.
- the execution unit may stop cooling the information processing device after a predetermined time has elapsed if a predetermined condition is met.
- Any of the cooling execution devices may include a prediction unit that predicts the operating status of the information processing device based on the detection result acquired by the acquisition unit, and the execution unit may execute cooling for the information processing device based on the prediction result of the operating status of the information processing device by the prediction unit.
- the prediction unit may predict a temperature change of the information processing device, and the execution unit may execute cooling of the information processing device using a cooling means according to the prediction result of the temperature change of the information processing device by the prediction unit.
- the prediction unit may predict when the temperature of the information processing device will be below a threshold, and the execution unit may stop cooling the information processing device when the predetermined condition is satisfied, that is, the time predicted by the prediction unit.
- the detection result acquired by the acquisition unit may be the point information.
- any of the cooling execution devices may include a prediction unit that predicts the operating status and temperature change of the information processing device based on the detection result acquired by the acquisition unit, and an execution unit that executes cooling of the information processing device based on the prediction result of the operating status of the information processing device by the prediction unit, and adjusts the amount of cooling for the information processing device based on the prediction result of the temperature change of the information processing device by the prediction unit.
- the prediction unit may further predict a temperature peak of the information processing device, and the execution unit may execute cooling of the information processing device with an amount of cooling according to the temperature peak predicted by the prediction unit.
- the execution unit may execute cooling on the information processing device so that the amount of cooling corresponds to the peak temperature before the peak temperature predicted by the prediction unit is reached.
- the information processing device is mounted on a moving body, and the prediction unit predicts that the temperature of the information processing device will rise if the detection result indicates that the object is moving toward the moving body, and the execution unit may increase the amount of cooling for the information processing device when the prediction unit predicts a temperature rise of the information processing device.
- a cooling execution method is provided that is executed by a computer.
- the cooling execution method may obtain a detection result of an object related to the operation of a control device mounted on the vehicle that controls the automatic driving of the vehicle, and execute cooling in the control device based on the detection result.
- a cooling execution program for causing a computer to execute processing.
- the cooling execution program may obtain a detection result of detecting an object related to the operation of a control device mounted on the vehicle that controls the automatic driving of the vehicle, and may cause the control device to execute cooling based on the detection result.
- a vehicle equipped with a control device that controls autonomous driving includes a power unit including a motor, an acquisition unit that acquires detection results of an object related to the operation of the control device, and an execution unit that executes cooling in the control device based on the detection results, and when the vehicle decelerates due to the detection results, the control device controls each part of the vehicle, including the cooling, using regenerative power generated by the motor.
- control device may control the deceleration of the vehicle so that each part of the vehicle is controlled by only the regenerative power from the time when the deceleration of the vehicle starts in response to the detection result until the vehicle stops.
- Any of the vehicles may be provided with a brake actuator that operates the brakes of the vehicle, and the control device may control the brake actuator using the regenerative power when the vehicle decelerates in accordance with the detection result.
- Any of the vehicles may be provided with a steering actuator for steering the vehicle, and the control device may control the steering actuator using the regenerative power when the vehicle decelerates as a result of the detection.
- the acquisition unit may extract a point indicating the location of an object from one frame of an image of the object as the detection result, and acquire motion information indicating the movement of the point indicating the location of the object along a predetermined coordinate axis at a frame rate of 100 frames/second or more.
- Any of the vehicles may further include a prediction unit that predicts operation of the control device using the detection results, and the prediction unit may predict operation of the control device using a learning model generated by machine learning using the detection results and the operating status of the control device at the time the detection results were obtained as learning data.
- the prediction unit may further predict temperature changes in each of a plurality of parts of the control device, and the execution unit may control the cooling of the parts of the control device.
- FIG. 11 is an explanatory diagram for explaining the learning phase in the system 1.
- FIG. 2 is an explanatory diagram for explaining a cooling execution phase in the system 1.
- An example of a SoCBox 400 and a cooling unit 600 are shown in schematic form.
- An example of a SoCBox 400 and a cooling unit 600 are shown in schematic form.
- An example of a SoCBox 400 and a cooling unit 600 are shown in schematic form.
- 13 shows a schematic diagram of a vehicle according to a modified example. This is a schematic diagram showing an example of a vehicle equipped with a Central Brain.
- 1 is a first block diagram illustrating an example of a configuration of an information processing device.
- FIG. 1 is a first block diagram illustrating an example of a configuration of an information processing device.
- FIG. 2 is a second block diagram showing an example of the configuration of the information processing device.
- FIG. 11 is an explanatory diagram showing an example of point information output by a MoPU;
- FIG. 13 is a third block diagram showing an example of the configuration of an information processing device.
- FIG. 4 is a fourth block diagram showing an example of the configuration of an information processing device.
- FIG. 11 is an explanatory diagram showing an example of association between point information and label information.
- FIG. 5 is a fifth block diagram showing an example of the configuration of an information processing device.
- FIG. 6 is a sixth block diagram showing an example of the configuration of an information processing device.
- FIG. 1 is a diagram illustrating detection of coordinates of an object in a time series
- FIG. 7 is a seventh block diagram showing an example of the configuration of an information processing device.
- FIG. 2 is an explanatory diagram for explaining an image of an object captured by an event camera.
- 1 is an explanatory diagram illustrating an example of a hardware configuration of a computer that functions as a management server, a SoCBox, an information processing device, or a cooling execution device;
- a cooling execution device that detects the operation of the SoCBox processing and executes cooling of the SoCBox.
- the SoCBox may be activated when a moving object on the road is detected. For example, if the SoCBox detects a moving object on the road while the vehicle is driving autonomously, it may perform calculations to control the object. However, the SoCBox heats up quickly, making it difficult to perform advanced calculations in the vehicle (a challenge to fully autonomous driving). Therefore, it is conceivable that by detecting moving objects on the road, the heat dissipation caused by the SoCBox's operation can be predicted and the SoCBox can be cooled. For example, by predicting heat dissipation from the SoCBox by detecting moving objects and cooling it at the same time as heat dissipation, the SoCBox can be prevented from becoming too hot and advanced calculations can be made possible in the vehicle.
- heat dissipation from the SoC Box can be predicted and simultaneously cooled, preventing the SoC Box from becoming too hot and enabling advanced calculations in the vehicle.
- FIG. 1 shows an example of a system 1.
- the system 1 includes a management server 101, a SoCBox 400, a cooling execution device 500, and a cooling unit 600.
- the SoCBox 400, cooling execution device 500, and cooling unit 600 are mounted on a vehicle.
- the SoCBox 400 controls the autonomous driving of the vehicle using sensor values from multiple sensors mounted on the vehicle. Because controlling the autonomous driving of the vehicle places a very high processing load on the vehicle, the SoCBox 400 can become very hot. If the SoCBox 400 becomes too hot, it may not operate normally and may have a negative effect on the vehicle.
- the cooling execution device 500 for example, predicts the operation of the SoCBox 400 and starts cooling the SoCBox 400 based on the operation. For example, when the cooling execution device 500 detects a moving object that is the cause of the operation of the SoCBox 400, it immediately starts cooling the SoCBox 400. By starting cooling before the start of heat generation caused by the operation of the SoCBox 400 or at the same time as the start of heat generation, it is possible to reliably prevent the SoCBox 400 from becoming too hot.
- the cooling execution device 500 may use AI to predict the operation of the SoCBox 400. Learning of the operation of the SoCBox 400 may be performed by using data collected by the vehicle 200.
- the management server 101 collects data from the vehicle 200 and performs the learning.
- the entity that performs the learning is not limited to the management server 101, and may be another device.
- the vehicle 200 is equipped with a SoCBox 400 and a MoPU (Motion Processing Unit) 217 as a sensor.
- the SoCBox 400 controls the autonomous driving of the vehicle 200 using the sensor values of multiple sensors, including the MoPU 217, that are equipped in the vehicle 200, and external information received from multiple types of servers 31.
- MoPU 217 may be built into a low-resolution camera (not shown) installed in the vehicle.
- MoPU 217 outputs motion information indicating the motion of the photographed object at a frame rate of, for example, 100 frames per second.
- MoPU 217 outputs vector information of the motion of a point indicating the location of the object along a specified coordinate axis as motion information.
- the motion information output from MoPU 217 does not include information necessary to identify what the photographed object is (e.g., whether it is a person or an obstacle), but only includes information indicating the motion (direction and speed of movement) of the object's center point (or center of gravity) on the coordinate axes (x-axis, y-axis, z-axis).
- Server 31 may be an example of an external device. Examples of multiple types of servers 31 include a server that provides traffic information, a server that provides weather information, etc. SoCBox 400 transmits to management server 101 sensor values and external information, including detection results by MoPU 217, used to control autonomous driving, the operating status of SoCBox 400 when controlled, and the temperature change of SoCBox 400 when controlled.
- the management server 101 performs learning using information received from one or more SoCBox 400 and multiple sensors.
- the management server 101 performs machine learning using the detection results of MoPU 217 acquired by the SoCBox 400 and the operating status of the SoCBox 400 when the SoCBox 400 acquired this information as learning data, thereby generating a learning model in which the information acquired by the SoCBox 400 is used as input and the operating status of the SoCBox 400 is used as output.
- Vehicle 300 is a vehicle having a cooling function according to this embodiment.
- Vehicle 300 is equipped with MoPU 217 as a sensor, SoCBox 400, cooling execution device 500, and cooling unit 600.
- Cooling execution device 500 may receive from management server 101 and store the learning model generated by management server 101.
- the cooling execution device 500 may obtain sensor values from multiple sensors mounted on the vehicle 300, including detection results from the MoPU 217, from the multiple sensors or from the SoCBox 400, and when it predicts that the SoCBox 400 will operate, it may start cooling the SoCBox 400 with the cooling unit 600. Specifically, when the cooling execution device 500 detects a moving object or the like as a result of detection by the MoPU 217, it predicts that the SoCBox 400 will operate, and starts cooling the SoCBox 400 with the cooling unit 600.
- the cooling execution device 500 may also obtain sensor values from multiple sensors mounted on the vehicle 300, including detection results from the MoPU 217, from the multiple sensors or from the SoCBox 400, and input the obtained information into a learning model to predict the operation of the SoCBox 400.
- the SoCBox 400, the cooling execution device 500, the management server 101, and the server 31 may communicate via the network 20.
- the network 20 may include a vehicle network.
- the network 20 may include the Internet.
- the network 20 may include a Local Area Network (LAN).
- the network 20 may include a mobile communication network.
- the mobile communication network may conform to any of the following communication methods: 5G (5th Generation) communication method, LTE (Long Term Evolution) communication method, 3G (3rd Generation) communication method, and 6G (6th Generation) communication method or later.
- Figure 2 is an explanatory diagram for explaining the learning phase in system 1.
- sensors 210 mounted on vehicle 200 include a camera 211, a LiDAR (Light Detection and Ranging) 212, a millimeter wave sensor 213, an ultrasonic sensor 214, an IMU sensor 215, a GNSS (Global Navigation Satellite System) sensor 216, an MoPU 217, and a temperature sensor 218.
- Vehicle 200 does not have to be equipped with all of these, and may be equipped with some or other sensors other than these.
- SoCBox400 acquires sensor information from each sensor included in sensor 210. SoCBox400 may also perform communication via network 20, and receives external information from each of multiple servers 31 via network 20. SoCBox400 then uses the acquired information to execute autonomous driving control of vehicle 200.
- SoCBox400 records the computing power as the operation of SoCBox400.
- SoCBox400 may record its computing power periodically or irregularly.
- SoCBox400 may record the sensor information received from sensor 210, the external information received from server 31, and its computing power when it acquires this information and executes autonomous driving control, and transmit this to management server 101.
- the management server 101 includes an information acquisition unit 102, a model generation unit 104, and a model provision unit 106.
- the information acquisition unit 102 acquires various information.
- the management server 101 may receive information transmitted by the SoCBox 400.
- the model generation unit 104 performs machine learning using the information acquired by the information acquisition unit 102 to generate a learning model.
- the model generation unit 104 may generate a learning model in which the information acquired by SoCBox 400 is input and the operating status of SoCBox 400 is output by executing machine learning using the information acquired by SoCBox 400 and the operating status of SoCBox 400 at the time SoCBox 400 acquired the information as learning data. Specifically, the model generation unit 104 inputs the detection result by MoPU 217 as the information acquired by SoCBox 400, and generates a learning model in which the status and amount of change of the power computing power are output as the operating status of SoCBox 400.
- the model providing unit 106 provides the learning model generated by the model generating unit 104.
- the model providing unit 106 may transmit the learning model to the cooling execution device 500 mounted on the vehicle 300.
- System 1 may be configured to predict temperature changes in each of multiple parts of SoCBox 400.
- vehicle 200 may be equipped with multiple temperature sensors 218 that each measure the temperature change in each of multiple parts of SoCBox 400.
- SoCBox 400 may transmit to management server 101 the sensor information received from sensor 210, external information received from server 31, and the temperature changes measured by the multiple temperature sensors 218 when acquiring this information and executing autonomous driving control.
- the model generation unit 104 performs machine learning using the information acquired by SoCBox 400, the temperature changes in each of the multiple parts of SoCBox 400 when SoCBox 400 acquired the information, and the operating status of SoCBox 400 as learning data, thereby generating a learning model that uses the information acquired by SoCBox 400 as input and the temperature changes in each of the multiple parts of SoCBox 400 and the operating status of SoCBox 400 as output.
- FIG. 3 is an explanatory diagram for explaining the cooling execution phase in system 1.
- sensors 310 mounted on vehicle 300 include a camera 311, a LiDAR 312, a millimeter wave sensor 313, an ultrasonic sensor 314, an IMU sensor 315, a GNSS sensor 316, an MoPU 317, and a temperature sensor 318.
- Vehicle 300 does not have to be equipped with all of these, and may be equipped with some or other sensors.
- the cooling execution device 500 includes an information acquisition unit 502, a cooling execution unit 504, a model storage unit 506, and a prediction unit 508.
- the information acquisition unit 502 acquires information acquired by the SoCBox 400.
- the information acquisition unit 502 acquires the sensor information that the SoCBox 400 acquires from the sensor 310 from the sensor 310 or from the SoCBox 400.
- the information acquisition unit 502 may receive from the SoCBox 400 the sensor information that the SoCBox 400 acquires from the sensor 310.
- the information acquisition unit 502 may receive from the sensor 310 the same sensor information that the SoCBox 400 acquires from the sensor 310. In this case, each sensor of the sensors 310 may transmit the sensor information to the SoCBox 400 and the cooling execution device 500, respectively.
- the cooling execution unit 504 starts cooling the SoCBox 400 based on the detection result by the MoPU 217 contained in the sensor information. For example, the cooling execution unit 504 starts cooling the SoCBox 400 when the detection result by the MoPU 217 indicates that a moving object has been detected.
- the cooling execution unit 504 uses the cooling unit 600 to perform cooling of the SoCBox 400.
- the cooling unit 600 cools the SoCBox 400 by air cooling means, water cooling means, or liquid nitrogen cooling means.
- the operation of SoCBox 400 is predicted when a moving object is detected.
- Prediction unit 508 may predict the operation of SoCBox 400 using AI.
- the model storage unit 506 stores the learning model received from the management server 101.
- the prediction unit 508 predicts the operating status of the SoCBox 400 by inputting the detection results by the MoPU 217 acquired by the information acquisition unit 502 into the learning model stored in the model storage unit 506.
- the learning model outputs the status and amount of change of the computing power of the SoCBox 400 as the operating status.
- the prediction unit 508 may also predict and output the temperature changes of each of the multiple parts of the SoCBox 400 along with the operating status.
- the cooling execution unit 504 may start cooling the SoCBox 400 in response to the operation status of the SoCBox 400 predicted by the AI. For example, the cooling execution unit 504 starts cooling the SoCBox 400 when the power computing power status of the SoCBox 400 predicted as the operation status and the amount of change exceed a predetermined threshold.
- the cooling execution unit 504 may also start cooling the SoCBox 400 in response to the operation status of the SoCBox 400 predicted by the AI and the temperature change of the parts in the SoCBox 400. For example, the cooling execution unit 504 starts cooling the corresponding parts in the SoCBox 400 in response to the power computing power status of the SoCBox 400 predicted as the operation status and the amount of change exceed a predetermined threshold and when the temperature change exceeds a predetermined threshold.
- the cooling unit 600 may also include multiple types of cooling means.
- the cooling unit 600 may include multiple types of air cooling means.
- the cooling unit 600 may include multiple types of water cooling means.
- the cooling unit 600 may include multiple types of liquid nitrogen cooling means.
- the cooling unit 600 may include multiple of one or multiple types of air cooling means, one or multiple types of water cooling means, and one or multiple liquid nitrogen cooling means.
- the multiple cooling means may be arranged so that each cooling means cools a different part of the SoCBox 400.
- the prediction unit 508 may predict temperature changes in each of the multiple parts of the SoCBox 400 using information acquired by the information acquisition unit 502.
- the cooling execution unit 504 may start cooling the SoCBox 400 using a cooling means selected from the multiple cooling means that cool each of the multiple parts of the SoCBox 400 based on the prediction result by the prediction unit 508.
- the cooling execution unit 504 may cool the SoCBox 400 using a cooling means according to the temperature of the SoCBox 400 predicted by the prediction unit 508. For example, the higher the temperature of the SoCBox 400, the more cooling means the cooling execution unit 504 uses to cool the SoCBox 400. As a specific example, when it is predicted that the temperature of the SoCBox 400 will exceed a first threshold, the cooling execution unit 504 starts cooling using one of the multiple cooling means, and when it is still predicted that the temperature of the SoCBox 400 will rise and exceed a second threshold, the cooling execution unit 504 increases the number of cooling means to be used.
- the cooling execution unit 504 may use a more powerful cooling means to cool the SoCBox 400 as the temperature of the SoCBox 400 increases. For example, the cooling execution unit 504 may start cooling using air cooling means when it is predicted that the temperature of the SoCBox 400 will exceed a first threshold, start cooling using water cooling means when it is still predicted that the temperature of the SoCBox 400 will rise and exceed a second threshold, and start cooling using liquid nitrogen cooling means when it is still predicted that the temperature of the SoCBox 400 will rise and exceed a third threshold.
- SoCBox400 may have multiple processing chips, each of which may be located at a different position on SoCBox400.
- Each of the multiple cooling means may be located at a position corresponding to each of the multiple processing chips.
- cooling will be performed by the cooling means corresponding to the processing chip being used, thereby achieving efficient cooling.
- the SoCBox 400 is cooled when a moving object is detected by the MoPU 217.
- the present invention is not limited to this.
- power regeneration using heat generated by the SoCBox 400 may be started.
- a Peltier element may be installed in the SoCBox 400, and when a moving object is detected by the MoPU 217, power generation by the Peltier element using the heat generated by the SoCBox 400 may be started.
- power can be obtained immediately when the SoCBox 400 generates heat, improving energy efficiency.
- FIG. 4 shows an example of the SoCBox 400 and the cooling unit 600.
- FIG. 4 shows an example in which the cooling unit 600 is configured with a single cooling means.
- the cooling execution device 500 detects a moving object, the cooling unit 600 starts cooling, thereby cooling the entire SoCBox 400.
- FIG. 5 shows an example of a schematic diagram of the SoCBox 400 and the cooling unit 600.
- FIG. 5 shows an example where the cooling unit 600 is configured with multiple cooling means for cooling each of the multiple parts of the SoCBox 400.
- the cooling execution device 500 uses the temperature sensor 318 to predict temperature changes in each of the multiple parts of the SoCBox 400, and when it predicts that any part will start to generate heat or that the temperature of any part will exceed a predetermined threshold, performs cooling using only the cooling means corresponding to that part, thereby achieving efficient cooling.
- FIG. 6 shows an example of the SoCBox 400 and the cooling unit 600.
- FIG. 6 shows an example in which the cooling unit 600 is configured with two types of cooling means.
- the cooling execution device 500 predicts the temperature change of each of the multiple parts of the SoCBox 400, and when it predicts that any part will start to generate heat or that the temperature of any part will exceed a predetermined threshold, performs cooling using only the cooling means corresponding to that part, thereby achieving efficient cooling.
- the cooling execution device 500 increases the number of cooling means used as the temperature of the SoCBox 400 increases; in other words, in this example, first, cooling using one of the two types of cooling means is started, and when the temperature of the SoCBox 400 further increases, cooling using the other cooling means is started, thereby making it possible to efficiently use energy for cooling.
- cooling execution unit 504 uses cooling unit 600 to cool SoCBox 400 and decelerates vehicle 300. Then, as described below, the cooling execution device 500, cooling unit 600, etc. are driven using regenerative power (regenerative energy) generated when vehicle 300 decelerates.
- the vehicle 300 of this embodiment includes a power unit 704 for driving including a motor 702, a battery 706 that supplies power to the motor 702, and a power control unit 708.
- the power unit 704 may include only the motor 702, or may include both the motor 702 and an internal combustion engine.
- the motor 702 can generate the driving force required for the vehicle 300 to run, and can also function as a generator to generate regenerative power when the vehicle 300 is decelerating.
- the power control unit 708 can adjust the power sent from the battery 706 to the motor 702, and can send regenerative power to the cooling execution device 500 and the cooling section 600 when the vehicle decelerates. Furthermore, the power control unit 708 can adjust the amount of regenerative power generated by the motor 702 based on instructions from the SoCBox 400, and adjust the deceleration of the vehicle 300. In other words, the amount of regenerative power can be adjusted by adjusting the deceleration of the vehicle 300.
- regenerative power is generated when the vehicle decelerates (indicating that a moving object has been detected), and this regenerative power can be supplied to the cooling execution device 500 and the cooling unit 600, thereby reducing the amount of power used by the battery 706. Furthermore, by adjusting the deceleration rate, all of the power used by the cooling execution device 500 and the cooling unit 600 can be covered by regenerative power, and at least the cooling execution device 500 and the cooling unit 600 can be prevented from using power from the battery 706 during deceleration.
- the power control unit 708 can also supply the regenerative power to power-using devices other than the cooling execution device 500 and the cooling unit 600 (such as a brake actuator 710 and a steering actuator 712, which will be described later). In addition, if there is a surplus of regenerative power, the power control unit 708 can also use the surplus regenerative power to charge the battery 706.
- the vehicle 300 has a brake actuator 710 that performs the final braking operation for deceleration, and a steering actuator 712 that performs the steering operation of the vehicle 300 based on the detection results of the MoPU 317.
- the power used by the brake actuator 710 and the power used by the steering actuator 712 can be covered by regenerative power.
- the steering actuator 712 can operate the steering wheel (not shown) of the vehicle 300 to cause the vehicle 300 to avoid the object.
- the vehicle can be safely stopped by operating the brake actuator 710 using regenerative power.
- SoCBox 400 can control the deceleration of vehicle 300 by brake actuator 710 so that the power used by at least the control system, cooling system, and brake system, such as SoCBox 400 (an example of a control device), cooling execution device 500, cooling section 600, power control unit 708, and brake actuator 710, can be met by regenerative power alone.
- SoCBox 400 an example of a control device
- SoCBox 400 increases the deceleration to increase the amount of regenerative power.
- the power of battery 706 is used to cool SoCBox 400, and once regeneration is activated, power control unit 708 can be operated to use the regenerative power to cool SoCBox 400.
- SoCBox400 controls power control unit 708 and can switch the supply destination of regenerative power depending on the remaining charge of battery 706. In this case, however, cooling of SoCBox400 is given top priority. If battery 706 has some charge left, the power of battery 706 can also be used for cooling, etc.
- the information processing device can provide a driving system that can realize autonomous driving in real time based on data obtained by AI/multivariate analysis/goal seek/strategy planning/optimal probability solution/optimal speed solution/optimal course management/multiple sensor inputs at the edge by Level 6 and is adjusted based on the delta optimal solution.
- the vehicle 100 is an example of an "object”.
- Level 6 refers to a level of autonomous driving, and is equivalent to a level higher than Level 5, which represents fully autonomous driving.
- Level 5 represents fully autonomous driving, it is at the same level as a human driving, and there is still a chance of accidents occurring.
- Level 6 represents a level higher than Level 5, and is equivalent to a level where the chance of accidents occurring is lower than at Level 5.
- the computing power at Level 6 is about 1000 times that of Level 5. Therefore, it is possible to achieve high-performance operation control that was not possible at Level 5.
- FIG 8 is a schematic diagram showing an example of a vehicle 100 equipped with a Central Brain 15.
- a plurality of Gate Ways are communicatively connected to the Central Brain 15.
- the Central Brain 15 is connected to an external cloud server via the Gate Ways.
- the Central Brain 15 is configured to be able to access the external cloud server via the Gate Ways.
- the Central Brain 15 is configured not to be able to be directly accessed from the outside.
- the Central Brain 15 outputs a request signal to the cloud server every time a predetermined time has elapsed. Specifically, the Central Brain 15 outputs a request signal representing an inquiry to the cloud server every billionth of a second. As an example, the Central Brain 15 controls the automatic operation of Level L6 based on multiple pieces of information acquired via the Gate Way.
- FIG. 9 is a first block diagram showing an example of the configuration of an information processing device 10.
- the information processing device 10 includes an IPU (Image Processing Unit) 11, a MoPU (Motion Processing Unit) 12, a Central Brain 15, and a memory 16.
- the Central Brain 15 includes a GNPU (Graphics Neural network Processing Unit) 13 and a CPU (Central Processing Unit) 14.
- the IPU 11 is built into an ultra-high resolution camera (not shown) installed in the vehicle 100.
- the IPU 11 performs predetermined image processing such as Bayer transformation, demosaicing, noise removal, and sharpening on the image of the object existing around the vehicle 100 captured by the ultra-high resolution camera, and outputs the processed image of the object at a frame rate of 10 frames/second and a resolution of 12 million pixels, for example.
- the IPU 11 also outputs identification information that identifies the captured object from the image of the object captured by the ultra-high resolution camera.
- the identification information is information necessary to identify what the captured object is (for example, whether it is a person or an obstacle).
- the IPU 11 outputs label information indicating the type of the captured object (for example, information indicating whether the captured object is a dog, a cat, or a bear) as the identification information. Furthermore, the IPU 11 outputs position information indicating the position of the captured object in the camera coordinate system of the ultra-high resolution camera.
- the image, label information, and position information output from the IPU 11 are supplied to the Central Brain 15 and the memory 16.
- the IPU 11 is an example of a "second processor," and the ultra-high resolution camera is an example of a "second camera.”
- MoPU12 is built into a separate camera (not shown) other than the ultra-high resolution camera installed in the vehicle 100.
- MoPU12 outputs point information capturing the captured object as a point from an image of the object captured at a frame rate of 100 frames/second or more by a separate camera facing in a direction corresponding to the ultra-high resolution camera, for example at a frame rate of 100 frames/second or more.
- the point information output from MoPU12 is supplied to the Central Brain15 and the memory 16. In this way, the image used by MoPU12 to output the point information and the image used by IPU11 to output the identification information are images captured by the separate camera and the ultra-high resolution camera facing in corresponding directions.
- the "corresponding direction” is the direction in which the shooting range of the separate camera and the shooting range of the ultra-high resolution camera overlap.
- the separate camera faces in a direction overlapping with the shooting range of the ultra-high resolution camera to capture the object.
- photographing an object with the ultra-high resolution camera and another camera facing in corresponding directions can be achieved, for example, by determining in advance the correspondence between the camera coordinate systems of the ultra-high resolution camera and the other camera.
- MoPU12 outputs, as point information, coordinate values of at least two coordinate axes in a three-dimensional orthogonal coordinate system of a point indicating the location of an object.
- the coordinate values indicate the center point (or center of gravity) of the object.
- MoPU12 outputs, as the coordinate values of the two coordinate axes, the coordinate value of the axis (x-axis) along the width direction in the three-dimensional orthogonal coordinate system (hereinafter referred to as the "x-coordinate value”) and the coordinate value of the axis (y-axis) along the height direction (hereinafter referred to as the "y-coordinate value").
- the x-axis is an axis along the width direction of the vehicle 100
- the y-axis is an axis along the height direction of the vehicle 100.
- the point information for one second output by MoPU12 includes x and y coordinate values for more than 100 frames, so based on this point information it is possible to grasp the movement (direction and speed of movement) of an object on the x and y axes in the above three-dimensional orthogonal coordinate system.
- the point information output by MoPU12 includes position information indicating the position of the object in the above three-dimensional orthogonal coordinate system and movement information indicating the movement of the object.
- the point information output from MoPU12 does not include information necessary to identify what the photographed object is (for example, whether it is a person or an obstacle), but only includes information indicating the movement (direction and speed of movement) of the object's center point (or center of gravity) on the x and y axes. And because the point information output from MoPU12 does not include image information, it is possible to dramatically reduce the amount of data output to Central Brain15 and memory 16.
- MoPU12 is an example of a "first processor"
- the separate camera is an example of a "first camera”.
- the frame rate of the separate camera incorporating MoPU12 is greater than the frame rate of the ultra-high resolution camera incorporating IPU11.
- the frame rate of the separate camera is 100 frames/second or more
- the frame rate of the ultra-high resolution camera is 10 frames/second.
- the frame rate of the separate camera is 10 times or more the frame rate of the ultra-high resolution camera.
- the Central Brain 15 associates the point information output from the MoPU 12 with the label information output from the IPU 11. For example, due to the difference in frame rate between the separate camera and the ultra-high resolution camera, there are states in which the Central Brain 15 acquires point information about an object but does not acquire label information. In this state, the Central Brain 15 recognizes the x and y coordinate values of the object based on the point information, but does not recognize what the object is.
- the Central Brain 15 derives the type of the label information (e.g., PERSON). The Central Brain 15 then associates the label information with the point information acquired above. As a result, the Central Brain 15 recognizes the x-coordinate value and y-coordinate value of the object based on the point information, and recognizes what the object is.
- the Central Brain 15 is an example of a "third processor.”
- the Central Brain 15 associates point information and label information for each object as follows. Due to the difference in frame rate between the other camera and the ultra-high resolution camera, the Central Brain 15 acquires point information for objects A and B (hereinafter referred to as "point information A” and "point information B"), but there are states in which it does not acquire label information. In this state, the Central Brain 15 recognizes the x and y coordinate values of object A based on point information A, and recognizes the x and y coordinate values of object B based on point information B, but does not recognize what those objects are.
- the Central Brain 15 derives the type of the piece of label information (e.g., PERSON). Then, the Central Brain 15 identifies point information to be associated with the piece of label information based on the position information output from the IPU 11 together with the piece of label information and the position information included in the acquired point information A and point information B. For example, the Central Brain 15 identifies point information including position information indicating a position closest to the position of the object indicated by the position information output from the IPU 11, and associates the point information with the piece of label information. If the point information identified above is point information A, the Central Brain 15 associates the piece of label information with point information A, recognizes the x-coordinate value and y-coordinate value of object A based on point information A, and recognizes what object A is.
- point information e.g., PERSON
- the Central Brain 15 associates the point information with the label information based on the position information output from the IPU 11 and the position information included in the point information output from the MoPU 12.
- the Central Brain 15 recognizes objects (people, animals, roads, traffic lights, signs, crosswalks, obstacles, buildings, etc.) around the vehicle 100 based on the image and label information output from the IPU 11. In addition, the Central Brain 15 recognizes the position and movement of recognized objects around the vehicle 100 based on the point information output from the MoPU 12. Based on the recognized information, the Central Brain 15 controls the automatic driving of the vehicle 100, for example, by controlling the motor that drives the wheels (speed control), brake control, and steering control. For example, the Central Brain 15 controls the automatic driving of the vehicle 100 to avoid collisions with objects based on the position information and movement information contained in the point information output from the MoPU 12. In the Central Brain 15, the GNPU 13 may be responsible for processing related to image recognition, and the CPU 14 may be responsible for processing related to vehicle control.
- Ultra-high resolution cameras are generally used for image recognition in autonomous driving. Here, it is possible to recognize what objects are included in an image captured by an ultra-high resolution camera. However, this alone is insufficient for automated driving in the Level 6 era. In the Level 6 era, it is also necessary to recognize the movement of objects with higher accuracy.
- the vehicle 100 traveling by automated driving can perform an avoidance operation to avoid an obstacle with higher accuracy.
- an ultra-high resolution camera can only acquire images at about 10 frames per second, and the accuracy of analyzing the movement of an object is lower than that of a camera equipped with the MoPU 12.
- a camera equipped with the MoPU 12 can output at a high frame rate of, for example, 100 frames/second.
- the information processing device 10 has two independent processors, an IPU 11 and a MoPU 12.
- the IPU 11 built into the ultra-high resolution camera is given the role of acquiring information necessary to identify what the photographed object is
- the MoPU 12 built into another camera is given the role of detecting the position and movement of the object.
- the MoPU 12 captures the photographed object as a point, and analyzes in which direction and at what speed the coordinates of that point move on at least the x-axis and y-axis in the above-mentioned three-dimensional orthogonal coordinate system. Since the entire contour of the object and what the object is can be detected from the image from the ultra-high resolution camera, for example, if only the way in which the center point of the object moves can be known by the MoPU 12, the behavior of the entire object can be known.
- a method of analyzing only the movement and speed of the center point of an object makes it possible to significantly reduce the amount of data output to the Central Brain 15 and the amount of calculations in the Central Brain 15, compared to determining how the entire image of the object moves. For example, when an image of 1000 pixels x 1000 pixels is output to the Central Brain 15 at a frame rate of 1000 frames per second, including color information, 4 billion bits per second of data will be output to the Central Brain 15.
- the MoPU 12 output only point information indicating the movement of the center point of the object, the amount of data output to the Central Brain 15 can be compressed to 20,000 bits per second. In other words, the amount of data output to the Central Brain 15 is compressed to 1/200,000.
- the Central Brain 15 associates the point information output from the MoPU 12 with the label information output from the IPU 11, thereby making it possible to grasp information regarding what objects are moving in what way.
- FIG. 10 is a second block diagram showing an example of the configuration of the information processing device 10.
- the information processing device 10 mounted on the vehicle 100 includes a MoPU 12L corresponding to the left eye, a MoPU 12R corresponding to the right eye, an IPU 11, and a Central Brain 15.
- MoPU12L includes a camera 30L, a radar 32L, an infrared camera 34L, and a core 17L.
- MoPU12R includes a camera 30R, a radar 32R, an infrared camera 34R, and a core 17R.
- MoPU12L and MoPU12R when there is no distinction between MoPU12L and MoPU12R, it will be referred to as “MoPU12”, when there is no distinction between camera 30L and camera 30R, it will be referred to as “camera 30”, when there is no distinction between radar 32L and radar 32R, it will be referred to as “radar 32”, when there is no distinction between infrared camera 34L and infrared camera 34R, it will be referred to as “infrared camera 34”, and when there is no distinction between core 17L and core 17R, it will be referred to as "core 17".
- the camera 30 of the MoPU 12 captures images of objects at a frame rate (120, 240, 480, 960, or 1920 frames/sec) greater than the ultra-high resolution camera of the IPU 11 (e.g., 10 frames/sec).
- the frame rate of the camera 30 is variable.
- the camera 30 is an example of a "first camera.”
- the radar 32 included in the MoPU 12 acquires a radar signal, which is a signal based on the electromagnetic waves irradiated onto an object and reflected from the object.
- the infrared camera 34 included in the MoPU 12 is a camera that captures infrared images.
- the core 17 (e.g., composed of one or more CPUs) provided in the MoPU 12 extracts feature points for each frame of image captured by the camera 30, and outputs the x- and y-coordinate values of the object in the above-mentioned three-dimensional orthogonal coordinate system as point information.
- the core 17 takes the center point (center of gravity) of the object extracted from the image as the feature point.
- the point information output by the core 17 includes position information and movement information, as in the above embodiment.
- the IPU11 is equipped with an ultra-high resolution camera (not shown) and outputs an image of an object captured by the ultra-high resolution camera, label information indicating the type of the object, and position information indicating the position of the object in the camera coordinate system of the ultra-high resolution camera.
- the Central Brain 15 acquires the point information output from the MoPU 12, and the image, label information, and position information output from the IPU 11. The Central Brain 15 then associates the point information with the label information about an object that exists at a position corresponding to the position information included in the point information output from the MoPU 12 and the position information output from the IPU 11. This makes it possible for the information processing device 10 to associate information about the object indicated by the label information with the position and movement of the object indicated by the point information.
- MoPU 12 changes the frame rate of camera 30 in response to a predetermined factor.
- MoPU 12 changes the frame rate of camera 30 in response to a score related to the external environment, which is an example of a predetermined factor.
- MoPU 12 calculates a score related to the external environment for vehicle 100, and changes the frame rate of camera 30 in response to the calculated score.
- MoPU 12 then outputs a control signal to camera 30 to cause it to capture images at the changed frame rate.
- camera 30 captures images at the frame rate indicated by the control signal.
- the information processing device 10 can capture images of objects at a frame rate suitable for the external environment.
- the information processing device 10 mounted on the vehicle 100 is equipped with multiple types of sensors (not shown).
- the MoPU 12 calculates the degree of danger related to the movement of the vehicle 100 as a score related to the external environment for the vehicle 100 based on the sensor information (e.g., weight center of gravity shift, detection of road material, detection of outside air temperature, detection of outside air humidity, detection of up, down, side, diagonal inclination angle of a slope, road freezing condition, detection of moisture content, material of each tire, wear condition, detection of air pressure, road width, presence or absence of overtaking prohibition, oncoming vehicles, vehicle type information of the front and rear vehicles, cruising state of those vehicles, or surrounding conditions (birds, animals, soccer balls, wrecked vehicles, earthquakes, housework, wind, typhoon, heavy rain, light rain, snowstorm, fog, etc.) etc.) acquired from the multiple types of sensors and point information.
- the sensor information e.g., weight center of gravity shift, detection of road material, detection of outside air temperature, detection of outside air humidity,
- the degree of danger indicates the degree of danger in which the vehicle 100 will be traveling in the future.
- the MoPU 12 changes the frame rate of the camera 30 according to the calculated degree of danger.
- the vehicle 100 is an example of a "moving body.” With this configuration, the information processing device 10 can change the frame rate of the camera 30 depending on the degree of danger related to the movement of the vehicle 100.
- the sensor is an example of a "detection unit,” and the sensor information is an example of "detection information.”
- the MoPU 12 increases the frame rate of the camera 30 as the calculated risk level increases.
- the MoPU 12 changes the frame rate of the camera 30 to 120 frames/second.
- the MoPU 12 changes the frame rate of the camera 30 to one of 240, 480, or 960 frames/second.
- the MoPU 12 changes the frame rate of the camera 30 to 1920 frames/second.
- the MoPU 12 may output a control signal to the radar 32 and the infrared camera 34 to acquire a radar signal and capture an infrared image at a numerical value corresponding to the frame rate, in addition to causing the camera 30 to capture an image at the selected frame rate.
- the MoPU 12 changes the frame rate of the camera 30 to one of 240, 480, or 960 frames/sec.
- the MoPU 12 changes the frame rate of the camera 30 to 120 frames/sec.
- the MoPU 12 changes the frame rate of the camera 30 to 120 frames/sec.
- a control signal may be output to the radar 32 and the infrared camera 34 so that the radar signal is acquired and the infrared image is captured at a value corresponding to the changed frame rate of the camera 30.
- MoPU12 may also calculate the risk level using big data related to driving that is known before the vehicle 100 starts driving, such as long-tail incident AI (Artificial Intelligence) DATA (e.g., trip data of a vehicle equipped with a level 5 autonomous driving control system) or map information, as information for predicting the risk level.
- big data related to driving such as long-tail incident AI (Artificial Intelligence) DATA (e.g., trip data of a vehicle equipped with a level 5 autonomous driving control system) or map information, as information for predicting the risk level.
- the degree of danger is calculated as a score related to the external environment, but the index that becomes the score related to the external environment is not limited to the degree of danger.
- MoPU 12 may calculate a score related to the external environment other than the degree of danger based on the moving direction or speed of an object captured by camera 30, and change the frame rate of camera 30 according to the score.
- MoPU 12 calculates a speed score that is a score related to the speed of an object captured by camera 30, and changes the frame rate of camera 30 according to the speed score.
- the speed score is set to be higher as the speed of the object is faster and lower as the speed of the object is slower.
- MoPU 12 increases the frame rate of camera 30 as the calculated speed score is higher and decreases the frame rate of camera 30 as the calculated speed score is lower. For this reason, when the calculated speed score is equal to or higher than a threshold value because the speed of the object is fast, MoPU 12 changes the frame rate of camera 30 to 1920 frames/second. Furthermore, if the speed score calculated is less than the threshold because the object's speed is slow, the MoPU 12 changes the frame rate of the camera 30 to 120 frames/second.
- a control signal may be output to the radar 32 and the infrared camera 34 so that the radar signal is acquired and the infrared image is captured at a value corresponding to the changed frame rate of the camera 30.
- MoPU 12 calculates a direction score, which is a score related to the moving direction of an object captured by camera 30, and changes the frame rate of camera 30 according to the direction score.
- the direction score is set to be higher when the moving direction of the object is toward the road and lower when the moving direction is away from the road.
- MoPU 12 increases the frame rate of camera 30 as the calculated direction score increases, and decreases the frame rate of camera 30 as the calculated direction score decreases.
- MoPU 12 identifies the moving direction of the object by using AI or the like, and calculates the direction score based on the identified moving direction.
- MoPU 12 changes the frame rate of camera 30 to 1920 frames/second.
- MoPU 12 changes the frame rate of camera 30 to 120 frames/second.
- a control signal may be output to the radar 32 and the infrared camera 34 so that the radar signal is acquired and the infrared image is captured at a value corresponding to the changed frame rate of the camera 30.
- MoPU12 may also output point information only for objects whose calculated score for the external environment is equal to or greater than a predetermined threshold. In this case, for example, MoPU12 may determine whether or not to output point information for an object based on the moving direction of the object captured by the camera 30. For example, MoPU12 may not output point information for an object that has a low impact on the traveling of the vehicle 100. Specifically, MoPU12 calculates the moving direction of an object captured by the camera 30, and does not output point information for an object such as a pedestrian moving away from the road. On the other hand, MoPU12 outputs point information for an object approaching the road (for example, an object such as a pedestrian that is about to run out onto the road). With this configuration, the information processing device 10 does not need to output point information for an object that has a low impact on the traveling of the vehicle 100.
- MoPU12 may determine whether or not to output point information for an object based on the moving direction of the object captured by the camera 30. For example, MoPU12 may not output point information for an object that
- MoPU12 calculates the risk level
- Central Brain15 may calculate the risk level.
- Central Brain15 calculates the risk level for the movement of vehicle 100 as a score for the external environment for vehicle 100 based on sensor information collected from multiple types of sensors and point information output from MoPU12. Then, Central Brain15 outputs an instruction to MoPU12 to change the frame rate of camera 30 according to the calculated risk level.
- the MoPU 12 outputs point information based on an image captured by the camera 30, but the disclosed technology is not limited to this embodiment.
- the MoPU 12 may output point information based on a radar signal and an infrared image instead of an image captured by the camera 30.
- the MoPU 12 can derive the x-coordinate value and the y-coordinate value of an object from the infrared image of the object captured by the infrared camera 34 in the same manner as the image captured by the camera 30.
- the radar 32 can acquire three-dimensional point cloud data of the object based on the radar signal. In other words, the radar 32 can detect the coordinate of the z-axis in the above three-dimensional orthogonal coordinate system.
- the MoPU 12 utilizes the principle of a stereo camera to combine the x- and y-coordinate values of the object photographed by the infrared camera 34 at the same timing as the radar 32 acquires the three-dimensional point cloud data of the object with the z-coordinate value of the object indicated by the three-dimensional point cloud data, and derives the coordinate values of the three coordinate axes (x-axis, y-axis, and z-axis) of the object as point information. The MoPU 12 then outputs the derived point information to the Central Brain 15.
- the MoPU 12 derives point information
- the disclosed technology is not limited to this aspect.
- the Central Brain 15 may derive point information instead of the MoPU 12.
- the Central Brain 15 derives point information by combining information detected by, for example, the camera 30L, the camera 30R, the radar 32, and the infrared camera 34.
- the Central Brain 15 derives coordinate values of the three coordinate axes (x-axis, y-axis, and z-axis) of the object as point information by performing triangulation based on the x-coordinate value and the y-coordinate value of the object photographed by the camera 30L and the x-coordinate value and the y-coordinate value of the object photographed by the camera 30R.
- the Central Brain 15 controls the automatic driving of the vehicle 100 based on the image and label information output from the IPU 11 and the point information output from the MoPU 12, but the disclosed technology is not limited to this aspect.
- the Central Brain 15 may control the operation of a robot based on the above information output from the IPU 11 and the MoPU 12.
- the robot may be a humanoid smart robot that performs work in place of a human.
- the Central Brain 15 controls the operation of the robot's arms, palms, fingers, feet, etc. based on the above information output from the IPU 11 and the MoPU 12, causing the robot to perform operations such as grasping, grabbing, holding, carrying, moving, carrying, throwing, kicking, and avoiding an object.
- the IPU 11 and MoPU 12 may be mounted at the positions of the robot's right and left eyes.
- the right eye may be equipped with an IPU 11 and MoPU 12 for the right eye
- the left eye may be equipped with an IPU 11 and MoPU 12 for the left eye.
- the information processing apparatus 10 according to the fourth embodiment has the same configuration as that of the second embodiment, as shown in FIG.
- the MoPU 12 outputs, as point information, the coordinate values of at least two diagonal vertices of a polygon that encloses the contour of an object recognized from an image captured by a separate camera.
- the coordinate values are the x and y coordinate values of the object in the above-mentioned three-dimensional orthogonal coordinate system.
- Figure 11 is an explanatory diagram showing an example of point information output by MoPU 12.
- bounding boxes 21, 22, 23, and 24 that surround the outline of each of four objects included in an image captured by a separate camera are shown.
- Figure 11 also shows an example in which MoPU 12 outputs, as point information, the coordinate values of two diagonal points of the vertices of the rectangular bounding boxes 21, 22, 23, and 24 that surround the outline of the object. In this way, MoPU 12 may capture objects not as points, but as objects of a certain size.
- the MoPU 12 may output, as point information, the coordinate values of multiple vertices of a polygon surrounding the contour of the object, instead of the coordinate values of two diagonal points of the vertices of the polygon surrounding the contour of the object recognized from an image captured by another camera.
- the MoPU 12 may output, as point information, the coordinate values of all four vertices of bounding boxes 21, 22, 23, and 24 that surround the contour of the object in a rectangular shape.
- the information processing apparatus 10 according to the fifth embodiment has the same configuration as that of the second embodiment, as shown in FIG.
- the vehicle 100 on which the information processing device 10 according to the fifth embodiment is mounted is equipped with sensors consisting of at least one of radar, LiDAR, high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance cameras, vision sensors, sound sensors, ultrasonic sensors, vibration sensors, infrared sensors, ultraviolet sensors, radio wave sensors, temperature sensors, and humidity sensors.
- sensors consisting of at least one of radar, LiDAR, high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance cameras, vision sensors, sound sensors, ultrasonic sensors, vibration sensors, infrared sensors, ultraviolet sensors, radio wave sensors, temperature sensors, and humidity sensors.
- Examples of sensor information that the information processing device 10 receives from the sensors include shifts in the center of gravity of the body weight, detection of the road material, detection of the outside air temperature, detection of the outside air humidity, detection of the up, down, side, and diagonal inclination angle of a slope, how the road is frozen, detection of the amount of moisture, the material of each tire, wear condition, detection of air pressure, road width, whether or not overtaking is prohibited, oncoming vehicles, vehicle type information of the front and rear vehicles, the cruising state of those vehicles, and surrounding conditions (birds, animals, soccer balls, wrecked vehicles, earthquakes, fires, wind, typhoons, heavy rain, light rain, blizzards, fog, etc.).
- the sensors are an example of a "detection unit,” and the sensor information is an example of "detection information.”
- the Central Brain 15 calculates control variables for controlling the autonomous driving of the vehicle 100 based on sensor information detected by the sensors.
- the Central Brain 15 acquires sensor information every billionth of a second.
- the Central Brain 15 calculates control variables for controlling the wheel speed, inclination, and suspension supporting the wheels of each of the four wheels of the vehicle 100.
- the inclination of the wheels includes both the inclination of the wheels relative to an axis horizontal to the road and the inclination of the wheels relative to an axis perpendicular to the road.
- the Central Brain 15 calculates a total of 16 control variables for controlling the wheel speed of each of the four wheels, the inclination of each of the four wheels relative to an axis horizontal to the road, the inclination of each of the four wheels relative to an axis perpendicular to the road, and the suspension supporting each of the four wheels.
- the Central Brain 15 controls the automatic driving of the vehicle 100 based on the control variables calculated above, the point information output from the MoPU 12, and the label information output from the IPU 11. Specifically, the Central Brain 15 controls the in-wheel motors mounted on each of the four wheels based on the above 16 control variables, thereby controlling the wheel speed, inclination, and suspension supporting each of the four wheels of the vehicle 100 to perform automatic driving. In addition, the Central Brain 15 recognizes the position and movement of recognized objects existing around the vehicle 100 based on the point information and label information, and controls the automatic driving of the vehicle 100 based on this recognized information, for example, to avoid collision with the object.
- the Central Brain 15 controls the automatic driving of the vehicle 100, so that, for example, when the vehicle 100 is traveling along a mountain road, it can perform optimal steering for that mountain road, and when the vehicle 100 is parked in a parking lot, it can drive at an optimal angle for that parking lot.
- the Central Brain 15 may be capable of inferring control variables from the above sensor information and information obtainable via a network from a server (not shown) or the like, using machine learning, more specifically, deep learning.
- the Central Brain 15 may be configured with AI.
- the Central Brain 15 can obtain control variables by performing multivariate analysis using the integral method as shown in the following formula (1) (see, for example, formula (2)) using the computational power of the above sensor information and long-tail incident AI data every billionth of a second (hereinafter also referred to as "Level 6 computational power") used to realize Level 6. More specifically, while obtaining the integral value of the delta values of various Ultra High Resolutions using the Level 6 computational power, each control variable is obtained at the edge level and in real time, and the result (i.e., each control variable) occurring in the next billionth of a second can be obtained with the highest probability theory.
- formula (1) see, for example, formula (2)
- Level 6 computational power the computational power of the above sensor information and long-tail incident AI data every billionth of a second
- an integral value obtained by time integrating delta values (e.g., small time change values) of a function that can identify each variable (e.g., the above sensor information and information obtainable via a network) such as air resistance, road resistance, road elements (e.g., debris), and slip coefficient is input to the deep learning model of Central Brain 15 (e.g., a trained model obtained by performing deep learning on a neural network).
- the deep learning model of Central Brain 15 outputs a control variable (e.g., a control variable with the highest confidence (i.e., evaluation value)) corresponding to the input integral value.
- the control variable is output in units of one billionth of a second.
- f(A) is a formula in which a function indicating the behavior of each variable, such as air resistance, road resistance, road elements (e.g., garbage), and slip coefficient, is simplified and expressed.
- formula (1) is a formula indicating the time integral v of "f(A)" from time a to time b.
- DL indicates deep learning (e.g., a deep learning model optimized by performing deep learning on a neural network)
- dA n /dt indicates the delta value of f(A, B, C, D, ..., N)
- A, B, C, D, ..., N indicate each variable, such as air resistance, road resistance, road elements (e.g., garbage), and slip coefficient
- f(A, B, C, D, ..., N) indicates a function indicating the behavior of A, B, C, D, ..., N
- V n indicates a value (control variable) output from a deep learning model optimized by performing deep learning on a neural network.
- an integral value obtained by time-integrating the delta value of a function is input to the deep learning model of Central Brain 15, this is merely one example.
- an integral value e.g., a result occurring in the next billionth of a second
- the delta value of a function indicating the behavior of each variable such as air resistance, road resistance, road elements, and slip coefficient
- the integral value with the highest confidence i.e., evaluation value
- an integral value is input to a deep learning model and an integral value is output from a deep learning model
- values corresponding to A, B, C, D, ..., N are used as example data
- a value corresponding to at least one control variable e.g., the result that will occur in the next billionth of a second
- teacher data is performed on a neural network, so that at least one control variable is inferred by an optimized deep learning model.
- the control variables obtained by the Central Brain 15 can be further refined by increasing the number of Deep Learning rounds. For example, more accurate control variables can be calculated using huge amounts of data such as tires, motor rotation, steering angle, road material, weather, garbage, effects of quadratic deceleration, slippage, loss of balance and how to control steering and speed to regain balance, as well as long-tail incident AI data.
- Fig. 12 is a third block diagram showing an example of the configuration of the information processing device 10. Note that Fig. 12 shows only a portion of the configuration of the information processing device 10.
- visible light images and infrared images of an object captured by the camera 30 are each input to the core 17 at a frame rate of 100 frames/second or more.
- the camera 30 is configured to include a visible light camera 30A capable of capturing visible light images of an object, and an infrared camera 30B capable of capturing infrared images of the object.
- the core 17 then outputs point information to the Central Brain 15 based on at least one of the input visible light images and infrared images.
- core 17 can identify an object from the visible light image of the object captured by visible light camera 30A, it outputs point information based on the visible light image.
- core 17 cannot capture an object from the visible light image due to a specified factor, it outputs point information based on the infrared image of the object captured by infrared camera 30B.
- core 17 detects the heat of the object using infrared camera 30B, and outputs point information of the object based on the infrared image that is the detection result.
- core 17 may output point information based on a visible light image and an infrared image.
- MoPU 12 also synchronizes the timing of capturing visible light images by visible light camera 30A with the timing of capturing infrared images by infrared camera 30B. Specifically, MoPU 12 outputs a control signal to camera 30 so that visible light images and infrared images are captured at the same timing. This synchronizes the number of images captured per second by visible light camera 30A and the number of images captured per second by infrared camera 30B (e.g., 1920 frames/second).
- Fig. 13 is a fourth block diagram showing an example of the configuration of the information processing device 10. Note that Fig. 13 shows only a part of the configuration of the information processing device 10.
- an image of an object captured by the camera 30 and a radar signal based on the electromagnetic waves reflected from the object by the radar 32 are input to the core 17 at a frame rate of 100 frames/second or more. Then, the core 17 outputs point information to the Central Brain 15 based on the input image of the object and the radar signal.
- the core 17 can derive the x-coordinate value and the y-coordinate value of the object from the input image of the object.
- the radar 32 can acquire three-dimensional point cloud data of the object based on the radar signal and detect the coordinate of the z-axis in the above-mentioned three-dimensional orthogonal coordinate system.
- the core 17 utilizes the principle of a stereo camera to combine the x-coordinate value and the y-coordinate value of the object captured by the camera 30 at the same timing as the radar 32 acquires the three-dimensional point cloud data of the object and the z-coordinate value of the object indicated by the three-dimensional point cloud data, and derive the coordinate values of the three coordinate axes (x-axis, y-axis, and z-axis) of the object as point information.
- the image of the object input to the core 17 may include at least one of a visible light image and an infrared image.
- MoPU 12 also synchronizes the timing at which images are captured by camera 30 with the timing at which radar 32 acquires 3D point cloud data of an object based on a radar signal. Specifically, MoPU 12 outputs control signals to camera 30 and radar 32 so that they capture images at the same time and acquire 3D point cloud data of an object. This synchronizes the number of images captured per second by camera 30 with the number of pieces of 3D point cloud data acquired per second by radar 32 (e.g., 1920 frames/second). In this way, the number of images captured per second by camera 30 and the number of pieces of 3D point cloud data acquired per second by radar 32 are greater than the frame rate of the ultra-high resolution camera equipped in IPU 11, i.e., the number of images captured per second by the ultra-high resolution camera.
- the information processing apparatus 10 according to the eighth embodiment has the same configuration as that of the second embodiment, as shown in FIG.
- the Central Brain 15 associates the point information output from the MoPU 12 with the label information at the same timing as when the IPU 11 outputs the label information. Furthermore, when new point information is output from the MoPU 12 after the point information and the label information are associated, the Central Brain 15 also associates the new point information with the label information.
- the new point information is point information of the same object as the object indicated by the point information associated with the label information, and is one or more pieces of point information between the time the association is performed and the time the next label information is output.
- the frame rate of the separate camera incorporating the MoPU 12 is 100 frames/second or more (for example, 1920 frames/second)
- the frame rate of the ultra-high resolution camera incorporating the IPU 11 is 10 frames/second.
- FIG. 14 is an explanatory diagram showing an example of the correspondence between point information and label information.
- the number of point information pieces output per second from MoPU 12 is referred to as the "output rate of point information”
- the number of label information pieces output per second from IPU 11 is referred to as the "output rate of label information.”
- FIG. 14 shows a time series of the output rate of point information P4 for object B14.
- the output rate of point information P4 for object B14 is 1920 frames/second. Furthermore, point information P4 moves from right to left in the figure.
- the output rate of label information for object B14 is 10 frames/second, which is lower than the output rate of point information P4.
- the timing when the next label information for object B14 is output from IPU11 is time t2. Therefore, at time t2, the Central Brain 15 derives label information "PERSON" for object B14 based on the label information output from IPU11. The Central Brain 15 then associates the label information "PERSON" derived at time t2 with the coordinate values (position information) of point information P4 output from MoPU12 at time t2.
- the Central Brain15 acquires point information P4 for object B14, but does not acquire label information.
- the Central Brain15 associates the point information P4 acquired during the period from time t1 to time t2 with the label information "PERSON" associated with the previous time t1 for the point information P4.
- the point information P4 acquired by the Central Brain15 during the period from time t1 to time t2 is an example of "new point information". In the example shown in FIG.
- the Central Brain 15 continuously outputs point information for that object at a high frame rate, so there is a low risk of losing track of the coordinate values (position information) of the object. Therefore, once the Central Brain 15 has associated point information with label information, it is possible to presumptively assign the most recent label information to point information acquired before acquiring the next label information.
- the ninth embodiment provides a vehicle 100 having a cooling function for the information processing device 10.
- FIG. 15 is an explanatory diagram showing the general configuration of a vehicle 100. As shown in FIG. 15, the vehicle 100 is equipped with an information processing device 10, a cooling execution device 110, and a cooling unit 120.
- the information processing device 10 is a device that controls the automatic driving of the vehicle 100, and has, as an example, the configuration shown in FIG. 9, which is similar to that of the second embodiment.
- the cooling execution device 110 acquires the object detection result by the information processing device 10, and causes the cooling unit 120 to cool the information processing device 10 based on the detection result.
- the cooling unit 120 cools the information processing device 10 using at least one cooling means, such as air cooling means, water cooling means, and liquid nitrogen cooling means.
- the object to be cooled in the information processing device 10 is described as the Central Brain 15 (more specifically, the CPU 14 constituting the Central Brain 15) that controls the automatic driving of the vehicle 100, but is not limited thereto.
- the information processing device 10 and the cooling execution device 110 are connected to each other so as to be able to communicate with each other via a network (not shown).
- the network may be any of a vehicle network, the Internet, a LAN (Local Area Network), and a mobile communication network.
- the mobile communication network may conform to any of the following communication methods: 5G (5th Generation) communication method, LTE (Long Term Evolution) communication method, 3G (3rd Generation) communication method, and 6G (6th Generation) communication method or later.
- FIG. 16 is a block diagram showing an example of the functional configuration of the cooling execution device 110. As shown in FIG. 16, the cooling execution device 110 has, as its functional configuration, an acquisition unit 112, an execution unit 114, and a prediction unit 116.
- the acquisition unit 112 acquires the object detection result by the information processing device 10. For example, as the detection result, the acquisition unit 112 acquires point information of the object output from the MoPU 12.
- the execution unit 114 executes cooling of the Central Brain 15 based on the object detection result acquired by the acquisition unit 112. For example, when the execution unit 114 recognizes that an object is moving based on the point information of the object output from the MoPU 12, it starts cooling of the Central Brain 15 by the cooling unit 120.
- the execution unit 114 is not limited to executing cooling of the Central Brain 15 based on the result of object detection, but may also execute cooling of the Central Brain 15 based on the result of prediction of the operating status of the information processing device 10.
- the prediction unit 116 predicts the operating status of the information processing device 10, specifically, the Central Brain 15, based on the object detection result acquired by the acquisition unit 112. For example, the prediction unit 116 acquires a learning model stored in a specified storage area. Then, the prediction unit 116 predicts the operating status of the Central Brain 15 by inputting the point information of the object output from the MoPU 12 acquired by the acquisition unit 112 into the learning model. Here, the learning model outputs the status and amount of change of the computing power of the Central Brain 15 as the operating status. In addition, the prediction unit 116 may predict and output the temperature change of the information processing device 10, specifically, the Central Brain 15, together with the operating status.
- the prediction unit 116 predicts the temperature change of the Central Brain 15 based on the number of point information of the object output from the MoPU 12 acquired by the acquisition unit 112. In this case, the prediction unit 116 predicts that the temperature change will be greater as the number of point information pieces increases, and predicts that the temperature change will be smaller as the number of point information pieces decreases.
- the execution unit 114 starts cooling the Central Brain 15 by the cooling unit 120 based on the prediction result of the operation status of the Central Brain 15 by the prediction unit 116. For example, the execution unit 114 starts cooling by the cooling unit 120 when the state and amount of change of the computing power of the Central Brain 15 predicted as the operation status exceeds a predetermined threshold. In addition, the execution unit 114 starts cooling by the cooling unit 120 when the temperature based on the temperature change of the Central Brain 15 predicted as the operation status exceeds a predetermined threshold.
- the execution unit 114 may also cause the cooling unit 120 to perform cooling using a cooling means according to the prediction result of the temperature change of the Central Brain 15 by the prediction unit 116. For example, the higher the predicted temperature of the Central Brain 15, the more cooling means the execution unit 114 may cause the cooling unit 120 to perform cooling. As a specific example, when it is predicted that the temperature of the Central Brain 15 will exceed a first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using one cooling means. On the other hand, when it is predicted that the temperature of the Central Brain 15 will exceed a second threshold that is higher than the first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using multiple cooling means.
- the execution unit 114 may use a more powerful cooling means to cool the Central Brain 15 as the predicted temperature of the Central Brain 15 increases. For example, when the execution unit 114 predicts that the temperature of the Central Brain 15 will exceed a first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using air cooling means. When the execution unit 114 predicts that the temperature of the Central Brain 15 will exceed a second threshold that is higher than the first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using water cooling means. When the execution unit 114 predicts that the temperature of the Central Brain 15 will exceed a third threshold that is higher than the second threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using liquid nitrogen cooling means.
- the execution unit 114 may determine the cooling means to be used for cooling based on the number of point information of the object output from the MoPU 12 acquired by the acquisition unit 112. In this case, the execution unit 114 may use a more powerful cooling means to cool the Central Brain 15 as the number of point information increases. For example, when the number of point information exceeds a first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using an air cooling means. Also, when the number of point information exceeds a second threshold higher than the first threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using a water cooling means. Furthermore, when the number of point information exceeds a third threshold higher than the second threshold, the execution unit 114 causes the cooling unit 120 to perform cooling using a liquid nitrogen cooling means.
- the Central Brain 15 may be activated when a moving object on the roadway is detected. For example, when a moving object on the roadway is detected while the vehicle 100 is driving autonomously, the Central Brain 15 may perform calculations to control the vehicle 100 with respect to the object. As described above, heat generation when the Central Brain 15, which controls the autonomous driving of the vehicle 100, performs advanced calculations is an issue. Therefore, the cooling execution device 110 according to the eighth embodiment predicts heat dissipation of the Central Brain 15 based on the object detection result by the information processing device 10, and performs cooling of the Central Brain 15 before or simultaneously with the start of heat dissipation. This prevents the Central Brain 15 from becoming too hot during the autonomous driving of the vehicle 100, enabling advanced calculations during the autonomous driving.
- the cooling execution device 110 executes cooling of the Central Brain 15 based on the object detection result by the information processing device 10 (MoPU 12).
- the cooling execution device 110 then stops cooling of the Central Brain 15 when a predetermined condition is met.
- the execution unit 114 of the cooling execution device 110 stops cooling of the Central Brain 15 by the cooling unit 120 when a predetermined condition is met. In this way, by stopping cooling of the Central Brain 15 when a predetermined condition is met, excessive cooling can be suppressed. This makes it possible to reduce power consumption of the vehicle 100.
- the predetermined conditions may include, for example, a condition that an object detected by the information processing device 10 is no longer detected, or a condition that an object detected by the information processing device 10 is moving outside the detection range. That is, the execution unit 114 may stop cooling the Central Brain 15 when the condition that an object detected by the information processing device 10 is no longer detected is satisfied, or may stop cooling the Central Brain 15 when the condition that an object detected by the information processing device 10 is moving outside the detection range is satisfied.
- the state in which an object detected by the information processing device 10 is no longer detected may include, for example, a state in which the detected object is located outside the detection range (outside the detection range) of the MoPU 12, that is, outside the shooting range of another camera, or a state in which the object is stationary.
- the state in which an object detected by the information processing device 10 is moving outside the detection range may include a state in which the detected object is moving (relatively moving) outside the detection range of the MoPU 12, that is, outside the shooting range of another camera.
- the condition that an object detected by the information processing device 10 is no longer detected is met, or when the condition that an object detected by the information processing device 10 is moving outside the detection range is met, cooling of the Central Brain 15 is stopped, thereby effectively suppressing excessive cooling of the Central Brain 15 (in other words, unnecessary cooling).
- the execution unit 114 may also determine whether to stop cooling the Central Brain 15 based on the relationship between a line of sight, which is predetermined in a separate camera incorporating the MoPU 12, and an object detected by the information processing device 10. For example, if an object detected by the information processing device 10 is moving toward the line of sight, the execution unit 114 may maintain cooling the Central Brain 15, and if the detected object is moving away from the line of sight, the execution unit 114 may stop cooling the Central Brain 15. If there is no change in the positional relationship between the object detected by the information processing device 10 and the line of sight, the execution unit 114 may determine whether to maintain or stop cooling the Central Brain 15, taking into account the movement of the detected object in the z-axis direction (the front-rear direction of the vehicle).
- the execution unit 114 may stop the cooling of the Central Brain 15 after a predetermined time has elapsed when a predetermined condition is satisfied. Specifically, the execution unit 114 may stop the cooling of the Central Brain 15 after a predetermined time has elapsed when a condition is satisfied that an object detected by the information processing device 10 is no longer detected, or when a condition is satisfied that an object detected by the information processing device 10 is heading outside the detection range.
- the predetermined time may be determined according to the type, size, acceleration, distance, etc. of the detected object.
- the Central Brain 15 can be cooled even during the period from when the predetermined condition is satisfied until the predetermined time has elapsed.
- the prediction unit 116 predicts the operating status of the Central Brain 15 by inputting the point information of the object output from the MoPU 12 acquired by the acquisition unit 112 into the learning model, and may further predict the time when the temperature of the Central Brain 15 will be below the threshold.
- the learning of the time when the temperature of the Central Brain 15 will be below the threshold may be performed by using data collected by the vehicle 100.
- a cloud server connected to the Central Brain 15 may collect data from the vehicle 100 and perform learning.
- teacher data is prepared that outputs the expected temperature of the Central Brain 15 in response to inputs such as the type, size, acceleration, and distance of an object detected by the MoPU 12, and the cloud server generates a learned cooling model based on the teacher data.
- the prediction unit 116 inputs the detection status of the Central Brain 15 into the cooling model and obtains the predicted temperature of the Central Brain 15, thereby making it possible to perform suitable cooling for the Central Brain 15.
- the cooling conditions of the Central Brain 15 may be output.
- learning the cooling model learning is performed using teacher data based on the cooling conditions.
- the subject that performs the learning is not limited to the cloud server, and may be another device.
- the execution unit 114 may stop cooling for the Central Brain 15 when a predetermined condition is satisfied, that is, the time when the temperature of the Central Brain 15 predicted by the prediction unit 116 becomes (reaches) a threshold value or less. In this way, when the condition that the time predicted by the prediction unit 116 is reached is met, cooling of the Central Brain 15 is stopped, thereby effectively preventing excessive cooling of the Central Brain 15.
- the cooling execution device 110 may execute cooling of the Central Brain 15 based on the prediction result of the operation status of the Central Brain 15 by the prediction unit 116, and may adjust the amount of cooling for the Central Brain 15 based on the prediction result of the temperature change of the Central Brain 15 by the prediction unit 116.
- the prediction of the temperature change by the prediction unit 116 can be obtained as a temperature curve of the Central Brain 15 by inputting the prediction result of the operation status of the Central Brain 15 to a trained model for temperature prediction.
- This trained model for temperature prediction is a model that has been machine-learned in advance with various actual operation statuses of the Central Brain 15 as input and actual temperatures corresponding to the operation statuses as output.
- Examples of the "various operation statuses" of the Central Brain 15 include the approach status of objects to the vehicle 100, the number of approaches, and the approach speed. That is, the execution unit 114 of the cooling execution device 110 increases the amount of cooling by the cooling means when the prediction result of the temperature change of the Central Brain 15 by the prediction unit 116 is a temperature rise, decreases the amount of cooling by the cooling means when the prediction result of the temperature change is a temperature drop, and maintains the amount of cooling by the cooling means when the prediction result of the temperature change is no temperature change.
- the execution unit 114 increases the amount of cooling to the Central Brain 15 when the state of the computing power of the Central Brain 15 predicted as the operating state and the amount of change exceed a predetermined threshold, and when the temperature rises or the temperature rise exceeds a predetermined threshold.
- the "predetermined threshold” may be set to a predetermined percentage of the peak value of the heat generation amount of the Central Brain 15, for example, a value of 80 percent.
- the "predetermined threshold” may be changed depending on the outside temperature, humidity, and the degree of running wind.
- the execution unit 114 uses an air cooling means
- the amount of cooling for the Central Brain 15 corresponds to the air volume.
- the amount of cooling for the Central Brain 15 corresponds to the temperature and circulation speed of the refrigerant.
- the execution unit 114 executes cooling of the Central Brain 15 based on the prediction result of the prediction unit 116 of the operating status of the Central Brain 15, and adjusts the amount of cooling for the Central Brain 15 based on the prediction result of the prediction unit 116 of the temperature change of the Central Brain 15, thereby making it possible to suppress overcooling and insufficient cooling of the Central Brain 15. In other words, the amount of cooling from the execution unit 114 can be brought closer to the appropriate amount.
- the cooling execution device 110 may further include a configuration in which the prediction unit 116 predicts a temperature peak in the Central Brain 15, and the execution unit 114 executes cooling of the Central Brain 15 with an amount of cooling corresponding to the temperature peak predicted by the prediction unit 116.
- the amount of cooling for the Central Brain 15 is set in advance according to the temperature peak. In this way, the execution unit 114 cools the Central Brain 15 with an amount of cooling corresponding to the temperature peak predicted by the prediction unit 116, and therefore overcooling and undercooling of the Central Brain 15 can be suppressed. In other words, the amount of cooling from the execution unit 114 can be brought closer to an appropriate amount.
- the cooling execution device 110 may also cause the execution unit 114 to execute cooling of the Central Brain 15 so that the amount of cooling corresponds to the temperature peak before the temperature peak predicted by the prediction unit 116 is reached.
- the trigger before the temperature peak is reached can be set to when the predicted temperature of the Central Brain 15 by the prediction unit 116 exceeds a predetermined set temperature, or when a predetermined time has passed since the temperature of the Central Brain 15 reaches its peak.
- the amount of cooling for the Central Brain 15 corresponds to the temperature peak at an early stage of the temperature rise of the Central Brain 15, so that the temperature rise of the Central Brain 15 can be efficiently suppressed. In other words, efficient cooling of the Central Brain 15 can be achieved.
- the prediction unit 116 of the cooling execution device 110 may predict that the temperature of the Central Brain 15 will rise if the object detected by the information processing device 10 (MoPU 12) is moving toward the vehicle 100 as a moving body.
- the execution unit 114 increases the amount of cooling for the Central Brain 15.
- an object detected by the information processing device 10 approaches the vehicle, further monitoring of the moving object becomes necessary, and the amount of calculation of the Central Brain 15 increases. Therefore, a temperature rise in the Central Brain 15 is expected. Therefore, by increasing the amount of cooling for the Central Brain 15 by the execution unit 114, insufficient cooling of the Central Brain 15 can be suppressed.
- the MoPU 12 included in the information processing device 10 according to the tenth embodiment derives a z coordinate value of an object as point information from an image of the object captured by the camera 30.
- each aspect of the information processing device 10 according to the ninth embodiment will be described in order.
- the information processing device 10 has the same configuration as that of the third embodiment, as shown in FIG. 10.
- the MoPU 12 derives the z coordinate value of an object as point information from images of the object captured by multiple cameras 30, specifically, cameras 30L and 30R.
- the MoPU 12 when one MoPU 12 is used, it is possible to derive the x and y coordinate values of an object as point information.
- the z coordinate value of an object is derive as point information based on images of the object captured by the camera 30L of MoPU 12L and the camera 30R of MoPU 12R, utilizing the principle of a stereo camera.
- the information processing device 10 according to the second aspect has the same configuration as that of the second embodiment, as shown in FIG. 17.
- the MoPU 12 derives the x, y, and z coordinate values of an object as point information from an image of the object captured by the camera 30 and a radar signal based on the electromagnetic waves irradiated by the radar 32 to the object and reflected from the object.
- the radar 32 is capable of acquiring three-dimensional point cloud data of the object based on the radar signal.
- the radar 32 is capable of detecting the z-axis coordinate in the above-mentioned three-dimensional orthogonal coordinate system.
- the MoPU 12 utilizes the principle of a stereo camera to combine the x and y coordinate values of the object captured by the camera 30 at the same timing as the radar 32 acquires the three-dimensional point cloud data of the object with the z-coordinate value of the object indicated by the three-dimensional point cloud data, and derives the coordinate values of the three coordinate axes of the object as point information.
- FIG. 17 is a fifth block diagram showing an example of the configuration of the information processing device 10. Note that FIG. 17 shows only a portion of the configuration of the information processing device 10.
- the MoPU 12 derives the z-coordinate value of the object as point information from the image of the object captured by the camera 30 and the result of capturing the structured light irradiated onto the object by the irradiation device 130.
- an image of an object captured by camera 30 and distortion information indicating distortion of the structured light pattern resulting from camera 140 capturing structured light irradiated onto an object by irradiation device 130 are each input to core 17 at a frame rate of 100 frames/second or more.
- Core 17 then outputs point information to Central Brain 15 based on the input image of the object and distortion information.
- one method for identifying the three-dimensional position or shape of an object is the structured light method.
- the structured light method irradiates an object with structured light patterned in dots, and obtains depth information from the distortion of the pattern.
- the structured light method is disclosed, for example, in the reference (http://ex-press.jp/wp-content/uploads/2018/10/018_teledyne_3rd.pdf).
- the illumination device 130 shown in FIG. 17 illuminates an object with structured light.
- the camera 140 captures the structured light illuminated onto the object by the illumination device 130.
- the camera 140 then outputs distortion information based on the distortion of the pattern of the captured structured light to the core 17.
- MoPU 12 synchronizes the timing of capturing images by camera 30 with the timing of capturing images of structured light by camera 140. Specifically, MoPU 12 outputs control signals to camera 30 and camera 140 so that they capture images at the same timing. This synchronizes the number of images captured per second by camera 30 and the number of images captured per second by camera 140 (e.g., 1920 frames/second). In this way, the number of images captured per second by camera 30 and the number of images captured per second by camera 140 are greater than the frame rate of the ultra-high resolution camera equipped in IPU 11, i.e., the number of images captured per second by the ultra-high resolution camera.
- the core 17 combines the x- and y-coordinate values of the object photographed by the camera 30 at the same time that the structured light is photographed by the camera 140 with distortion information based on the distortion of the pattern of the structured light to derive the z-coordinate value of the object as point information.
- FIG. 18 is a sixth block diagram showing an example of the configuration of the information processing device 10. Note that FIG. 18 shows only a portion of the configuration of the information processing device 10.
- the block diagram shown in FIG. 18 adds a Lidar sensor 18 to the configuration of the block diagram shown in FIG. 9.
- the Lidar sensor 18 is a sensor that acquires point cloud data including objects existing in three-dimensional space and the road surface on which the vehicle 100 is traveling.
- the information processing device 10 can derive position information in the depth direction of the object, i.e., the z coordinate value of the object. Note that it is assumed that the point cloud data acquired by the Lidar sensor 18 is acquired at longer intervals than the x and y coordinate values of the object output from the MoPU 12.
- the MoPU 12 also includes a camera 30, similar to the above aspect of the ninth embodiment.
- the MoPU 12 utilizes the principle of a stereo camera to combine the x and y coordinate values of an object photographed by the camera 30 at the same time that the Lidar sensor 18 acquires the point cloud data of the object with the z coordinate value of the object indicated by the point cloud data, and derives the coordinate values of the three coordinate axes of the object as point information.
- the MoPU12 derives the z coordinate value of the object at time t+1 as point information from the x, y, and z coordinate values of the object at time t and the x and y coordinate values of the object at the next point in time after time t (e.g., time t+1).
- Time t is an example of a "first point in time”
- time t+1 is an example of a "second point in time.”
- the z coordinate value of the object at time t+1 is derived using shape information, i.e., geometry. This will be described in detail below.
- FIG. 19 is a diagram showing a schematic of coordinate detection of an object over time.
- J indicates the position of an object represented by a rectangle, and the object's position moves over time from J1 to J2.
- the coordinate values of the object at time t when the object is located at J1 are (x1, y1, z1), and the coordinate values of the object at time t+1 when the object is located at J2 are (x2, y2, z2).
- the MoPU 12 derives the x-coordinate value and the y-coordinate value of the object from the image of the object captured by the camera 30.
- the MoPU 12 integrates the z-coordinate value of the object indicated by the point cloud data acquired from the Lidar sensor 18 with the above-mentioned x-coordinate value and y-coordinate value to derive the three-dimensional coordinate value (x1, y1, z1) of the object at time t.
- the MoPU 12 derives the z-coordinate value of the object at time t+1 based on the spatial geometry and the changes in the x-coordinate value and the y-coordinate value of the object from time t to time t+1.
- the spatial geometry includes the shape of the road surface obtained from the images captured by the super-high resolution camera equipped in the IPU 11 and the point cloud data of the Lidar sensor 18, and the shape of the vehicle 100.
- the geometry that indicates the shape of the road surface is generated in advance at time t.
- the MoPU 12 can simulate the vehicle 100 traveling on the road surface and estimate the amount of movement on each of the x-axis, y-axis, and z-axis.
- MoPU12 derives the x and y coordinate values of the object at time t+1 from the image of the object captured by camera 30.
- MoPU12 can derive the z coordinate value of the object at time t+1 by calculating, from a simulation, the amount of movement along the z axis when the object changes from its x and y coordinate values (x1, y1) at time t to its x and y coordinate values (x2, y2) at time t+1.
- MoPU12 then combines the x and y coordinate values with the z coordinate value to derive the three-dimensional coordinate value (x2, y2, z2) of the object at time t+1.
- the MoPU 12 may not be able to obtain the z-coordinate value of the object, which can be derived from the point cloud data of the Lidar sensor 18, as quickly as the x-coordinate value and the y-coordinate value of the object.
- the MoPU 12 derives the z-coordinate value of the object at time t+1 from the x-coordinate value, y-coordinate value, and z-coordinate value of the object at time t and the x-coordinate value and the y-coordinate value of the object at time t+1.
- the MoPU 12 can realize two-dimensional motion detection and three-dimensional motion detection by high-speed frame shots with high performance and low data volume.
- MoPU 12 derives the z coordinate value of an object as point information from an image of the object captured by camera 30, but the disclosed technology is not limited to this form.
- MoPU 12 may derive the z coordinate value of an object as point information.
- Central Brain 15 derives the z coordinate value of an object as point information by performing the processing performed by MoPU 12 in the above description on the image of the object captured by camera 30.
- Central Brain 15 derives the z coordinate value of an object as point information from images of the object captured by multiple cameras 30, specifically, camera 30L and camera 30R.
- the Central Brain 15 uses the principle of a stereo camera to derive the z coordinate value of the object as point information based on the images of the object captured by the camera 30L of the MoPU 12L and the camera 30R of the MoPU 12R.
- Fig. 20 is a seventh block diagram showing an example of the configuration of the information processing device 10. Note that Fig. 20 shows only a portion of the configuration of the information processing device 10.
- an image of an object captured by the event camera 30C (hereinafter, sometimes referred to as an "event image") is input to the core 17.
- the core 17 then outputs point information to the Central Brain 15 based on the input event image.
- the event camera is disclosed, for example, in the reference (https://dendenblog.xyz/event-based-camera/).
- FIG. 21 is an explanatory diagram for explaining an image of an object (event image) captured by event camera 30C.
- FIG. 21(A) is a diagram showing an object that is the subject of capture by event camera 30C.
- FIG. 21(B) is a diagram showing an example of an event image.
- FIG. 21(C) is a diagram showing an example in which the center of gravity of the difference between an image captured at the current time and an image captured at the previous time, which is represented by the event image, is calculated as point information.
- the difference between the image captured at the current time and the image captured at the previous time is extracted as points. For this reason, when event camera 30C is used, for example, as shown in FIG. 21(B), points of each moving part of the person area shown in FIG. 21(A) are extracted.
- the core 17 extracts the person object, and then extracts the coordinates of the feature point (e.g., only one point) that represents the person area. This makes it possible to reduce the amount of data transferred to the Central Brain 15 and memory 16. Since the event image allows the person object to be extracted at any frame rate, in the case of the event camera 30C, extraction can be performed at a frame rate equal to or higher than the maximum frame rate (e.g., 1920 frames/sec) of the camera 30 mounted on the MoPU 12 in the above embodiment, and point information of the object can be captured with high accuracy.
- the maximum frame rate e.g., 1920 frames/sec
- the MoPU 12 may include a visible light camera 30A in addition to the event camera 30C, as in the above embodiment.
- the visible light image and the event image of the object captured by the visible light camera 30A are input to the core 17.
- the core 17 then outputs point information to the Central Brain 15 based on at least one of the input visible light image and the event image.
- the core 17 when the core 17 can identify an object from a visible light image of the object captured by the visible light camera 30A, the core 17 outputs point information based on the visible light image.
- the core 17 when the core 17 cannot capture an object from the visible light image due to a predetermined factor, the core 17 outputs point information based on the event image.
- the predetermined factor includes at least one of the following: when the moving speed of the object is equal to or greater than a predetermined value, and when the change in the amount of ambient light per unit time is equal to or greater than a predetermined value.
- the core 17 identifies the object based on the event image and outputs the x-coordinate value and the y-coordinate value of the object as point information. Also, when the object cannot be captured from the visible light image due to a sudden change in the amount of ambient light, such as backlighting, the core 17 identifies the object based on the event image and outputs the x-coordinate value and the y-coordinate value of the object as point information.
- the information processing device 10 can use different cameras 30 to capture an object depending on the predetermined factor.
- FIG. 22 shows an example of a hardware configuration of a computer 1200 functioning as the management server 101, the SoCBox 400, the cooling execution device 500, the information processing device 10, or the cooling execution device 110.
- a program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to this embodiment, or to execute operations or one or more "parts” associated with the device according to this embodiment, and/or to execute a process or steps of the process according to this embodiment.
- Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.
- the computer 1200 includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210.
- the computer 1200 also includes input/output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input/output controller 1220.
- the DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc.
- the storage device 1224 may be a hard disk drive, a solid state drive, etc.
- the computer 1200 also includes a ROM 1230 and a legacy input/output unit such as a keyboard, which are connected to the input/output controller 1220 via an input/output chip 1240.
- the CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit.
- the graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.
- the communication interface 1222 communicates with other electronic devices via a network.
- the storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200.
- the DVD drive reads programs or data from a DVD-ROM or the like, and provides the programs or data to the storage device 1224.
- the IC card drive reads programs and data from an IC card and/or writes programs and data to an IC card.
- ROM 1230 stores therein a boot program or the like executed by computer 1200 upon activation, and/or a program that depends on the hardware of computer 1200.
- I/O chip 1240 may also connect various I/O units to I/O controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.
- the programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card.
- the programs are read from the computer-readable storage medium, installed in storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by CPU 1212.
- the information processing described in these programs is read by computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above.
- An apparatus or method may be constructed by realizing the operation or processing of information according to the use of computer 1200.
- CPU 1212 may execute a communication program loaded into RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program.
- communication interface 1222 reads transmission data stored in a transmission buffer area provided in RAM 1214, storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.
- the CPU 1212 may also cause all or a necessary portion of a file or database stored in an external recording medium such as a storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.
- an external recording medium such as a storage device 1224, a DVD drive (DVD-ROM), an IC card, etc.
- the CPU 1212 may then write back the processed data to the external recording medium.
- CPU 1212 may perform various types of processing on data read from RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search/replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to RAM 1214.
- CPU 1212 may also search for information in a file, database, etc. in the recording medium.
- CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.
- the above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200.
- a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.
- the blocks in the flowcharts and block diagrams in each of the above embodiments may represent stages of a process in which an operation is performed or "parts" of a device responsible for performing the operation. Particular stages and “parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and/or a processor provided with computer-readable instructions stored on a computer-readable storage medium.
- Dedicated circuitry may include digital and/or analog hardware circuitry, and may include integrated circuits (ICs) and/or discrete circuits.
- Programmable circuitry may include reconfigurable hardware circuitry including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.
- FPGAs field programmable gate arrays
- PLAs programmable logic arrays
- a computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create a means for performing the operations specified in the flowchart or block diagram.
- Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like.
- Computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.
- RAMs random access memories
- ROMs read-only memories
- EPROMs or flash memories erasable programmable read-only memories
- EEPROMs electrically erasable programmable read-only memories
- SRAMs static random access memories
- CD-ROMs compact disk read-only memories
- DVDs digital versatile disks
- Blu-ray disks memory sticks, integrated circuit cards, and the like.
- the computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.
- ISA instruction set architecture
- machine instructions machine-dependent instructions
- microcode firmware instructions
- state setting data or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.
- the computer-readable instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN) such as the Internet, so that the processor of the general-purpose computer, special-purpose computer, or other programmable data processing apparatus, or to a programmable circuit, executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams.
- processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.
- each processor e.g., IPU11, MoPU12, and Central Brain15
- the processor that executes each process is not limited.
- the processes executed by MoPU12 in the above embodiment may be executed by Central Brain15 instead of MoPU12, or may be executed by a processor other than IPU11, MoPU12, and Central Brain15.
- ⁇ Appendix 1> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; Equipped with Information processing device.
- the frame rate of the first camera is variable;
- the first processor changes the frame rate of the first camera in response to a predetermined factor.
- the first processor calculates a score related to an external environment for a predetermined object.
- An information processing device according to (3).
- the first processor changes a frame rate of the first camera in response to the calculated score regarding the external environment.
- ⁇ Appendix 2> (1) a first processor that outputs, from an image of an object photographed by a first camera, coordinate values of at least two coordinate axes in a three-dimensional orthogonal coordinate system of a point indicating a position of the photographed object; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the coordinate values output from the first processor with the identification information output from the second processor; Equipped with Information processing device.
- the first processor outputs the coordinate values of at least two diagonal vertices of a polygon surrounding the contour of the object recognized from the image captured by the first camera;
- An information processing device as described in (1).
- the first processor outputs the coordinate values of a plurality of vertices of a polygon surrounding a contour of the object recognized from the image captured by the first camera.
- An information processing device as described in (2).
- ⁇ Appendix 3> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor and controls automatic driving of a moving body based on the point information and the identification information; Equipped with Information processing device.
- the third processor Calculating a control variable for controlling the autonomous driving of the moving object based on the detection information detected by the detection unit; controlling the automatic driving of the moving body based on the calculated control variable, the point information, and the identification information; An information processing device as described in (1).
- ⁇ Appendix 4> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with The frame rate of the first camera is greater than the frame rate of the second camera.
- Information processing device
- the frame rate of the first camera is 10 times or more the frame rate of the second camera.
- the frame rate of the first camera is 100 frames/second or more, and the frame rate of the second camera is 10 frames/second.
- ⁇ Appendix 5> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with The first processor calculates a degree of danger related to movement of a predetermined moving object as a score related to an external environment for the moving object based on the detection information detected by the detection unit and the point information.
- Information processing device
- the frame rate of the first camera is variable;
- the first processor changes a frame rate of the first camera in accordance with the calculated degree of risk.
- the degree of danger indicates the degree of danger through which the moving object will travel in the future.
- a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with The third processor calculates a degree of danger related to movement of a predetermined moving object as a score related to an external environment for the moving object based on the detection information detected by the detection unit and the point information.
- Information processing device that outputs point information that captures an object as a point from an image of the object photographed by a first camera
- a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera
- a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with The third processor calculate
- the frame rate of the first camera is variable;
- the third processor outputs to the first processor an instruction to change a frame rate of the first camera in accordance with the calculated degree of risk.
- ⁇ Appendix 6> (1) a first processor that outputs point information that captures the photographed object as a point based on at least one of a visible light image and an infrared image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with Information processing device.
- the first processor when the first processor cannot capture the object from a visible light image of the object captured by a visible light camera included in the first camera due to a predetermined factor, the first processor outputs the point information based on an infrared image of the object captured by an infrared camera included in the first camera.
- An information processing device as described in (1).
- the first processor synchronizes a timing of capturing the visible light image by the visible light camera and a timing of capturing the infrared image by the infrared camera.
- An information processing device as described in (2).
- ⁇ Appendix 7> (1) a first processor that outputs point information that captures the photographed object as a point based on an image of the object photographed by the first camera and a radar signal based on a reflected wave from the object of an electromagnetic wave irradiated to the object by a radar; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with Information processing device.
- the first processor synchronizes a timing of capturing the image by the first camera with a timing of acquiring three-dimensional point cloud data of the object based on the radar signal by the radar.
- An information processing device as described in (1).
- ⁇ Appendix 8> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs label information indicating a type of the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the label information output from the second processor; Equipped with Information processing device.
- the third processor associates position information of the object indicated by the point information with the label information about the object that exists at the position indicated by the position information.
- An information processing device as described in (1).
- the third processor associates the point information output from the first processor with the label information at the same timing as the timing at which the second processor outputs the label information; An information processing device as described in (2).
- ⁇ Appendix 9> (1) an acquisition unit that outputs point information that captures an object as a point and identification information that identifies the object from images of the object captured by a plurality of cameras facing corresponding directions, and acquires a detection result of the object by an information processing device that associates the point information with the identification information; an execution unit that executes cooling of the information processing device based on the detection result acquired by the acquisition unit; Equipped with Cooling execution device.
- (2) a prediction unit that predicts an operating status of the information processing device based on the detection result acquired by the acquisition unit; the execution unit executes cooling of the information processing device based on a result of the prediction by the prediction unit of the operating status of the information processing device.
- a cooling execution device as described in (1).
- the prediction unit predicts a temperature change of the information processing device, the execution unit causes cooling of the information processing device to be performed using a cooling unit corresponding to a result of the prediction of the temperature change of the information processing device by the prediction unit; A cooling execution device as described in (2).
- the detection result acquired by the acquisition unit is the point information.
- a cooling device according to any one of (1) to (3).
- ⁇ Appendix 10> (1) a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with the first processor derives, from the image of the object captured by the first camera, a coordinate value in a depth direction of the object in a three-dimensional orthogonal coordinate system of a point indicating a position of the object as the point information; Information processing device.
- the first processor derives coordinate values in the depth direction as the point information from images of the object captured by a plurality of the first cameras.
- An information processing device as described in (1).
- the first processor derives coordinate values of the object in a width direction, a height direction, and a depth direction as the point information from an image of the object captured by the first camera and a radar signal based on a reflected wave from the object of an electromagnetic wave irradiated to the object by a radar.
- An information processing device according to (1) or (2).
- the first processor derives coordinate values in a width direction, a height direction, and a depth direction of the object as the point information from a result of photographing an image of the object photographed by the first camera and structured light irradiated on the object by an irradiation device.
- An information processing device according to any one of (1) to (3).
- the first processor derives, as the point information, a coordinate value in the depth direction at a second time point from coordinate values in a width direction, a height direction, and the depth direction of the object in the three-dimensional orthogonal coordinate system at a first time point and coordinate values in the width direction and the height direction at a second time point that is a time point subsequent to the first time point;
- An information processing device according to any one of (1) to (4).
- a first processor that outputs point information that captures an object as a point from an image of the object photographed by a first camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the first camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with the third processor derives, from the image of the object captured by the first camera, a coordinate value in a depth direction of the object in a three-dimensional orthogonal coordinate system of a point indicating a position of the object as the point information; Information processing device.
- ⁇ Appendix 11> (1) a first processor that outputs point information that captures an object as a point from an image of the object captured by an event camera; a second processor that outputs identification information that identifies the photographed object from an image of the object photographed by a second camera facing in a direction corresponding to the event camera; a third processor that associates the point information output from the first processor with the identification information output from the second processor; Equipped with Information processing device.
- the first processor when the first processor cannot capture the object from the visible light image of the object captured by the visible light camera due to a predetermined factor, the first processor outputs the point information based on the image of the object captured by the event camera.
- An information processing device as described in (1).
- the predetermined factor includes at least one of a case where a moving speed of the object is equal to or greater than a predetermined value and a case where a change in the amount of ambient light per unit time is equal to or greater than a predetermined value.
- the event camera is a camera that outputs an event image that represents a difference between an image captured at a current time and an image captured at a previous time.
- An information processing device according to any one of (1) to (3).
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Thermal Sciences (AREA)
- Traffic Control Systems (AREA)
- Condensed Matter Physics & Semiconductors (AREA)
- Computer Hardware Design (AREA)
- Power Engineering (AREA)
Abstract
Description
自動運転向けのSoC(System on Chip)が高度な演算処理を行う際に、発熱が課題となる。そこで、本実施形態では、SoCBoxの処理の作動を検知して、SoCBoxの冷却を実行する冷却実行装置を提供する。
することで、SoC Boxが高温になるのを防ぎ、車両での高度な演算を可能にする。
化量が予め定められた閾値を超えた場合、SoCBox400の冷却を開始する。また、冷却実行部504は、AIによって予測されたSoCBox400の作動状況、及びSoCBox400における部位の温度変化に応じて、SoCBox400の冷却を開始してもよい。例えば、冷却実行部504は、作動状況として予測されたSoCBox400のパワーコンピューティングパワーの状況、及び変化量が予め定められた閾値を超えた場合、かつ温度変化が予め定められた閾値を超えた場合、SoCBox400における該当する部位の冷却を開始する。
上記実施の形態の車両300の変形例を以下に説明する。
冷却実行部504は、MoPU217による検出結果が、動く物体を検出したことを示す場合、冷却部600を用いて、SoCBox400の冷却を実行すると共に、車両300を減速させる。そして、以下に説明するように、車両300が減速する際に生成される回生電力(回生エネルギー)を用いて冷却実行装置500、及び冷却部600等が駆動される。
なお、回生がかかるまでは、バッテリー706の電力を用いてSoCBox400の冷却を行い、回生がかかれば、回生電力を用いてSoCBox400の冷却を行うようにパワーコントロールユニット708を動作させることができる。
次に、本実施形態に係る第2の実施形態について説明する。本開示に係る情報処理装置は、一例として、少なくとも一部が車両100に搭載されて、車両100の自動運転制御を行う。また、当該情報処理装置は、Autonomous DrivingをLevel6によるAI/多変量解析/ゴールシーク/戦略立案/最適確率解/最適スピード解/最適コースマネジメント/エッジにおける多種センサ入力により得られたデータを基にリアルタイムで実現でき、デルタ最適解に基づいて調整される走行システムを提供し得る。車両100は「対象物」の一例である。
ここで、超高解像度カメラによって撮影された画像から、当該画像に含まれる物体が何であるかを認識することは可能である。しかしながら、Level6時代の自動運転ではこれだけでは不十分である。Level6時代では、物体の動きをより高精度に認識することも必要である。MoPU12によって物体の動きをより高精度に認識することで、例えば、自動運転によって走行する車両100が障害物を回避する回避動作を、より高精度で行うことが可能となる。しかしながら、超高解像度カメラでは、1秒間に10フレーム程度しか画像を取得することができず、物体の動きを解析する精度はMoPU12を搭載したカメラに比べると低い。一方、MoPU12を搭載したカメラでは、例えば100フレーム/秒の高フレームレートでの出力が可能である。
次に、本実施形態に係る第2の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
次に、本実施形態に係る第3の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
一例として、第4の実施形態に係る情報処理装置10は、第2の実施形態と同様の図9に示す構成を備えている。
として出力してもよい。
次に、本実施形態に係る第5の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
一例として、第5の実施形態に係る情報処理装置10は、第2の実施形態と同様の図9に示す構成を備えている。
次に、本実施形態に係る第5の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
図12は、情報処理装置10の構成の一例を示す第3のブロック図である。なお、図12は、情報処理装置10の一部の構成のみを示している。
次に、本実施形態に係る第7の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
図13は、情報処理装置10の構成の一例を示す第4のブロック図である。なお、図13は、情報処理装置10の一部の構成のみを示している。
次に、本実施形態に係る第8の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
一例として、第8の実施形態に係る情報処理装置10は、第2の実施形態と同様の図9に示す構成を備えている。
次に、本実施形態に係る第9の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
車両100の自動運転を制御する情報処理装置10が高度な演算処理を行う際に、発熱が課題となる。そこで、第9の実施形態は、情報処理装置10に対する冷却機能を有する車両100を提供する。
また、冷却実行装置110は、予測部116によるCentral Brain15の作動状況の予測結果に基づいて、Central Brain15に対する冷却を実行させるとともに、予測部116によるCentral Brain15の温度変化の予測結果に基づいて、Central Brain15に対する冷却量を調整させてもよい。予測部116による温度変化の予測は、温度予測用の学習済モデルに対して、Central Brain15の作動状況の予測結果を入力することで、Central Brain15の温度カーブとして得ることができる。この温度予測用の学習済モデルは、Central Brain15における実際の種々の作動状況を入力とし、当該作動状況に対する実際の温度を出力として予め機械学習させたモデルである。Central Brain15における「種々の作動状況」としては、車両100に対する物体の接近状況、接近数、接近速度等がある。すなわち、冷却実行装置110の実行部114は、予測部116によるCentral Brain15の温度変化の予測結果が温度上昇の場合に冷却手段による冷却量を増やし、当該温度変化の予測結果が温度低下の場合に冷却手段による冷却量を減らし、当該温度変化の予測結果が温度変化なしの場合に冷却手段による冷却量を維持する。例えば、実行部114は、作動状況として予測されたCentral Brain15のパワーコンピューティングパワーの状況、及び変化量が予め定められた閾値を超える場合、かつ温度上昇する、あるいは温度上昇が予め定められた閾値を超える場合、Central Brain15への冷却量を増やす。ここで、「予め定められた閾値」には、Central Brain15の発熱量のピーク値に対する所定の割合、例えば、80パーセントの値に設定してもよい。また、「予め定められた閾値」は、外気温、湿度、走行風の程度によって変動させてもよい。
次に、本実施形態に係る第10の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
第10の実施形態に係る情報処理装置10が備えるMoPU12は、カメラ30により撮影された物体の画像から、点情報として物体のz座標値を導出する。以下、第9の実施形態に係る情報処理装置10の各態様について順に説明する。
MoPU12は、カメラ30により撮影された物体の画像から、当該物体のx座標値及びy座標値を導出する。続いて、MoPU12は、Lidarセンサ18から取得した点群データが示す当該物体のz座標値と上記x座標値及びy座標値とを統合して、時刻tにおける当該物体の3次元座標値(x1、y1、z1)を導出する。
MoPU12は、空間のジオメトリと、時刻tから時刻t+1への物体のx座標値及びy座標値の変化とに基づいて、時刻t+1における当該物体のz座標値を導出する。空間のジオメトリには、IPU11が備える超高解像度カメラにより撮影された画像及びLidarセンサ18の点群データから得られる路面の形状と、車両100の形状とが含まれる。
次に、本実施形態に係る第11の実施形態について、上記実施形態との重複部分を省略又は簡略しつつ説明する。
図20は、情報処理装置10の構成の一例を示す第7のブロック図である。なお、図20は、情報処理装置10の一部の構成のみを示している。
記憶装置1224は、コンピュータ1200内のCPU1212によって使用されるプログラム及びデータを格納する。DVDドライブは、プログラム又はデータをDVD-ROM等から読み取り、記憶装置1224に提供する。ICカードドライブは、プログラム及びデータをICカードから読み取り、及び/又はプログラム及びデータをICカードに書き込む。
信ネットワーク又はインターネットに接続されたサーバシステム内に提供されるハードディスク又はRAMのような記録媒体が、コンピュータ可読記憶媒体として使用可能であり、それによりプログラムを、ネットワークを介してコンピュータ1200に提供する。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
を備える、
情報処理装置。
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサを備える、
(1)に記載の情報処理装置。
前記第1カメラのフレームレートは可変であり、
前記第1プロセッサは、所定要因に応じて前記第1カメラのフレームレートを変更する、
(1)又は(2)に記載の情報処理装置。
前記第1プロセッサは、所定の対象物に対する外部環境に関するスコアを算出する、
(3)に記載の情報処理装置。
前記第1プロセッサは、算出した前記外部環境に関するスコアに応じて、前記第1カメラのフレームレートを変更する、
(4)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体の存在位置を示す点の3次元直交座標系における少なくとも2つの座標軸の座標値を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記座標値及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備える、
情報処理装置。
前記第1プロセッサは、前記第1カメラにより撮影された画像から認識した前記物体の輪郭を囲む多角形の頂点の少なくとも対角となる2点の前記座標値を出力する、
(1)に記載の情報処理装置。
前記第1プロセッサは、前記第1カメラにより撮影された画像から認識した前記物体の輪郭を囲む多角形の複数の頂点の前記座標値を出力する、
(2)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体の存在位置を示す点の3次元直交座標系における少なくとも2つの座標軸の座標値を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記座標値及び前記識別情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体の存在位置を示す点の3次元直交座標系における少なくとも2つの座標軸の座標値を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記座標値及び前記識別情報を対応付ける、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付け、前記点情報及び前記識別情報に基づいて、移動体の自動運転を制御する第3プロセッサと、
を備える、
情報処理装置。
前記第3プロセッサは、
検知部が検知した検知情報に基づいて、前記移動体の自動運転を制御するための制御変数を算出し、
算出した前記制御変数と、前記点情報及び前記識別情報とに基づいて、前記移動体の自動運転を制御する、
(1)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、前記点情報及び前記識別情報に基づいて、移動体の自動運転を制御する、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、前記点情報及び前記識別情報に基づいて、移動体の自動運転を制御する、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備え、
前記第1カメラのフレームレートは、前記第2カメラのフレームレートより大きい、
情報処理装置。
前記第1カメラのフレームレートは、前記第2カメラのフレームレートの10倍以上である、
(1)に記載の情報処理装置。
前記第1カメラのフレームレートは100フレーム/秒以上であり、前記第2カメラのフレームレートは10フレーム/秒である、
(2)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラよりフレームレートが小さく、かつ前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラよりフレームレートが小さく、かつ前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備え、
前記第1プロセッサは、検知部が検知した検知情報及び前記点情報に基づいて、所定の移動体に対する外部環境に関するスコアとして、前記移動体の移動に関する危険度を算出する、
情報処理装置。
前記第1カメラのフレームレートは可変であり、
前記第1プロセッサは、算出した前記危険度に応じて前記第1カメラのフレームレートを変更する、
(1)に記載の情報処理装置。
前記危険度は、前記移動体が今後どの程度の危険な場所を走行するかの度合を示す、
(1)又は(2)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備え、
前記第3プロセッサは、検知部が検知した検知情報及び前記点情報に基づいて、所定の移動体に対する外部環境に関するスコアとして、前記移動体の移動に関する危険度を算出する、
情報処理装置。
前記第1カメラのフレームレートは可変であり、
前記第3プロセッサは、算出した前記危険度に応じて前記第1カメラのフレームレートを変更する指示を前記第1プロセッサに出力する、
(4)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、
検知部が検知した検知情報及び前記点情報に基づいて、所定の移動体に対する外部環境に関するスコアとして、前記移動体の移動に関する危険度を算出する、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、
検知部が検知した検知情報及び前記点情報に基づいて、所定の移動体に対する外部環境に関するスコアとして、前記移動体の移動に関する危険度を算出する、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の可視光画像及び赤外線画像の少なくとも一方に基づいて、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備える、
情報処理装置。
前記第1プロセッサは、所定要因により前記第1カメラに含まれる可視光カメラにより撮影された前記物体の可視光画像から前記物体を捉えられない場合、前記第1カメラに含まれる赤外線カメラにより撮影された前記物体の赤外線画像に基づいて前記点情報を出力する、
(1)に記載の情報処理装置。
前記第1プロセッサは、前記可視光カメラにより前記可視光画像を撮影するタイミングと、前記赤外線カメラにより前記赤外線画像を撮影するタイミングとを同期させる、
(2)に記載の情報処理装置。
第1カメラにより撮影された物体の可視光画像及び赤外線画像の少なくとも一方に基づいて、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の可視光画像及び赤外線画像の少なくとも一方に基づいて、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像及びレーダーにより前記物体に照射された電磁波の前記物体からの反射波に基づくレーダー信号から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備える、
情報処理装置。
前記第1プロセッサは、前記第1カメラにより前記画像を撮影するタイミングと、前記レーダーが前記レーダー信号に基づく前記物体の3次元点群データを取得するタイミングとを同期させる、
(1)に記載の情報処理装置。
前記第1カメラによって撮影される単位時間あたりの画像数及び前記レーダーによって取得される単位時間あたりの3次元点群データの数は、前記第2カメラによって撮影される単位時間あたりの画像数より多い、
(1)又は(2)に記載の情報処理装置。
第1カメラにより撮影された物体の画像及びレーダーにより前記物体に照射された電磁波の前記物体からの反射波に基づくレーダー信号から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像及びレーダーにより前記物体に照射された電磁波の前記物体からの反射波に基づくレーダー信号から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理を実行させるための情報処理プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体の種別を示すラベル情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記ラベル情報を対応付ける第3プロセッサと、
を備える、
情報処理装置。
前記第3プロセッサは、前記点情報が示す前記物体の位置情報と、前記位置情報が示す位置に存在する前記物体についての前記ラベル情報とを対応付ける、
(1)に記載の情報処理装置。
前記第3プロセッサは、前記第2プロセッサが前記ラベル情報を出力したタイミングと同じタイミングで前記第1プロセッサから出力された前記点情報を前記ラベル情報に対応付ける、
(2)に記載の情報処理装置。
前記第3プロセッサは、前記点情報及び前記ラベル情報を対応付けた後に前記第1プロセッサから新たな前記点情報が出力された場合、新たな前記点情報についても前記ラベル情報と対応付ける、
(2)又は(3)に記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体の種別を示すラベル情報を出力し、
前記点情報及び前記ラベル情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体の種別を示すラベル情報を出力し、
前記点情報及び前記ラベル情報を対応付ける、
処理を実行させるための情報処理プログラム。
(1)
対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得する取得部と、
前記取得部が取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる実行部と、
を備える、
冷却実行装置。
前記取得部が取得した前記検出結果に基づいて、前記情報処理装置の作動状況を予測する予測部を備え、
前記実行部は、前記予測部による前記情報処理装置の作動状況の予測結果に基づいて、前記情報処理装置に対する冷却を実行させる、
(1)に記載の冷却実行装置。
前記予測部は、前記情報処理装置の温度変化を予測し、
前記実行部は、前記予測部による前記情報処理装置の温度変化の予測結果に応じた冷却手段を用いて、前記情報処理装置に対する冷却を実行させる、
(2)に記載の冷却実行装置。
前記取得部が取得する前記検出結果は、前記点情報である、
(1)から(3)の何れか1つに記載の冷却実行装置。
対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得し、
取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる、
処理をコンピュータが実行する冷却実行方法。
コンピュータに、
対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得し、
取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる、
処理を実行させるための冷却実行プログラム。
(1)
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備え、
前記第1プロセッサは、前記第1カメラにより撮影された前記物体の画像から、前記点情報として前記物体の存在位置を示す点の3次元直交座標系における前記物体の奥行き方向の座標値を導出する、
情報処理装置。
前記第1プロセッサは、複数の前記第1カメラにより撮影された前記物体の画像から、前記点情報として前記奥行き方向の座標値を導出する、
(1)に記載の情報処理装置。
前記第1プロセッサは、前記第1カメラにより撮影された前記物体の画像及びレーダーにより前記物体に照射された電磁波の前記物体からの反射波に基づくレーダー信号から、前記点情報として前記物体の幅方向、高さ方向、及び前記奥行き方向の座標値を導出する、
(1)又は(2)に記載の情報処理装置。
前記第1プロセッサは、前記第1カメラにより撮影された前記物体の画像及び照射装置により前記物体に照射されたストラクチャードライトを撮影した結果から、前記点情報として前記物体の幅方向、高さ方向、及び前記奥行き方向の座標値を導出する、
(1)から(3)の何れか1つに記載の情報処理装置。
前記第1プロセッサは、第1時点の前記3次元直交座標系における前記物体の幅方向、高さ方向、及び前記奥行き方向の座標値と、前記第1時点の次の時点である第2時点における前記幅方向及び前記高さ方向の座標値とから、前記点情報として前記第2時点におけ
る前記奥行き方向の座標値を導出する、
(1)から(4)の何れか1つに記載の情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備え、
前記第3プロセッサは、前記第1カメラにより撮影された前記物体の画像から、前記点情報として前記物体の存在位置を示す点の3次元直交座標系における前記物体の奥行き方向の座標値を導出する、
情報処理装置。
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、
前記第1カメラにより撮影された前記物体の画像から、前記点情報として前記物体の存在位置を示す点の3次元直交座標系における前記物体の奥行き方向の座標値を導出する、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
第1カメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記第1カメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付け、
前記第1カメラにより撮影された前記物体の画像から、前記点情報として前記物体の存在位置を示す点の3次元直交座標系における前記物体の奥行き方向の座標値を導出する、
処理を実行させるための情報処理プログラム。
(1)
イベントカメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力する第1プロセッサと、
前記イベントカメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力する第2プロセッサと、
前記第1プロセッサから出力された前記点情報及び前記第2プロセッサから出力された前記識別情報を対応付ける第3プロセッサと、
を備える、
情報処理装置。
前記第1プロセッサは、所定要因により可視光カメラにより撮影された前記物体の可視光画像から前記物体を捉えられない場合、前記イベントカメラにより撮影された前記物体の画像に基づいて前記点情報を出力する、
(1)に記載の情報処理装置。
前記所定要因は、前記物体の移動速度が所定値以上の場合及び単位時間あたりにおける環境光の光量変化が所定値以上の場合の少なくとも1つを含む、
(2)に記載の情報処理装置。
前記イベントカメラは、現時刻に撮影された画像と前時刻に撮影された画像との差異部分を表すイベント画像を出力するカメラである、
(1)から(3)の何れか1つに記載の情報処理装置。
イベントカメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記イベントカメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理をコンピュータが実行する情報処理方法。
コンピュータに、
イベントカメラにより撮影された物体の画像から、撮影された前記物体を点として捉えた点情報を出力し、
前記イベントカメラと対応する方向を向いた第2カメラにより撮影された前記物体の画像から、撮影された前記物体を識別した識別情報を出力し、
前記点情報及び前記識別情報を対応付ける、
処理を実行させるための情報処理プログラム。
日本国特許出願第2022-170165号
日本国特許出願第2022-172777号
日本国特許出願第2022-175679号
日本国特許出願第2022-181362号
日本国特許出願第2022-182131号
日本国特許出願第2022-186040号
日本国特許出願第2022-187648号
日本国特許出願第2022-187649号
日本国特許出願第2022-189546号
日本国特許出願第2022-210884号
日本国特許出願第2023-036967号
日本国特許出願第2023-036975号
日本国特許出願第2023-078024号
日本国特許出願第2023-080388号
Claims (28)
- 車両の自動運転を制御する前記車両に搭載された制御装置の作動に関わる物体を検出した検出結果を取得する取得部と、
前記検出結果に基づいて、前記制御装置における冷却を実行する実行部と
を備える冷却実行装置。 - 前記取得部は、前記検出結果として、物体の1フレームの画像から前記物体の存在位置を示す点を抽出し、前記物体の存在位置を示す点の、所定の座標軸に沿った動きを示す動き情報を100フレーム/秒以上のフレームレートで取得する
請求項1に記載の冷却実行装置。 - 前記検出結果を用いて、前記制御装置の作動を予測する予測部をさらに備え、
前記予測部は、前記検出結果と、前記検出結果が取得された際の前記制御装置の作動状況と、を学習データとした機械学習によって生成された学習モデルを用いて、前記制御装置の作動を予測する、
請求項2に記載の冷却実行装置。 - 前記予測部は、前記制御装置における複数の部位のそれぞれの温度変化をさらに予測し、
前記実行部は、前記制御装置における前記部位の冷却を制御する
請求項3に記載の冷却実行装置。 - 車両の自動運転を制御する前記車両に搭載された制御装置の作動に関わる物体を検出した検出結果を取得し、
前記検出結果に基づいて、前記制御装置における冷却を実行する
を処理をコンピュータが実行する冷却実行方法。 - 車両の自動運転を制御する前記車両に搭載された制御装置の作動に関わる物体を検出した検出結果を取得し、
前記検出結果に基づいて、前記制御装置における冷却を実行する
を処理をコンピュータに実行させるための冷却実行プログラム。 - 対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得する取得部と、
前記取得部が取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる実行部と、
を備えた請求項1記載の冷却実行装置。 - 前記実行部は、予め定められた条件を満たした場合に、前記情報処理装置に対する冷却を停止させる、
請求項7記載の冷却実行装置。 - 前記実行部は、前記予め定められた条件としての前記情報処理装置によって検出された前記物体が検出されなくなるという条件を満たした場合に、前記情報処理装置に対する冷却を停止させる、
請求項8に記載の冷却実行装置。 - 前記実行部は、前記予め定められた条件としての前記情報処理装置によって検出された前記物体が検出範囲外に向かっているという条件を満たした場合に、前記情報処理装置に対する冷却を停止させる、
請求項8に記載の冷却実行装置。 - 前記実行部は、予め定められた条件を満たした場合に、前記情報処理装置に対する冷却を予め定められた時間経過後に停止させる、
請求項8~請求項10のいずれか1項に記載の冷却実行装置。 - 前記取得部が取得した前記検出結果に基づいて、前記情報処理装置の作動状況を予測する予測部を備え、
前記実行部は、前記予測部による前記情報処理装置の作動状況の予測結果に基づいて、前記情報処理装置に対する冷却を実行させる、
請求項8に記載の冷却実行装置。 - 前記予測部は、前記情報処理装置の温度変化を予測し、
前記実行部は、前記予測部による前記情報処理装置の温度変化の予測結果に応じた冷却手段を用いて、前記情報処理装置に対する冷却を実行させる、
請求項12に記載の冷却実行装置。 - 前記予測部は、前記情報処理装置の温度が閾値以下となる時期を予測し、
前記実行部は、前記予め定められた条件としての前記予測部によって予測された時期になるという条件を満たした場合に、前記情報処理装置に対する冷却を停止させる、
請求項12に記載の冷却実行装置。 - 前記取得部が取得する前記検出結果は、前記点情報である、
請求項8に記載の冷却実行装置。 - 前記取得部が取得した前記検出結果に基づいて、前記情報処理装置の作動状況及び温度変化を予測する予測部と、
前記予測部による前記情報処理装置の作動状況の予測結果に基づいて、前記情報処理装置に対する冷却を実行させるとともに、前記予測部による前記情報処理装置の温度変化の予測結果に基づいて、前記情報処理装置に対する冷却量を調整させる実行部と、
を備えた請求項7記載の冷却実行装置。 - 前記予測部は、前記情報処理装置の温度のピークをさらに予測し、
前記実行部は、前記予測部によって予測された前記温度のピークに応じた冷却量で前記情報処理装置に対する冷却を実行させる、
請求項16に記載の冷却実行装置。 - 前記実行部は、前記予測部によって予測された前記温度のピークに至る前に前記温度のピークに応じた冷却量となるように前記情報処理装置に対する冷却を実行させる、
請求項17に記載の冷却実行装置。 - 前記情報処理装置は移動体に搭載されており、
前記予測部は、前記検出結果において前記物体が前記移動体に近づく動きをしている場合、前記情報処理装置の温度が上昇すると予測し、
前記実行部は、前記予測部によって前記情報処理装置の温度上昇が予測された場合に、前記情報処理装置に対する冷却量を増量させる、
請求項17に記載の冷却実行装置。 - 対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得し、
取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる、
処理をコンピュータが実行する冷却実行方法。 - コンピュータに、
対応する方向を向いた複数のカメラで撮影された物体の画像から前記物体を点として捉えた点情報及び前記物体を識別した識別情報を出力するとともに、前記点情報及び前記識別情報を対応付ける情報処理装置による前記物体の検出結果を取得し、
取得した前記検出結果に基づいて、前記情報処理装置に対する冷却を実行させる、
処理を実行させるための冷却実行プログラム。 - 自動運転を制御する制御装置を備えた車両であって、
モータを含むパワーユニットと、
前記制御装置の作動に関わる物体を検出した検出結果を取得する取得部と、
前記検出結果に基づいて、前記制御装置における冷却を実行する実行部と、を含み、
前記制御装置は、前記検出結果に伴う前記車両の減速時、前記モータで生成された回生電力によって前記冷却を含む前記車両の各部を制御する車両。 - 前記制御装置は、前記検出結果に伴う前記車両の減速が開始されてから停止するまで、前記回生電力のみによって前記車両の各部が制御されるように、前記車両の減速度を制御する
請求項22に記載の車両。 - 前記車両のブレーキ操作を行うブレーキアクチュエータを備え、
前記制御装置は、前記検出結果に伴う前記車両の減速時、前記回生電力によって前記ブレーキアクチュエータを制御する
請求項22に記載の車両。 - 前記車両のステアリング操作を行うステアリングアクチュエータを備え、
前記制御装置は、前記検出結果に伴う前記車両の減速時、前記回生電力によって前記ステアリングアクチュエータを制御する
請求項23または請求項24に記載の車両。 - 前記取得部は、前記検出結果として、物体の1フレームの画像から前記物体の存在位置を示す点を抽出し、前記物体の存在位置を示す点の、所定の座標軸に沿った動きを示す動き情報を100フレーム/秒以上のフレームレートで取得する
請求項22記載の車両。 - 前記検出結果を用いて、前記制御装置の作動を予測する予測部をさらに備え、
前記予測部は、前記検出結果と、前記検出結果が取得された際の前記制御装置の作動状況と、を学習データとした機械学習によって生成された学習モデルを用いて、前記制御装置の作動を予測する
請求項26に記載の車両。 - 前記予測部は、前記制御装置における複数の部位のそれぞれの温度変化をさらに予測し、
前記実行部は、前記制御装置における前記部位の冷却を制御する
請求項27に記載の車両。
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202380074537.XA CN120112446A (zh) | 2022-10-24 | 2023-10-23 | 冷却执行装置、冷却执行方法、冷却执行程序以及车辆 |
| EP23882594.7A EP4610138A1 (en) | 2022-10-24 | 2023-10-23 | Cooling execution device, cooling execution method, cooling execution program, and vehicle |
Applications Claiming Priority (28)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2022170165 | 2022-10-24 | ||
| JP2022-170165 | 2022-10-24 | ||
| JP2022172777 | 2022-10-27 | ||
| JP2022-172777 | 2022-10-27 | ||
| JP2022175679 | 2022-11-01 | ||
| JP2022-175679 | 2022-11-01 | ||
| JP2022181362 | 2022-11-11 | ||
| JP2022-181362 | 2022-11-11 | ||
| JP2022182131 | 2022-11-14 | ||
| JP2022-182131 | 2022-11-14 | ||
| JP2022-186040 | 2022-11-21 | ||
| JP2022186040 | 2022-11-21 | ||
| JP2022187649 | 2022-11-24 | ||
| JP2022187648 | 2022-11-24 | ||
| JP2022-187648 | 2022-11-24 | ||
| JP2022-187649 | 2022-11-24 | ||
| JP2022-189546 | 2022-11-28 | ||
| JP2022189546 | 2022-11-28 | ||
| JP2022-210884 | 2022-12-27 | ||
| JP2022210884A JP2024070766A (ja) | 2022-11-11 | 2022-12-27 | 車両 |
| JP2023036967 | 2023-03-09 | ||
| JP2023-036967 | 2023-03-09 | ||
| JP2023036975A JP2024062338A (ja) | 2022-10-24 | 2023-03-09 | 冷却実行装置、冷却実行方法、及び冷却実行プログラム |
| JP2023-036975 | 2023-03-09 | ||
| JP2023078024A JP2024062356A (ja) | 2022-10-24 | 2023-05-10 | 冷却実行装置、冷却実行方法、及び冷却実行プログラム |
| JP2023-078024 | 2023-05-10 | ||
| JP2023080388A JP2024062359A (ja) | 2022-10-24 | 2023-05-15 | 冷却実行装置、冷却実行方法、及び冷却実行プログラム |
| JP2023-080388 | 2023-05-15 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024090389A1 true WO2024090389A1 (ja) | 2024-05-02 |
Family
ID=90830896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2023/038226 Ceased WO2024090389A1 (ja) | 2022-10-24 | 2023-10-23 | 冷却実行装置、冷却実行方法、冷却実行プログラム、及び車両 |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP4610138A1 (ja) |
| CN (1) | CN120112446A (ja) |
| WO (1) | WO2024090389A1 (ja) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018116840A1 (ja) * | 2016-12-19 | 2018-06-28 | 日立オートモティブシステムズ株式会社 | 冷却装置の制御装置 |
| JP2022035198A (ja) | 2020-08-20 | 2022-03-04 | ソフトバンク株式会社 | 移動体、通信システム、通信制御方法及びプログラム |
| JP2022170165A (ja) | 2021-04-28 | 2022-11-10 | フォルシアクラリオン・エレクトロニクス株式会社 | 振動発生装置 |
| JP2022172777A (ja) | 2021-05-07 | 2022-11-17 | 日本特殊陶業株式会社 | スパークプラグ |
| JP2022175679A (ja) | 2021-05-14 | 2022-11-25 | 株式会社ジェイ・エム・エス | ステント収納治具及びステントデリバリーシステム |
| JP2022182131A (ja) | 2021-05-27 | 2022-12-08 | トヨタ自動車株式会社 | 情報処理装置、情報処理方法、およびプログラム |
| JP2022181362A (ja) | 2021-05-26 | 2022-12-08 | Tdk株式会社 | リチウムイオン二次電池 |
| JP2022186040A (ja) | 2021-06-04 | 2022-12-15 | 棚橋工業株式会社 | 棚部材 |
| JP2022187649A (ja) | 2021-06-08 | 2022-12-20 | 株式会社東芝 | 情報処理装置、測位システムおよび移動体の測位方法 |
| JP2022187648A (ja) | 2021-06-08 | 2022-12-20 | ソニーセミコンダクタソリューションズ株式会社 | メモリセルアレイユニット |
| JP2022189546A (ja) | 2021-06-11 | 2022-12-22 | 日産化学株式会社 | 脂質の流動化を高める組成物 |
| JP2023036967A (ja) | 2018-09-10 | 2023-03-14 | ペキン シャオミ モバイル ソフトウェア カンパニー, リミテッド | セカンダリセルのビーム障害の報告方法、装置及び記憶媒体 |
| JP2023036975A (ja) | 2019-03-27 | 2023-03-14 | ウシオ電機株式会社 | 製品検査方法及び製品検査装置 |
| JP2023078024A (ja) | 2021-11-25 | 2023-06-06 | イリソ電子工業株式会社 | コネクタ |
| JP2023080388A (ja) | 2021-11-30 | 2023-06-09 | 国立研究開発法人産業技術総合研究所 | As(III)とAs(V)の分離方法と分離装置 |
-
2023
- 2023-10-23 CN CN202380074537.XA patent/CN120112446A/zh active Pending
- 2023-10-23 WO PCT/JP2023/038226 patent/WO2024090389A1/ja not_active Ceased
- 2023-10-23 EP EP23882594.7A patent/EP4610138A1/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018116840A1 (ja) * | 2016-12-19 | 2018-06-28 | 日立オートモティブシステムズ株式会社 | 冷却装置の制御装置 |
| JP2023036967A (ja) | 2018-09-10 | 2023-03-14 | ペキン シャオミ モバイル ソフトウェア カンパニー, リミテッド | セカンダリセルのビーム障害の報告方法、装置及び記憶媒体 |
| JP2023036975A (ja) | 2019-03-27 | 2023-03-14 | ウシオ電機株式会社 | 製品検査方法及び製品検査装置 |
| JP2022035198A (ja) | 2020-08-20 | 2022-03-04 | ソフトバンク株式会社 | 移動体、通信システム、通信制御方法及びプログラム |
| JP2022170165A (ja) | 2021-04-28 | 2022-11-10 | フォルシアクラリオン・エレクトロニクス株式会社 | 振動発生装置 |
| JP2022172777A (ja) | 2021-05-07 | 2022-11-17 | 日本特殊陶業株式会社 | スパークプラグ |
| JP2022175679A (ja) | 2021-05-14 | 2022-11-25 | 株式会社ジェイ・エム・エス | ステント収納治具及びステントデリバリーシステム |
| JP2022181362A (ja) | 2021-05-26 | 2022-12-08 | Tdk株式会社 | リチウムイオン二次電池 |
| JP2022182131A (ja) | 2021-05-27 | 2022-12-08 | トヨタ自動車株式会社 | 情報処理装置、情報処理方法、およびプログラム |
| JP2022186040A (ja) | 2021-06-04 | 2022-12-15 | 棚橋工業株式会社 | 棚部材 |
| JP2022187649A (ja) | 2021-06-08 | 2022-12-20 | 株式会社東芝 | 情報処理装置、測位システムおよび移動体の測位方法 |
| JP2022187648A (ja) | 2021-06-08 | 2022-12-20 | ソニーセミコンダクタソリューションズ株式会社 | メモリセルアレイユニット |
| JP2022189546A (ja) | 2021-06-11 | 2022-12-22 | 日産化学株式会社 | 脂質の流動化を高める組成物 |
| JP2023078024A (ja) | 2021-11-25 | 2023-06-06 | イリソ電子工業株式会社 | コネクタ |
| JP2023080388A (ja) | 2021-11-30 | 2023-06-09 | 国立研究開発法人産業技術総合研究所 | As(III)とAs(V)の分離方法と分離装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| EP4610138A1 (en) | 2025-09-03 |
| CN120112446A (zh) | 2025-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2024062337A (ja) | 情報処理装置、情報処理方法、及び情報処理プログラム | |
| WO2024090389A1 (ja) | 冷却実行装置、冷却実行方法、冷却実行プログラム、及び車両 | |
| JP2024062356A (ja) | 冷却実行装置、冷却実行方法、及び冷却実行プログラム | |
| WO2025084211A1 (ja) | 情報処理装置、情報処理方法、及び情報処理プログラム | |
| WO2024090328A1 (ja) | 情報処理装置、情報処理方法、及び情報処理プログラム | |
| EP4610128A1 (en) | Information processing device, vehicle, information processing method, and program | |
| JP2024066386A (ja) | 情報処理装置、情報処理方法、及び情報処理プログラム | |
| JP2024062325A (ja) | 情報処理装置及びプログラム |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23882594 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 202380074537.X Country of ref document: CN |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 202517049073 Country of ref document: IN |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2023882594 Country of ref document: EP |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2023882594 Country of ref document: EP Effective date: 20250526 |
|
| WWP | Wipo information: published in national office |
Ref document number: 202380074537.X Country of ref document: CN |
|
| WWP | Wipo information: published in national office |
Ref document number: 202517049073 Country of ref document: IN |
|
| WWP | Wipo information: published in national office |
Ref document number: 2023882594 Country of ref document: EP |