WO2020178035A1

WO2020178035A1 - Method and system for detecting an object on a surface of a vehicle

Info

Publication number: WO2020178035A1
Application number: PCT/EP2020/054481
Authority: WO
Inventors: Ciaran Hughes; Jonathan Horgan
Original assignee: Connaught Electronics Ltd
Current assignee: Connaught Electronics Ltd
Priority date: 2019-03-01
Filing date: 2020-02-20
Publication date: 2020-09-10
Anticipated expiration: 2021-09-01
Also published as: CN113508394A; KR20210115045A; JP2022522765A; CN113508394B; JP7236555B2; DE102019105275A1; KR102645934B1

Abstract

The present invention provides a method and system for detecting an object on a surface of a vehicle. A processor receives one or more images from an image capture device. A predetermined background is subtracted from a predetermined region of interest of said one or more images. The region of interest of the one or more images is traced to detect at least one object on the surface of the vehicle based on a difference in pixel intensity in the region of interest. The location of the detected object in the region of interest is mapped to a position on a representative image of the surface of the vehicle. Further, the representative image of the surface of the vehicle is displayed along with an alert to the user of the vehicle. Further, an alert symbol is overlaid at the mapped position on the representative image of the surface of the vehicle.

Description

Title

Method and system for detecting an object on a surface of a vehicle.

Field

The present disclosure relates to the detection of an object inadvertently left on a surface of a vehicle.

Background

It is very common for a user of a vehicle to inadvertently leave personal objects (e.g. purse, wallet, phone, laptop, etc.) on the top of a vehicle. If the user forgets that the personal object is on the top of the vehicle, the user may enter the vehicle and drive away. This may cause the personal object to fall and may get damaged. Therefore, in such a situation, before the user drives away it is desirable have a system that locates the personal object and warns the user that the personal item has been left on the top of the vehicle.

United States patent document US6163250A discloses a system which includes a sensor which detects an object located on the outside of the vehicle. The sensor can be formed of known and commercially available components such as a force or pressure gauge attached to the inside of the vehicle roof. Many types of sensors may be employed including any one or more of capacitive and/or force/pressure sensors and are known in the art.

However, the prior art fails to detect and display the location of the personal object on the roof. Therefore, there is a need for system that displays the location of the personal item on the top of the vehicle along with an alert to the user of the vehicle.

Summary

Embodiments of the present invention are directed to a method and system for detecting an object on a surface of a vehicle, and as set out in the appended claims. The method for detecting an object on a surface of a vehicle comprises of receiving, by a processor, one or more images from at least an image capture device. A predetermined region of interest is identified from said one or more images. The region of interest of the one or more images is traced to detect at least one object on the surface of the vehicle based on a difference in pixel intensity in the region of interest. The location of the detected object in the region of interest is mapped to a position on a representative image of the surface of the vehicle. Further, the representative image of the surface of the vehicle is displayed along with an alert to a user of the vehicle. In addition, an alert symbol is overlaid at the mapped position on the representative image of the surface of the vehicle.

The system for detecting an object on a surface of a vehicle comprises an image capture device, a display device and a processor operatively coupled to the image capture device and the display device. The processor is configured to receive one or more images from the image capture device and identify a predetermined region of interest from said one or more images. The processor is further configured to trace the region of interest and detect at least one object on the surface of the vehicle based on a difference in pixel intensity. The processor is further configured to map the position of the detected object detected in the region of interest to a position on a representative image of the surface of the vehicle. Further the processor is configured to display the representative image of the surface of the vehicle and overlay a symbol at the position on the representative image of the surface of the vehicle.

Thus the method and system of the present invention provides a user of the vehicle with an alert on the display device of the vehicle including the location of the object on the roof of the vehicle. There is also provided a computer program comprising program instructions for causing a computer program to carry out the above method which may be embodied on a record medium, carrier signal or read-only memory. Brief Description of the Drawings

The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which: -

FIG. 1 A exemplarily illustrates a side view of a vehicle in accordance with some embodiments of the present invention;

FIG. 1 B exemplarily illustrates a top view of the vehicle in accordance with some embodiments of the present invention

FIG. 2 is a flow chart illustrating an embodiment of the inventive method or process for detecting an object on a surface of a vehicle, in accordance with some embodiments of the present invention;

FIG. 3 is a functional block diagram illustrating the primary components of a system/apparatus for detecting an object on a surface of a vehicle, in accordance with some embodiments of the present invention; and

FIG. 4 exemplarily illustrates a display of the vehicle, in accordance with some embodiments of the present invention.

Detailed Description of the Drawings

FIG. 1 A exemplarily illustrates a side view of a vehicle in accordance with some embodiments of the present invention which shows the vehicle 101 , an image capture device 102 arranged on the roof of the vehicle and an object 103 left over on the roof of the vehicle 101 . The dotted lines in FIG. 1 A shows the field of view of the image capture device which also captures the roof of the vehicle.

FIG. 1 B exemplarily illustrates a top view of a vehicle in accordance with some embodiments of the present invention which shows the vehicle 101 , an image capture device 102 arranged on the roof of the vehicle and an object 103 left over on the roof of the vehicle 101 . FIG. 2 is a flow chart illustrating an embodiment of the inventive method or process for detecting an object on a surface of a vehicle, in accordance with some embodiments of the present invention. The method for detecting an object on a surface of a vehicle comprises receiving 201 one or more images from at least an image capture device. A predetermined background is subtracted 202 from a predetermined region of interest of said one or more images. It will be appreciated that other methods can be used to identify an object as distinct from the background. The region of interest of the one or more images is traced 203 to detect at least one object on the surface of the vehicle based on a difference in pixel intensity in the region of interest after the background is subtracted. The location of the detected object in the region of interest is mapped 204 to a position on a representative image of the surface of the vehicle. Further, the representative image of the surface of the vehicle is displayed 205 along with alert to a user of the vehicle. Further, an alert symbol is overlaid at the mapped position on the representative image of the surface of the vehicle. For example, FIG. 4 exemplarily illustrates a display 401 showing the representative image 402 of the top surface or the vehicle 101 and an alert symbol 403 is overlaid on the representative image. Further the alert symbol 403 substantially indicates the location or position of the detected object on the roof of the vehicle (see FIG. 1 B).

The generation of the predetermined region of interest is at least based on the position of the image capture device with respect to the surface of the vehicle. For example, the position of the image capture device with respect to the surface of the vehicle determines the region of interest. In other words, the region of interest is a portion of an image which captures the surface of the vehicle.

Further as mentioned above, generating the predetermined background is based on at least one image of the surface of the vehicle. For example, an image of the surface of the vehicle may be captured using the image capture device while there is no object placed on said surface. In another embodiment, the image capture device may capture a plurality of images at different times when the vehicle is in motion. The portion of the image which remains substantially static and does not change across images is considered the background of the image. Also, the contour of the static portion of said images may define the region of interest as mentioned above. The method further comprises extracting the image of the detected object and classifying the object using a machine learning algorithm or a neural network based model and providing an audio alert to a user of the vehicle based on the classification of the object. For example, if an apple is left on the surface of the vehicle, an alert may be sounded“Alert! An apple is on the roof of the vehicle”.

The method further comprises sending a signal to the electronic control unit (ECU) of the vehicle for preventing motion of the vehicle or limiting the speed of the vehicle. For example, if an object is detected on the surface of the vehicle, then the ECU is signalled to limit the speed of the vehicle or prevent the vehicle from moving, such that the object inadvertently left on the vehicle does not fall off.

The method while tracing the region of interest ignores objects attached to the vehicle, where the objects are a part of the vehicle. For example, a luggage holder mounted on the roof of the vehicle.

The above method could be applied to any surface of the vehicle e.g. a roof of the vehicle, a bonnet of the vehicle, or a car boot cover of the vehicle. FIG. 3 is a functional block diagram illustrating the primary components of a system/apparatus 300 for detecting an object on a surface of a vehicle, in accordance with some embodiments of the present invention. The system comprises a processor 301 , a memory 302, a transceiver 303, an image capture device 304 and a display 305. The processor 301 is operably coupled to the memory 302, the transceiver 303, the image capture device 304 and the display 305. Further, an electronic control unit (ECU) 31 1 is in communication with the processor 301 via said transceiver 303. Also, a speed governor 312 of the vehicle is operably coupled to the ECU 31 1. In a preferable embodiment the image capture device 304 is a fish eye camera.

The processor 301 is configured to receive one or more images from the image capture device 304. Further, the processor 301 subtracts a predetermined background from a predetermined region of interest from said one or more images. The processor 301 is further configured to trace the region of interest and detect at least one object on the surface of the vehicle based on a difference in pixel intensity within the region of interest. The processor 301 is further configured to map the position of the detected object detected in the region of interest to a position on a representative image of the surface of the vehicle. Further, the processor 301 causes the display 305 to display said representative image of the surface of the vehicle and overlay a symbol at the position on the representative image of the surface of the vehicle. For example, FIG. 4 exemplarily illustrates a display 401 showing the representative image 402 of the top surface or the vehicle 101 and an alert symbol 403 is overlaid on the representative image. Further the alert symbol 403 substantially indicates the location or position of the detected object on the roof of the vehicle (see FIG. 1 B). The generation of the predetermined region of interest by the processor 301 is at least based on the position of the image capture device 304 with respect to the surface of the vehicle. For example, the position of the image capture device 304 with respect to the surface of the vehicle determines the region of interest. In other words, the region of interest is a portion of an image which captures the surface of the vehicle.

Further the processor is configured to generate the predetermined background, which is based on at least one image of the surface of the vehicle. For example, an image of the surface of the vehicle may be captured using the image capture device 304 while there is no object placed on said surface and said image stored in the memory 302 as a background image. In another embodiment, the processor 301 may cause the image capture device 304 to capture a plurality of images at different times when the vehicle is in motion. Further, the processor may determine a portion of the image which remains substantially static/unchanged through said captured plurality of images, where said determined portion of image is stored in the memory 302 as the background image. Also, the contour of the static portion of said images may define the region of interest as mentioned above and the configuration of said contour stored in memory 302 as the region of interest.

The processor 301 is further configured to extract the image of detected object and classify the object using a machine learning algorithm or a neural network based model. The system further comprising an audio output device operably coupled to the processor 301 and the processor 301 configured to provide an audio alert to a user of the vehicle based on the classification of the object. For example, if an apple is left on the surface of the vehicle, an alert may be sounded “Alert! An apple is on the roof of the vehicle”.

The processor 301 is further configured to send a signal to the electronic control unit (ECU) 31 1 of the vehicle for preventing motion of the vehicle or limiting the speed of the vehicle. For example, if an object is detected on the surface of the vehicle, then the ECU is signalled to limit the speed of the vehicle or prevent the vehicle from moving using the speed governor 312, such that the object inadvertently left on the vehicle does not fall off.

Further, a person ordinarily skilled in the art will appreciate that the various illustrative logical/functional blocks, modules, circuits, techniques/algorithms and process steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or a combination of hardware and software. To clearly illustrate this interchangeability of hardware and a combination of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or a combination of hardware and software depends upon the design choice of a person ordinarily skilled in the art. Such skilled artisans may implement the described functionality in varying ways for each particular application, but such obvious design choices should not be interpreted as causing a departure from the scope of the present invention.

The process described in the present disclosure may be implemented using various means. For example, the process described in the present disclosure may be implemented in hardware, firmware, software, or any combination thereof. For a hardware implementation, the processing units, or processors(s) may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or a combination thereof. For a firmware and/or software implementation, software codes may be stored in a memory and executed by a processor. Memory may be implemented within the processor unit or external to the processor unit. As used herein the term“memory” refers to any type of volatile memory or nonvolatile memory. Furthermore, although elements of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. a memory stick or hard disk. The carrier may be an electrical or optical signal which may be transmited via an electrical or an optical cable or by radio or other means.

In the specification the terms "comprise, comprises, comprised and comprising" or any variation thereof and the terms include, includes, included and including" or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.

Claims

1. A method for detecting an object on a surface of a vehicle comprising:

receiving by a processor one or more images from at least an image capture device;

identifying a predetermined region of interest from said one or more images;

tracing the region of interest and detecting at least one object on the surface of the vehicle based on a difference in pixel intensity;

mapping the position of the detected object detected in the region of interest to a position on a representative image of the surface of the vehicle; and

displaying said representative image of the surface of the vehicle and overlaying a symbol at the position on the representative image of the surface of the vehicle.

2. The method of claim 1 , further comprising generating the predetermined region of interest at least based on the position of the image capture device with respect to the surface of the vehicle.

3. The method of claim 1 , further comprising subtracting a predetermined background from the predetermined region of interest from said one or more images

4. The method of claim 3, further comprising generating the predetermined background based on at least one image of the surface of the vehicle.

5. The method of claim 1 , further comprising extracting the image of detected object and classifying the object using a machine learning algorithm or a neural network based model.

6. The method of claim 5, further comprising providing an audio alert to a user of the vehicle based on the classification of the object.

7. The method of claim 1 further comprising sending a signal to the electronic control unit of the vehicle for preventing motion of the vehicle or limiting the speed of the vehicle.

8. The method of claim 1 wherein tracing the region of interest comprises ignoring objects attached to the vehicle, wherein said objects attached to the vehicle being a part of the vehicle.

9. The method claimed in any one of the preceding claims, wherein the surface of the vehicle is one or more of a roof of the vehicle, a bonnet of the vehicle, or a car boot cover of the vehicle.

10. A system for detecting an object on a surface of a vehicle comprising:

at least one image capture device;

a display device;

a processor operatively coupled to the at least one image capture device and the display device, wherein said processor configured to:

receive one or more images from the at least one image capture device;

identify a predetermined region of interest from said one or more images;

trace the region of interest and detecting at least one object on the surface of the vehicle based on a difference in pixel intensity;

map the position of the detected object detected in the region of interest to a position on a representative image of the surface of the vehicle; and

display using the display said representative image of the surface of the vehicle and overlaying a symbol at the position on the representative image of the surface of the vehicle.

1 1. The system of claim 10, wherein the processor is further configured to generate the predetermined region of interest at least based on the position of the image capture device with respect to the surface of the vehicle.

12. The system of claim 10, wherein the processor is further configured to subtract a predetermined background from the predetermined region of interest from said one or more images

13. The system of claim 12, wherein the processor is further configured to generate the predetermined background based on at least one image of the surface of the vehicle while there is no object or objects placed on the surface of the vehicle.

14. The system of claim 10, wherein the processor is further configured to extract the image of detected object and classifying the object using a machine learning algorithm or a neural network based model.

15. The system of claim 14, further comprising an audio output device operably coupled to the processor, wherein said processor provides an audio alert to a user of the vehicle based on the classification of the object using the audio output device.

16. The system of claim 10, further comprising an electronic control unit of the vehicle wherein the processor is configured to send a signal to the electronic control unit of the vehicle for preventing motion of the vehicle or limiting the speed of the vehicle.

17. The system of claim 10, wherein the image capture device is a fish eye camera.

18. A computer program comprising instructions which, when the program is executed by a data processing device/computer, cause the data processing device/computer to carry out the steps of the method of any of claims 1 to 9.

19. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 1 to 9.