WO2015013311A1

WO2015013311A1 - Vehicle imaging system

Info

Publication number: WO2015013311A1
Application number: PCT/US2014/047668
Authority: WO
Inventors: Ankit SINGH; Mario J. Enriquez ORTIZ; Christopher Kring
Original assignee: Johnson Controls Technology Co
Current assignee: Johnson Controls Technology Co
Priority date: 2013-07-22
Filing date: 2014-07-22
Publication date: 2015-01-29
Anticipated expiration: 2016-01-22

Abstract

A vehicle imaging system of a vehicle including a camera mounted to the vehicle and defining a field of view at least partially surrounding the vehicle, the camera configured to collect image data from within the field of view, a sensor mounted to the vehicle and operable to detect a position of a driver with respect to the vehicle, and a monitor positioned within the vehicle for viewing by the driver. Where the monitor displays an image including at least a portion of the image data, and where the image displayed by the monitor changes in response to the position of the driver detected by the sensor.

Description

VEHICLE IMAGING SYSTEM

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to co-pending U.S. Provisional Patent Application No. 61/856,961 filed on July 22, 2013, the entire content of which is incorporated herein by reference.

FIELD OF THE INVENTION

[0002] The present invention relates to a vehicle imaging system. More particularly, the invention relates to a system for collecting image data from around a vehicle with one or more cameras and producing an interactive display within the vehicle.

BACKGROUND OF THE INVENTION

[0003] Conventional rear view mirrors provide a driver a view of the area behind a vehicle. However, the field of view is limited to approximately 20 to 30 degrees with a flat mirror.

Although panoramic mirrors are available, they include inherent image distortion and have not been widely accepted in the automotive industry. To get an undistorted view all around the rear of the vehicle, side view mirrors are also provided on each side of the vehicle, and drivers are left to manually check any blind spot(s) between the fields of view of the mirrors by turning their head away from the forward path of the vehicle. Regardless of whether the driver is looking through the rear view mirror or directly back over his or her shoulder toward the area behind the vehicle, automotive designs often impede a clear view toward the rear vehicle corners with large C-pillars (or D-pillars), headrests, and headliners, often for increased crash safety and/or styling.

[0004] Although rear-facing "back-up" cameras have become quite prevalent in the automotive industry in recent years, these typically operate with a limited rearward view towards the ground, and only at a time when the driver is operating the vehicle in reverse.

SUMMARY OF THE INVENTION

[0005] In one aspect, the invention provides a vehicle imaging system including an exterior- facing camera configured to collect image data of an area surrounding a vehicle, a monitor within the vehicle configured to receive the image data and display at least a portion of the area surrounding the vehicle, and an interior-facing sensor configured to determine a position of a head of an occupant of the vehicle. At least one aspect of the monitor display is responsive to change based on the head position of the occupant.

[0006] In another aspect, the invention provides a vehicle imaging system including a plurality of exterior-facing cameras each configured to collect image data of a corresponding area surrounding a vehicle. A controller having one or more processors is configured to stitch together the image data of each of the plurality of exterior-facing cameras to create a panoramic image. A monitor within the vehicle is configured to display at least a portion of the panoramic image. In some aspects, the imaging system further includes means for defining a region of interest within the panoramic image, and wherein the monitor is configured to display the region of interest.

[0007] In still another aspect, the invention provides a vehicle imaging system of a vehicle. The vehicle imaging system including a camera mounted to the vehicle and defining a field of view, the camera configured to collect image data from within the field of view, a sensor mounted to the vehicle and operable to detect a position of a driver with respect to the vehicle, and a monitor positioned within the vehicle for viewing by the driver, where the monitor displays an image including at least a portion of the image data, and where the image displayed by the monitor changes in response to the position of the driver detected by the sensor.

[0008] In still another aspect, the invention provides a vehicle imaging system of a vehicle. The vehicle imaging system including a first camera mounted to the vehicle and defining a first field of view at least partially surrounding the vehicle, the first camera configured to collect image data within the first field of view, a second camera mounted to the vehicle and defining a second field of view at least partially surrounding the vehicle, the second camera configured to collect image data from within the second field of view, where the second field of view at least partially overlaps the first field of view. The vehicle imaging system also includes a controller in operable communication with the first camera and the second camera, the controller operable to combine the image data of the first field of view and the image data of the second field of view to create a master image, and a monitor positioned within the vehicle for viewing by a driver, and where the monitor displays an image that is at least a portion of the master image, and where the controller is configured to change the image displayed on the monitor.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Fig. 1 is a top view of a rear portion of a vehicle including cameras of an imaging system according to one aspect of the invention.

[0010] Fig. 2 is a schematic view illustrating a matrix of a panoramic image stitched together from the cameras of Fig. 1, and a region of interest within the matrix, that is displayed on a monitor within the vehicle.

[0011] Fig. 3 is a schematic view of a vehicle imaging system. DETAILED DESCRIPTION

[0012] Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways.

[0013] As shown in Fig. 1, a vehicle 20 is provided with an imaging system 24 for providing an improved rearward view to the driver of the vehicle 20. The vehicle 20 includes a forward end (not shown) and a rearward end 28. The driver occupies a seat within the vehicle which faces in the forward direction A. The driver generally has a clear panoramic outward view toward the front of the vehicle 20, and the vehicle 20 may include one or more mirrors configured to provide an outward view to the rear of the vehicle 20. In addition to the mirror(s) or in lieu thereof, the imaging system 24 includes one or more cameras 32. The cameras 32 can be positioned at the rear end 28 of the vehicle 20 as shown in Fig. 1, or in another configuration that provides a panoramic view (e.g., approximately 180 degrees). The cameras 32 are exterior- facing and can be positioned inside or outside the vehicle 20. Furthermore, the cameras 32 are generally rearward-facing. In the exemplary, non-limiting construction, the cameras 32 include a central camera 32 and two side cameras 32, each of which includes a wide angle lens. The two side cameras 32 are positioned laterally outward (e.g., about 75cm) to opposite sides of the center of the vehicle 20 (coinciding with axis 36), adjacent the rear corners of the vehicle 20, and forward (e.g., about 15cm) of the central camera 32. The side cameras 32 are aimed rearward, at an angle a from the vehicle axis 36. The cameras 32 may have overlapping fields of view to facilitate producing a panoramic image as described below. In the illustrated construction, each camera 32 has a 120-degree undistorted field of view, and the fields of view overlap to sweep a total field of view of 180 degrees. The cameras 32 are all supported by the vehicle 20 at a common height (e.g., 100cm) above the ground.

[0014] When the vehicle is operated, the cameras 32 operate to collect image data of an area, or field of view, surrounding the rearward end 28 of the vehicle 20. This can be done by each camera 32 of the multi-camera set capturing image data within a respective field of view and sending that data to a controller 30 containing one or more processors. The controller 30, in turn, identifies overlapping image data, and stitches the multiple images together (i.e., with a computer processor and suitable software) to create a panoramic or master image (see Fig. 3). The images are stitched based on a stitch distance between the cameras 32 (i.e., distance of the object from the camera) and changes in real-time to reduce distortion. The panoramic image, or a portion thereof, can then be sent to a monitor 34 where it can be displayed to the driver of the vehicle 20 as described in further detail below to provide the driver near real-time unobstructed panoramic viewing of the rear area behind the vehicle 20 (Fig. 3).

[0015] Within the vehicle 20, a screen or monitor 34 is provided at a location visible to the driver. The monitor 34 can be at a top, central location of the windshield 38 incorporated with or used in lieu of a passive rear view mirror. Alternately, the monitor 34 can be incorporated into the interior trim of the vehicle 20 at any position (e.g., central control stack, instrument panel, A- pillar, etc.). The monitor 34 can be configured to actively display an image from the cameras 32 at all times during vehicle operation, or at least at all times during vehicle movement. In other constructions, the monitor 34 may be coupled to a driver-operable control panel (not shown) to turn the display on and off and optionally, control other parameters of the image displayed.

[0016] In generating the stitched panoramic image, more image data will be available than can be presented to the driver without distortion. Therefore, the controller 30 may determine which part of the total panoramic image is displayed on the monitor 34. The controller 30 can also control the relative size or amount of the total panoramic image that is displayed on the monitor 34 (i.e., zoom factor) at a particular time. The controller 30 may determine such parameters in response to a tracked position or orientation of the driver's head 40 (or

particularly, the eyes) as shown schematically in Fig. 2. In particular, the controller 30 receives information from a sensor 22 regarding the relative position of the driver's head or eyes with respect to the vehicle 20.As shown in Fig. 2, the total panoramic or master image from the cameras 32 is represented as a matrix 44 having a height H and a width W. A region of interest (ROI) is defined within the matrix 44 and the image data within the ROI is sent to the monitor 34 for display to the driver. In one construction, the controller 30 is operable to continuously change the size of the ROI in real time (i.e., height ROI H and width ROI W of the ROI) based on the distance between the driver's head 40 and the monitor 34 and is operable to continuously change the position of the ROI within the matrix in real time (i.e., lengthwise X and heightwise Y offset from an origin) based on the position of the driver's head 40 relative to the monitor 34. Thus, the portion of the image displayed by the monitor 34 is responsive to the driver's movements to mimic the optics of a mirror. However, the imaging system 24 allows the driver to get an entire panoramic view around the rear end 28 of the vehicle 20 without exaggerated movements from the normal driving position and without ever turning away from the forward direction A. The position and/or orientation of the driver's head 40 may be tracked with an interior-facing sensor 22 such as a camera, along with corresponding face-tracking software. In alternative constructions, a separate display management system may be provided to determine which part of the total panoramic image is displayed on the monitor 34.

[0017] In some constructions, the imaging system 24 is provided with additional

functionality such as a back-up mode and/or a backseat view mode. In the back-up mode, which may be activated by driver control or automatically upon engaging reverse gear, a lower portion of the total panoramic image matrix is displayed on the monitor 34 to provide the driver with a better picture of the ground surface surrounding the rear end 28 of the vehicle 20. Although this may be done without any physical camera movement in some construction, one or more of the cameras 32 may physically move or tilt downward in back-up mode. The focus of the image may also change to a near field focus in back-up mode. The size and position of the ROI during back-up mode may be fixed or variable. In backseat view mode, the monitor 34 uses a camera within the vehicle 20 to capture an image of the backseat area (and one or more passengers therein) for display on the monitor 34. The camera may be the same camera used for tracking the driver's head 40 during normal operation, or may be a separate camera. Backseat view mode may be activated by a dedicated driver control, and can enable the driver to monitor 34 the backseat contents or passenger(s) without physically turning away from the forward direction A. The view to the backseat area may be variable by the driver to display different areas as desired. Furthermore, a similar mode may be provided for displaying a passenger sitting next to the driver in a front seat. With an alternate array of cameras and/or an alternate image stitching technique, the imaging system 24 may further be configured to display a bird's eye view of the rear end 28 of the vehicle 20, or of the entire vehicle 20, which may assist in parallel parking or other maneuvers.

Claims

1. An imaging system of a vehicle, the vehicle imaging system comprising:

a camera mounted to the vehicle and defining a field of view at least partially

surrounding the vehicle, the camera configured to collect image data within the field of view; a sensor mounted to the vehicle and operable to detect a position of a driver with respect to the vehicle; and

a monitor positioned within the vehicle for viewing by the driver, wherein the monitor displays an image including at least a portion of the image data, and wherein the image displayed by the monitor changes in response to the position of the driver detected by the sensor.

2. The vehicle imaging system of claim 1, wherein the monitor continuously changes the image displayed by the monitor in real time to mimic the optics of a conventional mirror.

3. The vehicle imaging system of claim 1, wherein the sensor is configured to detect a head of the driver.

4. The vehicle imaging system of claim 1, wherein the sensor is configured to detect an eye of the driver.

5. The vehicle imaging system of claim 1, wherein the sensor is a camera.

6. The vehicle imaging system of claim 1, wherein the camera is a first camera and wherein the field of view is a first field of view, the vehicle imaging system further comprising a second camera mounted to the vehicle and defining a second field of view at least partially surrounding the vehicle, wherein the second camera is configured to collect image data within the second field of view, wherein the first field of view at least partially overlaps the second field of view, and wherein the monitor is operable to display at least a portion of a combination of the image data collected from the first camera and the image data collected from the second camera.

7. The vehicle imaging system of claim 6, further comprising a controller in operable communication with the first camera and the second camera, the controller configured to combine the image data of the first camera and the image data of the second camera to create a master image.

8. The vehicle imaging system of claim 6, further comprising a third camera mounted to the vehicle and defining a third field of view at least partially surrounding the vehicle, the third camera configured to collect image data from within the third field of view, wherein the third field of view at least partially overlaps at least one of the first field of view and the second field of view, and wherein the monitor is operable to display at least a portion of a combination of the image data collected from the first camera, the image data collected from the second camera, and the image data collected from the third camera.

9. The vehicle imaging system of claim 8, wherein the first field of view, the second field of view, and the third field of view extend over a combined field of view of at least 180 degrees.

10. The vehicle imaging system of claim 1, wherein the monitor is positioned proximate a top center of a windshield of the vehicle.

11. The vehicle imaging system of claim 6, wherein the first camera is aimed along a first axis in a generally rear- facing direction of the vehicle, and wherein the second camera is aimed angularly outward with respect to the first axis.

12. The vehicle imaging system of claim 1, further comprising a third camera mounted to the vehicle and configured to collect image data of a back seat area of the vehicle.

13. The vehicle imaging system of claim 1, further comprising a third camera mounted to the vehicle and configured to collect image data of a front seat passenger area of the vehicle.

14. An imaging system of a vehicle, the vehicle imaging system comprising:

a first camera mounted to the vehicle and defining a first field of view at least partially surrounding the vehicle, the camera configured to collect image data from within the first field of view;

a second camera mounted to the vehicle and defining a second field of view at least partially surrounding the vehicle, the camera configured to collect image data from within the second field of view, wherein the second field of view at least partially overlaps the first field of view;

a controller in operable communication with the first camera and the second camera, the controller operable to combine the image data of the first camera and the image data of the second camera to create a master image; and

a monitor positioned within the vehicle for viewing by a driver, and wherein the monitor displays an image that is at least a portion of the master image, and wherein the controller is configured to change the image displayed on the monitor.

15. The vehicle imaging system of claim 14, wherein the controller continuously changes the image displayed on the monitor in real time to mimic the optics of a conventional mirror.

16. The vehicle imaging system of claim 14, further comprising a sensor mounted to the vehicle, the sensor operable to detect a position of the driver with respect to the vehicle and operable to communicate with the controller to change the image displayed on the monitor.

17. The vehicle imaging system of claim 16, wherein the sensor is operable to detect a relative location of a head of the driver with respect to the vehicle.

18. The vehicle imaging system of claim 17, wherein the controller is configured to change the image displayed on the monitor at least partially dependent upon the relative location of the head of the driver with respect to the vehicle as detected by the sensor.

19. The vehicle imaging system of claim 16, wherein the sensor detects a relative location of an eye of the driver with respect to the vehicle.

20. The vehicle imaging system of claim 19, wherein the controller is configured to change the image displayed on the monitor at least partially dependent upon the relative location of the eye of the driver with respect to the vehicle as detected by the sensor.

21. The vehicle imaging system of claim 14, further comprising a third camera mounted to the vehicle and defining a third field of view at least partially surrounding the vehicle, the third camera configured to collect image data from within the third field of view, and wherein the third field of view at least partially overlaps at least one of the first field of view and the second field of view.

22. The vehicle imaging system of claim 21, wherein the first camera is aimed along a first axis in a generally rear- facing direction of the vehicle, wherein the second camera is aimed angularly outward with respect to the first axis, and wherein the third camera is aimed angularly outward with respect to the first axis opposite the second camera.

23. The vehicle imaging system of claim 21, wherein the first field of view, second field of view, and third extend over a combined field of view greater 180 degrees.

24. The vehicle imaging system of claim 14, wherein the monitor is positioned proximate a top center of a windshield of the vehicle.

25. The vehicle imaging system of claim 14, wherein the image displayed on the monitor is a region of interest of the master image, and wherein the controller is operable to manipulate a size and a location of the region of interest of the master image.

26. The vehicle imaging system of claim 14, further comprising a third camera mounted to the vehicle and configured to collect image data of a back seat area of the vehicle.

27. The vehicle imaging system of claim 14, further comprising a third camera mounted to the vehicle and configured to collect image data of a front seat passenger area of the vehicle.