WO2008151372A1

WO2008151372A1 - Traffic monitoring systems

Info

Publication number: WO2008151372A1
Application number: PCT/AU2008/000844
Authority: WO
Inventors: Andrew Lewis Christie
Original assignee: Individual
Current assignee: Individual
Priority date: 2007-06-12
Filing date: 2008-06-12
Publication date: 2008-12-18
Anticipated expiration: 2009-12-12

Abstract

A method of operating a video recording device in a vehicle to record illegal or risky behaviour by other motorists. A camera is mounted in the vehicle to provide a field of view outwards through a window. The field of view is recorded over a substantially continuous period of time. A portion of the recorded view is selected by a user, typically the driver, while the recording is in progress. The selected portion is stored and may be sent to an authority or intermediary for processing. The device is typically a mobile phone or a portable navigational unit.

Description

TRAFFIC MONITORING SYSTEMS

BACKGROUND TO THE INVENTION

This invention relates to monitoring by the public of public activities, in particular but not only to traffic surveillance. The invention includes use of personal navigation devices and mobile phones to record and potentially penalize illegal or undesirable traffic activity.

A wide range of traffic offences occur on a daily basis without having any chance of being detected by traffic authorities. These offences include red light violations, prohibited U-turns, tailgating, failure to indicate changes of lane or direction, failure to stop or give way, weaving and slow travel in the wrong lane, not to mention speeding and outright accidents. Offences and other undesirable behavior on the road can lead to injury or death, or simply frustration and road rage.

Illegal, risky and generally undesirable traffic behavior often takes place because drivers are well aware that their chance of being detected and reprimanded is low. Other drivers, cyclists or pedestrians who happen to witness such events, but were unable to record them, can sometimes use their cell phones to verbally report the behavior to the authorities. However, proof of the behavior sufficient for the authorities to follow up is generally lacking or inconvenient.

A number of in-car camera systems exist but in general these systems record a continuous stream of information using a simple stop/start control. A relatively complex editing process may also be required. The systems therefore require a relatively large memory and are not generally suitable for use in mobile phones or navigation devices.

SUMMARY OF THE INVENTION It is an object of the invention to provide improved systems for traffic surveillance, or at least to provide an alternative to existing systems.

In one aspect the invention may be said to reside in a method of operating a video recording device in a vehicle, including: mounting a camera in the vehicle to provide a field of view outwards through a window, recording the field of view over a substantially continuous period of time as the vehicle moves, selecting a portion of the recorded view while the recording is in progress, and storing the selected portion.

Preferably the recording is a substantially continuous video clip and the portion is determined by a time interval which ends at the time of the selection. Alternatively the recording may be a substantially continuous series of clips and the portion is one of the clips. A series of portions may be selected and stored while the recording is in progress. Preferably the length of each portion is in the range 10 to 30 seconds, but may be selected by the user. In general each clip includes a recording of contemporary data from a satellite or land based positioning system with the field of view.

In one embodiment the device is a navigational unit and/or a mobile phone mounted on the window. Two or more cameras may be mounted in the vehicle and record corresponding fields of view through the same or different windows.

In another aspect the invention resides in a method of operating a video recording device, including: providing a field of view for a camera in the device, recording the field of view over a substantially continuous period of time, selecting a relatively short portion of the recorded view while the recording is in progress, and storing the selected portion separate from the recording. Preferably the portion is selected and stored with minimal interaction from the user, such as one or two key pushes.

In a further aspect the invention resides in a device such as a GPS navigational unit or a mobile phone which is programmed to enable a method as defined above. In a further aspect the invention resides in a method of reporting traffic activity, including: mounting a navigational device inside a vehicle, operating a camera in the device to continuously record video information relating to traffic activity outside the vehicle, selecting one or more portions of the recorded video information, and transmitting one or more of the selected portions to an information processor or a traffic authority.

Preferably the information includes a video clip and contemporary data from a positioning system. Preferably the navigational device is a mobile phone containing a GPS receiver. Alternatively the navigational device includes a memory port, connector and/or transmitter through which the information may be transferred to a mobile phone.

In one embodiment payment is received from the information processor or the traffic authority in response to the transmitted information.

In another aspect the invention resides in a method of reporting illegal activity, including: receiving electronic information about the activity from an observer, processing the electronic information, forwarding the processed information to an authority, receiving a payment from the authority, and forwarding a part of the payment to the observer.

In one embodiment the illegal activity is a traffic infringement and the observer is another driver. The payment from the authority is a portion of a fine incurred by a party to the illegal activity. In one embodiment the electronic information is gathered and/or reported by the observer using a mobile phone and the part payment to the observer is a mobile phone credit. In general the electronic information includes a video clip with contemporary data from a satellite positioning system and the observer is a mobile phone operator.

In one embodiment processing the information includes determining characteristics of objects or people represented in the information. It may also include assessing whether the activity is or is not likely to be liable for a fine or other punishment by the authority. In general multiple items of electronic information are received from a plurality of observers. Assessment may include correlating two or more items of information relating to a single activity from two or more observers, or correlating two or more items of information relating to different activities by a single party.

In another aspect the invention includes a method of detecting and punishing illegal activity, including: receiving electronic information relating to the activity from an information processor, issuing a fine or other punishment to a party which was involved with the activity, and making a payment to the information processor. Preferably the method includes receiving payment of the fine from the party, and determining the payment to the information processor as a portion of the fine.

The invention also resides in any alternative combination of features which are indicated in this specification. All equivalents of these features are deemed to be included whether or not explicitly set out.

LIST OF FIGURES

Preferred embodiments of the invention will be described with reference to the accompanying drawings, of which:

Figure 1 schematically shows a vehicle with a recording device mounted on or near the front window,

Figures 2A and 2B show how memory in the device may be used to record a field of view,

Figure 3 shows typical electronic components of the device in Figure 1, Figure 4 shows a simple user interface for a navigational unit which may be used in Figure 1 ,

Figure 5 shows a vehicle with multiple recording devices, Figure 6 shows how memory in the devices of Figure 5 may be used to record respective fields of view, Figure 7 shows typical electronic components of a central device in Figure 5, Figure 8 shows a simple user interface for the central device in Figure 5, Figure 9 shows an alternative arrangement of multiple recording devices, Figure 10 shows how memory in the devices of Figure 5 may be used to record fields of view,

Figure 11 shows a simple user interface for a cell phone device when used in

Figure 1 , 5 or 9,

Figures 12A, 12B show a secondary recording device,

Figures 13A, 13B show how a broad field of view may be created by combining two or more smaller fields,

Figure 14A shows how a video clip showing illegal activity may be transferred from an observer to a traffic authority,

Figure 14B shows how a fine paid to the authority in relation to illegal activity may be used to reward to observer, Figure 15 outlines a recording process which may be carried out by one or more of the devices.

DESCRIPTION OF PREFERRED EMBODIMENTS

Referring to the drawings it will be appreciated that the invention may be implemented in a range of different ways and may be useful in a range of different environments other than traffic. It will also be appreciated that the embodiments described here are given by way of example only.

Figure 1 shows a vehicle such as a car with a video recording device mounted on the front window or on the dashboard, operated by the driver or a front passenger. The mounting is adjustable and removable and enables a field of view through the front window. The device is typically either a portable navigational unit or a cell phone, having a built-in camera and a receiver for a satellite positioning system such as GPS. Alternatively the device may be a relatively simple cell phone or a dedicated camera with a wired or wireless connection to a separate navigational unit, for example. In the case of a cell phone, the device could be positioned by the local cellular system rather than through GPS. The device could also be integrated with the dashboard of the vehicle, although a single portable unit which can receive power from an existing source in the vehicle, and be mounted on any window, is likely to be more convenient for most vehicles and users.

Figures 2A and 2B show how memory in the device can be used to record the field of view and allow the user to select portions of the overall recording when required. Multiple portions can be saved for review at a later time if required. The device indicates the available memory space to the user and will generally allow the memory to be expanded, or removed and replaced with fresh capacity. Simultaneous with recording the image the device preferably also records data from a positioning system such as GPS, including position coordinates, date, time and possibly also speed or velocity calculated from the coordinates. Other geographic data such as street names from a GIS can also be recorded to assist analysis later. The information may be recorded in a separate track or as part of some or all of the image frames.

In Figure 2A, the device records a relatively long video sequence as a continuous loop using a relatively large part of the memory. The memory requirements will generally depend on the quality of the image being recorded. If the driver or a passenger decides to save a portion of the sequence the device will copy a corresponding clip into a separate part of the memory, or will flag the portion for copying later, and continue the main recording. In general, the saved portion is an interval which ends at or near the instant a save function is actuated, and which started a predetermined time beforehand. The length of the interval may be determined by a number of key pushes, with two pushes doubling the time or adding a fixed number seconds for example. Portions may be selectable later during a manual review of the overall recording, and portions may also be cut in length during a later review. The lengths of the overall recording and the interval are selectable by the user, depending on the capabilities of the device, and would typically be in the range 5-10 minutes and 10-30 seconds respectively, for example. On the other hand, an entire trip might be stored in some large memory devices. Relatively short times of say 1 minute and 5 seconds could also be used.

A flowchart outlining the process of Figure 2A is provided in Figure 15. The user starts the record loop in the device at a suitable time, such as when the vehicle drives from a parked position. The loop is a temporary store of image data and will generally be overwritten in due course. Some time after start the user actuates a save function and a processor in the device determines which portion of the loop is to be saved. A save action may take place at that time if the processor is capable of doing so without significantly interrupting the loop, or the portion may be flagged for saving later. The device then continues to record the loop, and may make further saves independently of the first. If the device is stopped by the user the current state of the loop is also accessible by the user if required. The user is also able to start a second loop, removed from the first loop, if the user decides to save an as yet undetermined portion of the first loop sometime after the event, say a few minutes. The first loop is thereby saved in memory and can be viewed and edited at a later time to find the required portion. This process would normally be able to be repeated before the memory was filled. Other unrelated items may of course also be stored in the memory and the processor may need to calculate suitable loop sizes for the device to operate effectively.

In Figure 2B, the device records a relatively short video sequence which may be selected and saved in its entirety by the user. The sequence is overwritten after a relatively short time, perhaps a matter of minutes or longer. Two temporary clips are shown in this example, forming a generally continuous loop. The user may select either the current clip or the previous clip. Selecting the current clip causes the recording to end and a new recording to commence. The selected clip is then saved into a separate part of the memory, or at least flagged so as not to be overwritten later. Alternatively the loop can be shifted all or in part to another area of the memory, leaving the saved clip in place. In the absence of a selection the recording stops after a predetermined interval and a new clip commences. The previous clip is eventually overwritten. A recording strategy of this kind could involve a loop with one or more short clips, perhaps up to 10 or more. A visual indicator such as a bar graph may be provided on the device to indicate the length of the current clip and the number of temporary and/or saved clips, for example. Other strategies can be envisaged to provide similar economy of memory space and/or convenience for the user.

In general, it should be possible to operate the device with a minimum of inconvenience to the driver or passenger, and without needing to stop the vehicle or interrupt driving in any significant way. The user can also decide to save a record of an observed event within a reasonable period after the event has taken place. A single push button action by the driver would ideally be sufficient, such as actuating a simple save function after the event. Multiple pushes may be used to control the length of the saved portion of video, or to initiate other functions. Each saved clip can be reviewed and sent to a central collector at a later date, preferably when the vehicle is at rest.

Figure 3 shows the components of a typical device which might be used to implement the methods shown in Figures 1 and 2A, 2B. These components could be found in a portable navigational unit or possibly a cell phone (which would also have additional features related to use in the cellular network). The device includes a processor, camera and memory which together with software to carry out a recording strategy such as indicated above. These communicate through a suitable bus or similar structure. Standard components such as user keys, a screen (eg. a touch screen), GPS receiver, and a power supply are also included. The power supply would normally need to include a wired or wireless connection to a source in the vehicle in order to provide video recording over a lengthy period of time, such as ajourney of several hours. A range of ports are also provided to allow connection to a cell phone for example, for transmission of video clips to the traffic authority, for connection to an additional camera with an additional field of view, or for more conventional connections such as removable memory units and direct uploading/downloading to other devices in general. Other components such as a local wireless transmitter/receiver for a network in the vehicle could also be provided. Figure 4A shows a possible user interface, such as a touch screen, for the device in Figure 3. The screen includes a view area which displays either conventional navigational images, or a view as seen through one or more cameras in the device, or in other connected devices. Touch controls are provided on one side of the screen in this example, including change views, run/stop camera, save current clip, review clip or overall video, and send to cell phone or traffic authority. A wide range of controls can be provided depending on individual features in the particular product.

Figure 4B shows a pair of cameras on the reverse side of the device, located on either side of the window mounting. One or more cameras can be provided in various ways, but ideally each has at least one degree of orientation which is independent of the body of the device. In this example each camera can be tilted inwards or outwards to splay their respective fields on view either side of centre line, and/or rotated about a roughly horizontal axis to raise or lower the field of view. Alternatively a simple camera could be provided in the body of the device, to offset from the window mounting, and oriented with the device in the usual way. The mounting is adapted to reduce vibration, and additional stabilization may or may not be required to produce legible recordings, depending on the road conditions.

Figure 5 shows a vehicle having a larger number of devices for recording traffic on more than one side of the vehicle, perhaps all sides simultaneously. In this example, a main device FT is mounted forwards with secondary devices LT, RT mounted on each side, and device RR mounted facing rearwards. These are typically connected in a local wireless network controlled through the main device FT (such as a navigational unit or cell phone) which is able to display the views as required. A pair of devices mounted front and rear only would of course be a simpler and cheaper system, depending on the driver's level of paranoia or desire to do public good.

Figure 6 shows a preferred recording strategy for the relatively complex system shown in Figure 5. Each secondary device preferably records in a simple scheme such as indicated in Figure 2B above, under control of the main device, which itself may use a different strategy. The user (typically the driver) uses the FT interface to signal a save action in any one of the secondary devices. One or more buttons would be provided for each secondary device. The required clip is then saved and transmitted to the FT device. A save action in any one of the devices could also be accompanied by an outwardly directed flash indicating to outside observers that a clip has been saved (this could be effective with tailgaters, for example). Continuous illumination of an LED is preferable to warn rather than enrage other drivers.

Figure 7 shows how the device FT would need an additional transmitter/receiver to form a local network within the vehicle, based on Bluetooth for example. Still further components are required when FT is a cell phone. Power transfer to each of the secondary devices could be carried out in a wired or wireless fashion, or the devices could be powered by battery and operated intermittently as required by the driver. Being mounted in the window solar power would also be possible.

Figure 8 shows a possible user interface for a device FT such as shown in Figure 6. The screen provides a normal navigational view or may be switched to show any one or all of the various camera views. More complex controls enabling the user to start, stop or save in relation to individual devices are required, but can be implemented in relation to the current view. Alternatively, a single keypush could simply save video from all devices, to be sorted and/or edited later by the user.

Figure 9 shows an alternative arrangement in which each of the devices FT, LT, RT, RR is secondary to a main device CP, typically a cell phone. The devices form a local network within the vehicle, although less than a full set of devices may be used in practice.

Figure 10 shows a recording strategy in relation to Figure 9, in which each device saves the current clip or a previous clip when instructed from the main device CP, and transmits the clip to the main device. Power supply and alternative recording strategies may be implemented as discussed above, for example. Figure 11 shows a user interface for an idealised cell phone. The main screen typically displays output from the camera in the phone, or an in-built navigation system, or may switch to a view from another device in the network (if any). The screen could be partially or wholly a touch screen, or may indicate the functions of keys immediately below the screen as appropriate.

Figures 12A, 12B show a secondary device which could be used in the networks of Figures 5 or 9, for example. The device is window mounted using a suction cup, and carries a camera on a flexible arm, typically with a wide angle lens. Alternatively a more rigid connector between the suction cup and the camera may be required, in which case a solid arm with a universal pivot could be provided. An accelerometer system and software for image stabilisation could also be included. A suction cup which enables an adjustable camera view through the centre of the arrangement is a further but more complex alternative. A minimal number of electronic components enable the device to carry out a recording strategy and communicate with the main device.

Figures 13 A, 13B show how a field of view may be expanded in a particular direction using two or more cameras or devices. In Figure 13A, a single device carries two cameras, such as shown in Figure 4B, each preferably with at least partly independent orientation. In Figure 13B, two nearby devices are located and oriented independently. They could be a navigational unit and a cell phone for example, possibly coupled together. They could be a main device coupled to a secondary device such as shown in Figures 12A, 12B. A wired connection could be practical where the devices are close together and already have suitable ports.

Figures 14A, 14B show how clips from multiple public sources could be processed and used to assist traffic enforcement, or at least awareness of undesirable behaviour. Individual drivers use their navigational units and/or cell phones to record short video clips of other motorists undertaking illegal or risky manoeuvres, or to record an accident which they may have witnessed. These clips are either sent directly to the traffic authority or forwarded through an intermediary which may process the data, or carry some other action on behalf of the authority. The authority assesses each clip, or clips as selected by the intermediary, and may issue an infringement notice to the owner of the vehicle in the clip, in a similar manner as notices are currently issued. Infringement may incur a fine or demerit points for example. Alternatively, the authority or the intermediary may simply issue a warning, and maintain records for future reference. Portions of the fine paid by an infringer may be returned to the intermediary and the creator the clip. Clips are typically sent into the system from a cell phone and the owner of the phone could be awarded call credits for example.

The intermediary acts as a processor of information, analyzing and checking the clips and the related data, and responding to the observers. The intermediary would use software to enhance number plates, or otherwise improve contrast in the images, for example. Many clips taken in dim light or at night, or with strong light sources in the image, such as headlights or the sun, might be unreadable in any case. Authenticity of the images is checked for tampering, and data relating to the location of the event is also checked. The speed of another vehicle might be estimated in some clips, using the velocity of the recording vehicle, and geometrical effects which are apparent from frame to frame due to motion of the other vehicle.

The recording devices may include hardware and/or software which adjusts the parameters of the device as a recording is made, in order to reduce the effect of headlights or the sun for example, or widen the field of view. An accelerometer system or similar electrical-mechanical system with software for image stabilisation would normally be included. The effect of the headlights on a tailgating vehicle could thereby be reduced to enable details of the number plate to be adequately recorded, for example. The devices could also record audio signals from within or near the vehicle.

It will be appreciated that a wide range of variations in the devices and their operation are possible. Devices according to the invention, particularly cell phones, could be mounted on motorcycles or bicycles for example. An awareness among motorists that they could be reported by other road users, with recorded evidence, at any time, rather than only when traffic personnel are on hand, could at least reduce the likelihood of risky behavior and improve road safety. Similarly, the simple action of recording another motorist's poor behaviour might be sufficient to sooth these savage beasts in some cases, avoiding vengeful consequences which might otherwise arise.

An alternative use of the monitoring system could be in raising an argument against infringement, in relation to official camera detectors or a mistaken traffic officer. On realizing that they have been photographed by a road side camera, or are about to be pulled over by an officer, the motorist saves a clip or longer recording of the activity, and may thereby be able to demonstrate, or not, that they are innocent of an alleged offence.

Claims

1. A method of operating a video recording device in a vehicle, including: mounting a camera in the vehicle to provide a field of view outwards through a window, recording the field of view over a substantially continuous period of time as the vehicle moves, selecting a portion of the recorded view while the recording is in progress, and storing the selected portion.

2. A method according to claim 1 wherein the recording is a substantially continuous video clip and the portion is determined by a time interval which ends at the time of the selection.

3. A method according to claim 1 wherein the recording is a substantially continuous series of clips and the portion is one of the clips.

4. A method according to claim 1 further including recording data from a positioning system or GIS with the field of view.

5. A method according to claim 1 wherein the device is a navigational unit and/or a mobile phone mounted on the window.

6. A method of operating a video recording device, including: providing a field of view for a camera in the device, recording the field of view over a substantially continuous period of time, selecting a relatively short portion of the recorded view while the recording is in progress, and storing the selected portion separate from the recording.

7. A method according to claim 6 wherein the portion is selected and stored with a single key push by the user.

8. A handheld navigational unit or a mobile phone which is programmed to enable a method as claimed in any one claims 1-7.

9. A video recording device having a processor, a camera, a memory, and a user interface, wherein the processor is responsive to a user through the interface, to operate the camera to video record a field of view, to select a portion of the recorded view while the recording is in progress, and to store the selected portion.

10. A device according to claim 1 wherein the portion is determined by a time interval which ends at the time of the selection.

11. A device according to claim 9 further having a receiver for a positioning system, with the processor recording position related data related to the video recording.

12. A device according to claim 9 being a mobile phone or portable navigational unit.

13. A method of reporting traffic activity, including: mounting a navigational device inside a vehicle, operating a camera in the device to continuously record video information relating to traffic activity outside the vehicle, selecting one or more portions of the recorded video information, and transmitting one or more of the selected portions to an information processor or a traffic authority.

14. A method according to claim 13 wherein the information includes a video clip and contemporary data from a positioning system.

15. A method according to claim 13 wherein the navigational device is a mobile phone containing a GPS receiver and a GIS.

16. A method of reporting illegal activity, including: receiving electronic information about the activity from an observer, processing the electronic information, forwarding the processed information to an authority, receiving a payment from the authority, and forwarding a part of the payment to the observer.

17. A method according to claim 16 wherein the illegal activity is a traffic infringement and the observer is another driver.

18. A method according to claim 16 wherein the payment from the authority is a portion of a fine incurred by a party to the illegal activity.

19. A method according to claim 16 wherein processing the information includes determining characteristics of objects or people represented in the information.

20. A method of detecting and punishing illegal activity, including: receiving electronic information relating to the activity from an information processor, issuing a fine or other punishment to a party which was involved with the activity, and making a payment to the information processor.