WO2019214642A1 - System and method for guiding autonomous machine - Google Patents

Abstract

A system for guiding an autonomous machine. The system comprises: an autonomous machine, wherein a rolling shutter camera is mounted on the autonomous machine; and an optical communication device, comprising a light source, wherein the light source is configured to operate in at least two modes, and the at least two modes comprise a first mode and a second mode. In the first mode, a light source control signal having a first frequency controls a light attribute emitted by the light source to continuously change at the first frequency, such that stripes are present on an image of the light source acquired by the rolling shutter camera photographing the light source; in the second mode, no stripes or stripes different from the stripes in the first mode are present on an image of the light source acquired by the rolling shutter camera photographing the light source.

Description

System and method for guiding autonomously movable machine Technical field

The present invention relates to guidance for machines capable of autonomous movement, and more particularly to a system and method for guiding autonomously movable machine through an optical communication device.

Background technique

The guidance of machines capable of autonomous movement (for example, drones) is currently implemented by technologies such as GPS and IMU, but the positioning accuracy of these technologies is limited. For example, GPS usually has errors of several meters or even tens of meters, and its signal propagation is affected by the environment in which it is used, often resulting in delay errors. Therefore, current navigation techniques typically only direct the drone to a location near the target location (eg, within a few tens of meters of the target location), but it is difficult to ultimately direct the drone to a very precise target location.

In recent years, many manufacturers are considering using drones for goods distribution. For example, a U.S. patent No. 95,621,216 B1 describes a drone cargo delivery system that navigates a drone based on a GPS and an altimeter and can remotely assist the navigation through the camera of the drone. However, the above system cannot achieve precise navigation of the drone. In another solution that Amazon exposes, the drone is first guided to the destination by GPS, and then the unattended machine looks for a unique "mark" in its field of view, which is a good customer. An electronic identification welcome pad with a predetermined pattern placed at the landing site. If a "tag" is found, the drone will fly through the visual guide to the mark and drop the package. However, the above approach requires the buyer to have a courtyard suitable for receiving the goods and place a unique "mark" in the courtyard. Moreover, since the "mark" itself cannot be used to distinguish between different buyers, if there are multiple "tags" placed by multiple buyers near the destination, the drone cannot determine which one to place the package to. Mark". Therefore, the above scheme is not applicable to people living in urban apartments.

Traditional two-dimensional codes can be used to identify different users, but the recognition distance of the two-dimensional code is very limited. For example, for a two-dimensional code, when scanning with a camera, the camera must typically be placed at a relatively short distance, typically about 15 times the width of the two-dimensional code. For example, for a two-dimensional code having a width of 20 cm, a drone equipped with a camera needs to travel to about 3 meters from the two-dimensional code to recognize the two-dimensional code. Therefore, for long-distance recognition, the two-dimensional code cannot be realized, or a very large two-dimensional code must be customized, but this brings about an increase in cost, and in many cases due to various other restrictions (such as space size limitation) Impossible. Moreover, when the two-dimensional code is recognized, the camera needs to photograph the two-dimensional code substantially, and if the deviation angle is too large, the recognition cannot be performed.

A CMOS imaging device is a widely used imaging device, as shown in FIG. 1, including an array of image sensitive cells (also referred to as image sensors) and some other components. The image sensor array can be a photodiode array, with each image sensor corresponding to one pixel. Each column of image sensors corresponds to a column amplifier, and the output signal of the column amplifier is sent to an A/D converter (ADC) for analog-to-digital conversion and then output through an interface circuit. For any image sensor in the image sensor array, it is now cleared at the beginning of the exposure, and then the signal value is read after waiting for the exposure time. CMOS imaging devices typically employ rolling shutter imaging. In CMOS imaging devices, data readout is serial, so clear/exposure/readout can only be done line-by-line in a pipeline-like manner and will be processed after all rows of the image sensor array have been processed. It is synthesized into one frame of image. Thus, the entire CMOS image sensor array is actually progressively exposed (in some cases CMOS image sensor arrays may also be exposed in multiple lines at a time), which results in small delays between rows. Due to this small delay, when the light source flashes at a certain frequency, some undesired streaks appear on the image taken by the CMOS imaging device, which affects the shooting effect.

It has been found that it is theoretically possible to use the fringes on the image taken by the CMOS imaging device to convey information (similar to a bar code) and to try to pass as much information as possible through the stripes, but this usually requires the CMOS imaging device and the light source to be as far as possible Close, and preferably always at a roughly fixed distance, and also requires fine time synchronization, precise identification of the boundaries of individual stripes, accurate detection of the width of each stripe, etc., thus, in practice its stability and Reliability is not satisfactory and is not widely used. Moreover, this approach obviously does not apply to long-distance identification by a traveling drone.

Summary of the invention

One aspect of the invention relates to a system for guiding a machine capable of autonomous movement, comprising:

Autonomously movable machine having a rolling shutter camera mounted thereon;

An optical communication device comprising a light source configured to be operable in at least two modes, the at least two modes comprising a first mode and a second mode,

And wherein, in the first mode, controlling, by a light source control signal having a first frequency, an attribute of light emitted by the light source to continuously change at a first frequency to capture the light source by the rolling shutter camera a fringe is present on the image of the obtained light source, and in the second mode, no streaks are present or appear on the image of the light source obtained when the light source is photographed by the rolling shutter camera Stripes of different stripes in the first mode.

Preferably, the first mode and the second mode are used to convey different information.

Preferably, in the second mode, controlling, by a light source control signal having a second frequency different from the first frequency, an attribute of light emitted by the light source to continuously change at a second frequency to pass the scrolling The shutter camera does not exhibit streaks on the image of the light source obtained when the light source is photographed, or exhibits a stripe different from the stripe in the first mode.

Preferably, the second frequency is greater than the first frequency.

Preferably, in the second mode, an attribute of light emitted by the light source continuously changes at the first frequency, and an image of the light source obtained when the light source is photographed by the rolling shutter camera Stripes that differ from the stripes in the first mode are presented.

Another aspect of the invention relates to a method of guiding a machine capable of autonomous movement using the above system, comprising:

Collecting information transmitted by a surrounding optical communication device through a rolling shutter camera mounted on the autonomously movable machine, and identifying the transmitted information;

Determining whether the optical communication device is a target optical communication device based on the transmitted information;

If the optical communication device is a target optical communication device, the autonomously movable machine or a portion thereof is controlled to travel to the optical communication device.

Preferably, the above method further comprises: if the optical communication device is not a target optical communication device, then:

Identifying the optical communication device based on information transmitted by the optical communication device, and obtaining a relative positional relationship between the optical communication device and the target optical communication device;

Determining a relative positional relationship between the autonomously movable machine or a portion thereof and the optical communication device;

Determining a relative positional relationship between the target optical communication device and the autonomously movable machine or a portion thereof;

The autonomously moveable machine or portion thereof is directed to the target optical communication device based at least in part on a relative positional relationship between the target optical communication device and the autonomously movable machine or portion thereof.

Preferably, wherein determining a relative positional relationship between the autonomously movable machine or a portion thereof and the optical communication device comprises: determining, by relative positioning, the autonomously movable machine or a portion thereof and the optical communication device Relative positional relationship between them.

Preferably, wherein collecting, by the rolling shutter camera mounted on the autonomously movable machine, information transmitted by a surrounding optical communication device and identifying the transmitted information comprises: obtaining the light by the rolling shutter camera a continuous multi-frame image of the communication device; for each frame image, determining whether a portion of the image corresponding to the position of the light source has stripes or which type of stripes exist; and determining information represented by each frame image .

Preferably, the above method further comprises: first controlling the autonomously movable machine to travel to the vicinity of the target optical communication device.

Preferably, first controlling the autonomously movable machine to travel to the vicinity of the target optical communication device comprises: guiding the autonomously movable machine to the vicinity of the target optical communication device at least partially by a satellite navigation system; and / Or directing the autonomously movable machine to the vicinity of the target optical communication device, at least in part, using a relative positional relationship between the other optical communication device and the target optical communication device.

Preferably, wherein the at least partially utilizing the relative positional relationship between the other optical communication device and the target optical communication device to direct the autonomously movable machine to the vicinity of the target optical communication device comprises:

The autonomously movable machine identifies other optical communication devices while traveling, and obtains a relative positional relationship between the other optical communication devices and the target optical communication device;

Determining a relative positional relationship between the autonomously movable machine and the other optical communication device;

Determining a relative positional relationship between the target optical communication device and the autonomously movable machine;

The autonomously moveable machine is directed to the vicinity of the target optical communication device based at least in part on a relative positional relationship between the target optical communication device and the autonomously movable machine.

Preferably, determining whether the optical communication device is the target optical communication device based on the transmitted information comprises: determining whether the transmitted information includes the predetermined information explicitly or implicitly.

Preferably, wherein the predetermined information is a predetermined identifier or a verification code.

Preferably, determining whether the optical communication device is a target optical communication device based on the transmitted information comprises: determining, by the autonomously movable device, whether the optical communication device is a target optical communication device; or The autonomously moving machine transmits the transmitted information to the server, and the server determines whether the optical communication device is the target optical communication device based on the transmitted information, and transmits the determination result to the autonomously movable machine.

Another aspect of the invention relates to a machine capable of autonomous movement, comprising a rolling shutter camera, a processor and a memory, wherein the memory stores a computer program that can be used to implement when executed by the processor The above method.

Another aspect of the invention relates to a storage medium in which is stored a computer program that, when executed, can be used to implement the method described above.

DRAWINGS

The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

1 is a schematic view of a CMOS imaging device;

2 is a pattern of an image acquired by a CMOS imaging device;

Figure 3 is a light source in accordance with one embodiment of the present invention;

Figure 4 is a light source in accordance with another embodiment of the present invention;

5 is an imaging timing chart of a CMOS imaging device;

6 is another imaging timing diagram of a CMOS imaging device;

Figure 7 shows an image of the CMOS imaging device at different stages when the light source is operating in the first mode;

8 illustrates an imaging timing diagram of a CMOS imaging device when the light source operates in the first mode, in accordance with an embodiment of the present invention;

9 illustrates an imaging timing diagram of a CMOS imaging device when the light source operates in the second mode, in accordance with an embodiment of the present invention;

10 illustrates an imaging timing diagram of a CMOS imaging device when a light source operates in a first mode in accordance with another embodiment of the present invention;

11 shows an imaging timing diagram of a CMOS imaging device for implementing a stripe different from that of FIG. 8 in accordance with another embodiment of the present invention;

Figures 12-13 show two striped images of a light source obtained at different settings;

Figure 14 shows a streak-free image of the obtained light source;

Figure 15 is an image view of an optical tag employing three separate light sources, in accordance with one embodiment of the present invention;

16 is an image view of an optical tag including a positioning mark, in accordance with one embodiment of the present invention;

Figure 17 illustrates an optical tag including a reference light source and two data sources in accordance with one embodiment of the present invention;

FIG. 18 shows an imaging timing chart of the CMOS imaging device for the optical tag shown in FIG. 17;

Figure 19 illustrates a method of UAV guidance by optical tags in accordance with one embodiment of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings.

One embodiment of the present invention is directed to an optical communication device capable of transmitting different information by emitting different lights. The optical communication device is also referred to herein as a "light tag" and both are used interchangeably throughout this application. The optical communication device includes a light source and a controller capable of controlling the light source to operate in two or more modes by a light source control signal, the two or more modes including the first mode and the second mode, Wherein, in the first mode, the light source control signal has a first frequency such that an attribute of light emitted by the light source continuously changes at a first frequency to deliver first information, and in the second mode, The property of the light emitted by the light source continues to change at the second frequency or does not change to deliver second information that is different from the first information.

The attribute of light in this application refers to any property that the CMOS imaging device can recognize, for example, it may be an attribute that the human eye can perceive, such as the intensity, color, and wavelength of light, or other attributes that are not perceptible to the human eye. For example, the intensity, color or wavelength of the electromagnetic wavelength outside the visible range of the human eye changes, or any combination of the above properties. Thus, a change in the properties of light can be a single property change, or a combination of two or more properties can change. When the intensity of the light is selected as an attribute, it can be achieved simply by selecting to turn the light source on or off. In the following, for the sake of simplicity, the light source is turned on or off to change the properties of the light, but those skilled in the art will appreciate that other ways to change the properties of the light are also possible. It should be noted that the attribute of the light varying at the first frequency in the first mode may be the same as or different from the attribute of the light changing at the second frequency in the second mode. Preferably, the properties of the light that change in the first mode and the second mode are the same.

When the light source operates in the first mode or the second mode, the light source can be imaged using a rolling shutter imaging device such as a CMOS imaging device or a device having a CMOS imaging device (eg, a cell phone, a tablet, smart glasses, etc.), ie , imaging by rolling the shutter. Hereinafter, a mobile phone as a CMOS imaging device will be described as an example, as shown in FIG. 2 . The line scanning direction of the mobile phone is shown as a vertical direction in FIG. 2, but those skilled in the art can understand that the line scanning direction can also be a horizontal direction depending on the underlying hardware configuration.

The light source can be a light source of various forms as long as one of its properties that can be perceived by the CMOS imaging device can be varied at different frequencies. For example, the light source may be an LED light, an array of a plurality of LED lights, a display screen or a part thereof, and even an illuminated area of light (for example, an illuminated area of light on a wall) may also serve as a light source. The shape of the light source may be various shapes such as a circle, a square, a rectangle, a strip, an L shape, a cross shape, a spherical shape, or the like. Various common optical devices can be included in the light source, such as a light guide plate, a soft plate, a diffuser, and the like. In a preferred embodiment, the light source may be a two-dimensional array of a plurality of LED lamps, one dimension of which is longer than the other dimension, preferably a ratio of between about 6-12:1. For example, the LED light array can be composed of a plurality of LED lamps arranged in a row. The LED light array can be rendered as a substantially rectangular light source when illuminated, and the operation of the light source is controlled by a controller.

Figure 3 illustrates a light source in accordance with one embodiment of the present invention. When imaging the light source shown in FIG. 3 using a CMOS imaging device, it is preferable to make the long side of the light source shown in FIG. 3 perpendicular to the row direction of the CMOS imaging device (for example, the line scanning direction of the mobile phone shown in FIG. 2) Or roughly vertical, to image as many stripes as possible under otherwise identical conditions. However, sometimes the user does not understand the line scanning direction of the mobile phone. In order to ensure that the mobile phone can recognize in various postures, and the maximum recognition distance can be achieved under both the vertical screen and the horizontal screen, the light source can be a plurality of rectangular shapes. Combination, for example, an L-shaped light source as shown in FIG.

In another embodiment, the light source may not be limited to a planar light source, but may be implemented as a stereoscopic light source, for example, a strip-shaped cylindrical light source, a cubic light source, or the like. The light source can be placed, for example, on a square, suspended at a substantially central location of an indoor venue (eg, a restaurant, a conference room, etc.) so that a nearby user in each direction can capture the light source through the mobile phone, thereby obtaining the light source. Information.

FIG. 5 shows an imaging timing diagram of a CMOS imaging device, each of which corresponds to a row of sensors of the CMOS imaging device. When imaging is performed on each line of the CMOS imaging sensor array, two stages are mainly involved, namely, exposure time and readout time. The exposure time of each line may overlap, but the readout time does not overlap.

It should be noted that only a small number of rows are schematically illustrated in FIG. 5, and in actual CMOS imaging devices, depending on the resolution, there are typically thousands of rows of sensors. For example, for 1080p resolution, it has 1920 x 1080 pixels, the number 1080 indicates 1080 scan lines, and the number 1920 indicates 1920 pixels per line. For 1080p resolution, the read time per line is approximately 8.7 microseconds (ie, 8.7 x 10 ^-6 seconds).

If the exposure time is too long, resulting in a large overlap of exposure time between adjacent lines, it may appear as a significant transitional fringe when imaging, for example, multiple strips between pure black pixel rows and pure white pixel rows have different grays. The pixel row of degrees. The present invention is expected to be able to present pixel lines as sharp as possible. For this reason, the exposure time of a CMOS imaging device (such as a mobile phone) can be set or adjusted (for example, set or adjusted by an APP installed on a mobile phone) to select a relative Short exposure time. In a preferred embodiment, the exposure time can be made approximately equal to or less than the readout time of each row. Taking the 1080p resolution as an example, the readout time of each line is approximately 8.7 microseconds. In this case, it is conceivable to adjust the exposure time of the mobile phone to about 8.7 microseconds or less. Fig. 6 shows an imaging timing chart of the CMOS imaging device in this case. In this case, the exposure time of each line does not substantially overlap, or the number of overlapping portions is small, so that stripes having relatively clear boundaries can be obtained at the time of imaging, which is more easily recognized. It should be noted that FIG. 6 is only a preferred embodiment of the present invention, and a longer (for example, twice or three times, four times or four times the readout time of each row, etc.) or a shorter exposure time is also feasible. of. For example, in the imaging process of the striped image shown in Figs. 12 and 13 of the present application, the readout time per line is approximately 8.7 microseconds, and the exposure time per line set is 14 microseconds. Further, in order to exhibit streaks, the length of one cycle of the light source may be set to about twice or more of the exposure time, and preferably may be set to about four times or more of the exposure time.

Figure 7 is a view showing an image of a CMOS imaging device at different stages when the light source is operated in the first mode using a controller, in which the property of the light emitted by the light source is changed at a certain frequency, in this example Medium to turn the light source on and off.

The upper part of Fig. 7 shows a state change diagram of the light source at different stages, and the lower part shows an image of the light source on the CMOS imaging device at different stages, wherein the row direction of the CMOS imaging device is vertical and from the left Scan to the right. Since the image captured by the CMOS imaging device is progressively scanned, when the high-frequency flicker signal is captured, the portion of the obtained image on the image corresponding to the imaging position of the light source forms a stripe as shown in the lower part of FIG. 7, specifically, In time period 1, the light source is turned on, in which the scanning line of the leftmost portion of the exposure exhibits bright streaks; in time period 2, the light source is turned off, in which the scanned lines of the exposure exhibit dark stripes; in time period 3, the light source is turned on, The scanned lines exposed during this time period exhibit bright streaks; in time period 4, the light source is turned off, during which the scanned lines of exposure exhibit dark stripes.

The frequency of the flashing of the light source can be set by the light source control signal, or the length of the light strip can be adjusted each time the light source is turned on and off, and the longer opening or closing time generally corresponds to a wider stripe. For example, for the case shown in FIG. 6, if the light source is turned on and off each time, the exposure time is set to be substantially equal to the exposure time of each line of the CMOS imaging device (this exposure time can be set by the APP installed on the mobile phone or manually set. ), it is possible to present stripes with a width of only one pixel when imaging. In order to enable long-distance identification of optical tags, the narrower the stripes, the better. However, in practice, the stripe having a width of only one pixel may be less stable or less recognizable due to light interference, synchronization, etc., therefore, in order to improve the stability of recognition, it is preferable to realize a stripe having a width of two pixels. . For example, for the case shown in FIG. 6, the stripe having a width of about two pixels can be realized by setting the duration of each turn on or off of the light source to be approximately equal to about twice the exposure time of each line of the CMOS imaging device. Specifically, as shown in FIG. 8, wherein the signal of the upper portion of FIG. 8 is a light source control signal, the high level corresponds to the turn-on of the light source, and the low level corresponds to the turn-off of the light source. In the embodiment shown in FIG. 8, the duty ratio of the light source control signal is set to about 50%, and the exposure duration of each line is set to be substantially equal to the readout time of each line, but those skilled in the art can understand that other Settings are also possible, as long as they can show distinguishable stripes. For simplicity of description, the synchronization between the light source and the CMOS imaging device is used in FIG. 8 such that the time of turning on and off of the light source substantially corresponds to the start or end time of the exposure time of a certain line of the CMOS imaging device, but the field The skilled person will understand that even if the two are not synchronized as shown in Fig. 8, they can exhibit significant streaks on the CMOS imaging device. At this time, there may be some transition stripes, but there must be a line of exposure when the light source is always off ( That is, the darkest stripe) is the line that is exposed when the light source is always on (ie, the brightest stripe), which is separated by one pixel. The light and dark variations (i.e., fringes) of such pixel rows can be easily detected (e.g., by comparing the brightness or grayscale of some of the pixels in the imaged area of the source). Furthermore, even if there is no line (ie, the darkest stripe) that is exposed when the light source is always off and a line that is exposed when the light source is always on (ie, the brightest stripe), if there is a light-on portion t1 less than a certain time during the exposure time a line having a small proportion of the length of the entire exposure time (ie, a darker stripe), and a line in which the light source opening portion t2 is greater than a certain length of time or a proportion of the entire exposure time (ie, a brighter stripe) during the exposure time, and T2-t1> light and dark stripe difference threshold (for example, 10 microseconds), or t2/t1> light and dark stripe ratio threshold (for example, 2), and the brightness change between these pixel lines can also be detected. The light and dark stripe difference threshold and the ratio threshold are related to the optical label illumination intensity, the photosensitive device property, the shooting distance, and the like. Those skilled in the art will appreciate that other thresholds are also possible as long as computer-resolvable stripes are present. When the streaks are identified, the information conveyed by the light source at this time, such as binary data 0 or data 1, can be determined.

The stripe recognition method according to an embodiment of the present invention is as follows: obtaining an image of the optical label, and dividing the imaging area of the light source by means of projection; collecting stripe in different configurations (for example, different distances, different light source flicker frequencies, etc.) Images and unstripe pictures; normalize all collected pictures to a specific size, such as 64*16 pixels; extract each pixel feature as input feature, build a machine learning classifier; perform two-class discrimination to determine a striped picture Still a non-striped picture. For stripe recognition, one of ordinary skill in the art can also process by any other method known in the art, which will not be described in detail.

For a strip light source with a length of 5 cm, when using a mobile phone that is currently on the market, setting the resolution to 1080p, when shooting 10 meters away (that is, the distance is 200 times the length of the light source), The strip light source occupies about 6 pixels in its length direction, and if each stripe width is 2 pixels, it will appear in a range of widths of a plurality of apparent pixels within the width of the 6 pixels. At least one distinct stripe that can be easily identified. If a higher resolution is set, or optical zoom is used, the stripe can be recognized at a greater distance, for example, when the distance is 300 or 400 times the length of the light source.

The controller can also operate the light source in the second mode. In one embodiment, in the second mode, the light source control signal can have another frequency than the first mode to change the properties of the light emitted by the light source, such as turning the light source on and off. In one embodiment, the controller can increase the turn-on and turn-off frequencies of the light source in the second mode compared to the first mode. For example, the frequency of the first mode may be greater than or equal to 8000 times/second, and the frequency of the second mode may be greater than the frequency of the first mode. For the situation illustrated in Figure 6, the light source can be configured to turn the light source on and off at least once during the exposure time of each row of the CMOS imaging device. FIG. 9 shows a case where the light source is turned on and off only once during the exposure time of each line, wherein the signal of the upper portion of FIG. 9 is a light source control signal whose high level corresponds to the turn-on of the light source, and the low level corresponds to the light source. The light source is turned off. Since the light source is turned on and off in the same way during the exposure time of each line, the exposure intensity energy obtained at each exposure time is roughly equal, so there is no significant difference in brightness between the individual pixel rows of the final image of the light source. So there are no stripes. Those skilled in the art will appreciate that higher turn-on and turn-off frequencies are also possible. In addition, for simplicity of description, synchronization between the light source and the CMOS imaging device is used in FIG. 9 such that the turn-on time of the light source substantially corresponds to the start time of the exposure time of a certain line of the CMOS imaging device, but those skilled in the art can It is understood that even if the two are not synchronized as in Fig. 9, there is no significant difference in brightness between the respective pixel rows of the final image of the light source, so that no streaks exist. When the streaks are not recognized, the information conveyed by the light source at this time, such as binary data 1 or data 0, can be determined. For the human eye, the light source of the present invention does not perceive any flicker when operating in the first mode or the second mode described above. In addition, in order to avoid the flicker phenomenon that may be perceived by the human eye when switching between the first mode and the second mode, the duty ratios of the first mode and the second mode may be set to be substantially equal, thereby realizing in different modes. The roughly the same luminous flux.

In another embodiment, in the second mode, direct current may be supplied to the light source such that the light source emits light whose properties do not substantially change, thereby obtaining one of the light sources obtained when the light source is photographed by the CMOS image sensor. No streaks appear on the frame image. In addition, in this case, substantially the same luminous flux in different modes can also be achieved to avoid flicker that may be perceived by the human eye when switching between the first mode and the second mode.

Figure 8 above describes an embodiment in which stripes are rendered by varying the intensity of the light emitted by the source (e.g., by turning the light source on or off). In another embodiment, as shown in Figure 10, The wavelength or color of the light emitted by the light source changes to present stripes. In the embodiment shown in Fig. 10, the light source includes a red light that emits red light and a blue light that emits blue light. The two signals in the upper portion of FIG. 10 are a red light control signal and a blue light control signal, respectively, wherein a high level corresponds to the turn-on of the corresponding light source and a low level corresponds to the turn-off of the corresponding light source. The red light control signal and the blue light control signal are phase shifted by 180°, that is, the two levels are opposite. The red light control signal and the blue light control signal enable the light source to alternately emit red light and blue light outward, so that when the light source is imaged by the CMOS imaging device, red and blue stripes can be presented.

By determining whether or not there is a streak on a portion of the image of one frame taken by the CMOS imaging device corresponding to the light source, information transmitted by each frame of the image, such as binary data 1 or data 0, can be determined. Further, by taking a continuous multi-frame image of the light source by the CMOS imaging device, a sequence of information composed of binary data 1 and 0 can be determined, and information transmission of the light source to the CMOS imaging device (for example, a mobile phone) is realized. In one embodiment, when a continuous multi-frame image of the light source is captured by the CMOS imaging device, control may be performed by the controller such that the switching time interval between the operating modes of the light source is equal to the length of time that a complete frame of the CMOS imaging device is imaged Thereby, the frame synchronization of the light source and the imaging device is realized, that is, information of 1 bit is transmitted per frame. For a shooting speed of 30 frames per second, 30 bits of information can be transmitted per second, and the encoding space reaches 2 ^30. The information can include, for example, a start frame mark (frame header), an optical tag ID, a password, a verification code. , URL information, address information, timestamps or different combinations thereof, and so on. The order relationship of the above various kinds of information can be set in accordance with a structuring method to form a packet structure. Each time a complete packet structure is received, it is considered to obtain a complete set of data (a packet), which can be read and verified. The following table shows the packet structure in accordance with one embodiment of the present invention:

Frame header Attribute (8bit) Data bit (32bit) Check digit (8bit) End of frame

In the above description, the information transmitted by the frame image is determined by judging whether or not there is a streak at the imaging position of the light source in each frame of the image. In other embodiments, different information conveyed by the frame image may be determined by identifying different fringes at the imaging location of the light source in each frame of image. For example, in the first mode, the property of the light emitted by the light source is controlled by the light source control signal having the first frequency to continuously change at the first frequency, thereby being capable of being imaged on the light source obtained when the light source is photographed by the CMOS image sensor. Presenting a first stripe; in the second mode, controlling the property of the light emitted by the light source by the light source control signal having the second frequency to continuously change at the second frequency, thereby being obtainable when the light source is photographed by the CMOS image sensor A second stripe different from the first stripe is present on the image of the light source. The difference in stripes may be based, for example, on different widths, colors, brightnesses, etc., or any combination thereof, as long as the difference can be identified.

In one embodiment, stripes of different widths may be implemented based on different light source control signal frequencies. For example, in the first mode, the light source may operate as shown in FIG. 8 to achieve a width of approximately two pixels. a stripe; in the second mode, the durations of the high level and the low level in each period of the light source control signal in FIG. 8 can be respectively changed to twice the original, as shown in FIG. A second stripe with a width of approximately four pixels is implemented.

In another embodiment, stripes of different colors may be implemented. For example, the light source may be set to include a red light that emits red light and a blue light that emits blue light. In the first mode, the blue light may be turned off. And the red lamp works as shown in Fig. 8 to realize red and black stripes; in the second mode, the red lamp can be turned off, and the blue lamp works as shown in Fig. 8, thereby realizing blue and black stripes. . In the above embodiment, the red and black stripes and the blue and black stripes are realized using the light source control signals having the same frequency in the first mode and the second mode, but it is understood that the first mode and the second mode may also be used. Light source control signals at different frequencies.

In addition, those skilled in the art can understand that more than two kinds of information can be further represented by implementing more than two kinds of stripes. For example, in an embodiment including the red light and the blue light in the above light source, the third mode can be further set. In this third mode, the red and blue lights are controlled in the manner shown in Figure 10 to achieve a red-blue stripe, a third type of information. Obviously, alternatively, another type of information, that is, the fourth type of information, can be further transmitted through the fourth mode in which the stripes are not presented. The above four modes can be arbitrarily selected for information transmission, and other modes can be further combined as long as different patterns generate different stripe patterns.

Figure 12 shows the use of 1080p resolution imaging for LEDs that are flashing at a frequency of 16,000 times per second (each period has a duration of 62.5 microseconds with an on duration and a closure duration of approximately 31.25 microseconds each) The stripe on the image obtained by the experiment, with the exposure time of each line set to 14 microseconds. As can be seen from Figure 12, stripes of approximately 2-3 pixel width are presented. Figure 13 shows that the blinking frequency of the LED lamp in Figure 12 is adjusted to 8000 times per second (the duration of each cycle is 125 microseconds, wherein the opening duration and the closing duration are each about 62.5 microseconds), under other conditions. Streaks on the image obtained by experiment under constant conditions. As can be seen from Figure 13, a stripe of approximately 5-6 pixel width is presented. Figure 14 shows the adjustment of the blinking frequency of the LED lamp of Figure 12 to 64,000 times per second (the duration of each cycle is 15.6 microseconds, wherein the opening duration and the closing duration are each about 7.8 microseconds), under other conditions. The image obtained by experiment without change has no streaks on it, because the length of each line of exposure is 14 microseconds, which basically covers one opening time and one closing time of the LED lamp.

In the above, for convenience of description, the square wave is taken as an example to describe the light source control signal having the corresponding frequency, but those skilled in the art can understand that the light source control signal can also use other waveforms such as a sine wave, a triangular wave or the like.

The use of one light source is described above, and in some embodiments, two or more light sources may also be employed. The controller can independently control the operation of each light source. Figure 15 is an image view of an optical tag employing three independent light sources in which the imaging positions of the two light sources have streaks, and the imaging position of one light source has no streaks, according to one embodiment of the present invention, One frame of image can be used to convey information, such as binary data 110.

In one embodiment, the optical tag may further include one or more positioning indicators located adjacent to the information delivery source, the positioning identification being, for example, a lamp of a particular shape or color, which may remain bright, for example, during operation. The location identification can help a user of a CMOS imaging device, such as a cell phone, to easily discover optical tags. In addition, when the CMOS imaging device is set to a mode in which the optical tag is photographed, the imaging of the positioning mark is conspicuous and easy to recognize. Thus, the one or more location markers disposed adjacent the information transfer light source can also assist the handset in quickly determining the location of the information transfer light source to facilitate identifying whether the imaged region corresponding to the information transfer light source has streaks. In one embodiment, in identifying whether there are streaks, the location identification may first be identified in the image such that the approximate location of the optical tag is found in the image. After the location identification is identified, one or more regions in the image may be determined based on the relative positional relationship between the location identification and the information delivery light source, the region encompassing an imaging location of the information delivery light source. These areas can then be identified to determine if there are streaks or what stripes are present. 16 is an image diagram of an optical tag including a positioning mark including three horizontally disposed information transfer light sources, and two vertically positioned two positional identification lights located on both sides of the information transfer light source, in accordance with an embodiment of the present invention. .

In one embodiment, an ambient light detection circuit can be included in the optical tag that can be used to detect the intensity of ambient light. The controller can adjust the intensity of the light emitted by the light source when it is turned on based on the intensity of the detected ambient light. For example, when the ambient light is relatively strong (for example, during the day), the intensity of the light emitted by the light source is relatively large, and when the ambient light is relatively weak (for example, at night), the intensity of the light emitted by the light source is relatively small.

In one embodiment, an ambient light detection circuit can be included in the optical tag that can be used to detect the frequency of ambient light. The controller can adjust the frequency of the light emitted by the light source when it is turned on based on the frequency of the detected ambient light. For example, when the ambient light has the same frequency flashing light source, the light emitted by the light source is switched to another unoccupied frequency.

In the actual application environment, if there is a large amount of noise, or when the recognition distance is very far, the accuracy of the recognition may be affected. Therefore, in order to improve the accuracy of the recognition, in one embodiment of the present invention, in addition to the above-described light source for transmitting information (for clarity, hereinafter referred to as "data source" for clarity), It may also include at least one reference light source. The reference source itself is not used to convey information, but rather to aid in identifying the information conveyed by the data source. The reference source can be physically similar to the data source, but operates in a predetermined mode of operation, which can be one or more of various modes of operation of the data source. In this way, the decoding of the data source can be converted to a calculation that matches (eg, correlates) the image of the reference source, thereby improving the accuracy of the decoding.

17 shows an optical tag including a reference light source and two data light sources, wherein three light sources are arranged side by side, the first light source serves as a reference light source, and the other two light sources respectively serve as a first embodiment. A data source and a second data source. It should be noted that the number of reference light sources in the optical label may be one or more, and is not limited to one; likewise, the number of data light sources may be one or more, and is not limited to two. In addition, because the reference source is used to provide auxiliary recognition, its shape and size do not have to be the same as the data source. For example, in one embodiment, the length of the reference source can be half of the data source.

In one embodiment, each of the first data source and the second data source shown in FIG. 17 is configured to operate in three modes to, for example, display a streak-free image, respectively having a stripe width of 2 pixels. Image, image with a stripe width of 4 pixels. The reference light source may be configured to always operate in one of three modes to display one of the three images described above, or alternately operate in different modes to alternately display any two or all of the above three images in different frames, This provides a baseline or reference for image recognition of the data source. For example, the reference light source alternately displays an image with a stripe width of 2 pixels and an image with a stripe width of 4 pixels in different frames. The image of the data source in each frame may be compared with the current frame and an adjacent frame (for example, before An image of a reference light source in a frame or a subsequent frame (the images must include an image with a stripe width of 2 pixels and an image with a stripe width of 4 pixels) to determine the type of the image; or It is also possible to collect successive multi-frame images of the reference light source in one time period, and to average the features of each group of images by taking the image of the odd frame number and the image of the even frame number as a group respectively (for example, finding each group) The average of the stripe width of the image), and which set of images according to the stripe width corresponds to an image with a stripe width of 2 pixels or an image with a stripe width of 4 pixels, thereby obtaining an image with a stripe width of 2 pixels. The average feature and the average width of the image with a stripe width of 4 pixels, after which it can be determined whether the image of the data source in each frame conforms to one of these average features.

Since the reference light source is located at substantially the same position as the data light source and is subjected to the same ambient lighting conditions, interference, noise, etc., it can provide one or more reference images or reference images for image recognition in real time, thereby improving The accuracy and stability of the identification of information conveyed by the data source. For example, the data pattern can be accurately identified by comparing the imaging of the data source with the imaging of the reference source to identify the data it is transmitting.

Further, according to the imaging principle of CMOS, when a plurality of light sources change their properties at the same frequency but different phases, a stripe pattern of the same width but different phases is generated, and the stripe pattern of the same width but different phases can be matched using a matching method. Accurate judgment. In one embodiment, the reference light source can be controlled to operate in a predetermined mode of operation in which, for example, a stripe of 4 pixels in width is present on the image of the reference source. At this time, if the data source is simultaneously controlled to operate in the operation mode, and the phase of the data source is consistent with the reference source, the stripes appearing on the image of the data source are similar to the stripes appearing on the image of the reference source (eg , the width is also 4 pixels) and there is no phase difference; if the data source is controlled to operate in the working mode at the same time, but the phase of the data source and the reference source are inconsistent (for example, inverted or 180° out of phase), the data source The stripes appearing on the image are similar to the stripes appearing on the image of the reference source (eg, the width is also 4 pixels) but there is a phase difference.

FIG. 18 shows an imaging timing chart of the CMOS imaging device for the optical tag shown in FIG. The respective control signals of the reference source, the first data source, and the second data source are shown in the upper portion of FIG. 18, wherein a high level may correspond to the turn-on of the light source and a low level may correspond to the turn-off of the light source. As shown in FIG. 18, the three control signals have the same frequency, and the first data source control signal is in phase with the reference source control signal, and the second data source control signal is 180 degrees out of phase with the reference source control signal. In this way, when the optical tag is imaged using a CMOS imaging device, the reference light source, the first data source, and the second data source will each exhibit a stripe having a width of approximately four pixels, but the first data source and The fringe phase on the imaging of the reference source is uniform (eg, the line of the reference light source's bright stripe coincides with the line of the first data source's bright stripe, the line of the reference source's dark stripe and the first data source The lines of dark stripes are consistent, and the phase of the fringes of the second data source and the reference source are inverted (eg, the line where the light stripe of the reference source is located and the dark strip of the second data source) The lines are identical, and the line of the dark stripe of the reference source is identical to the line of the bright stripe of the second data source).

By providing a reference source and phase control of the data source, the amount of information that the data source can deliver at a time can be further improved with improved recognition capabilities. For the optical tag shown in FIG. 17, if the first data source and the second data source are configured to operate in the first mode and the second mode, wherein no stripes are present in the first mode and presented in the second mode stripe. If no reference source is provided, each data source can deliver one of two types of data, such as 0 or 1, in one frame of image. The second mode itself can be used to deliver more than one type of data by providing a reference source and operating it in the second mode and further providing phase control when the data source is operating in the second mode. Taking the method shown in Fig. 18 as an example, the second mode combined with the phase control itself can be used to deliver one of the two kinds of data, so that each data source can deliver one of the three kinds of data in one frame of image.

The above manner enables the phase control of the data source by introducing a reference light source, so that the coding density of the data source of the optical tag can be improved, and the coding density of the entire optical tag can be increased accordingly. For example, for the embodiment described above, if a reference light source is not employed (ie, the reference light source is used as the third data light source), each data light source can deliver one of two kinds of data in one frame of image, thus the entire light label (contains three data sources) can be one of ^three species of transmitting data in an image composition; and if the reference light source, each data source can be one of three data transfer in an image, and thus the entire optical tag (contains two data sources) can pass one of the three combinations of ^two kinds of data in an image. This effect is more pronounced if the number of data sources in the optical tag is increased, for example, for the embodiment described above, if an optical tag containing five sources is used, the entire tag is used without the reference source ( Contains five data sources) One of ²⁵ combinations of data can be passed in one frame; and when one of the sources is selected as the reference source, the entire optical tag (including four data sources) can be in one frame. Pass one of 3 ⁴ data combinations. Similarly, by increasing the number of reference sources in the optical tag, the encoding density of the entire optical tag can be further increased. Some experimental data for matching calculations (eg, correlation calculations) of the image of the data source and the image of the reference source are provided below. Among them, the meaning of the calculation results is defined as follows:

0.00～±0.30 micro correlation

±0.30～±0.50 real correlation

±0.50 to ±0.80 significant correlation

±0.80～±1.00 highly correlated

Among them, positive values represent positive correlation and negative values represent negative correlation. If the frequency and phase of the data source and the reference source are the same, then in an ideal state, the images of the two sources are exactly the same, so the result of the correlation calculation is +1, indicating a complete positive correlation. If the frequency of the data source and the reference source are the same, but the phases are opposite, then under ideal conditions, the image of the two sources has the same stripe width, but the position of the bright and dark stripes is exactly opposite, so the result of the correlation calculation is -1 , indicating complete negative correlation. It can be understood that in the actual imaging process, it is difficult to obtain a completely positively correlated and completely negatively correlated image due to the presence of interference, errors, and the like. If the data source and the reference source operate in different modes of operation to display stripes of different widths, or one of them does not display stripes, the images of the two are typically micro-correlated.

Tables 1 and 2 below show the correlation calculation results when the data source and the reference source are of the same frequency at the same frequency, and the correlation calculation results when the data source and the reference source are in the opposite phase of the same frequency. For each case, five images were taken, and the reference light source image in each frame image was correlated with the data source image in the frame image.

Data image 1 Data image 2 Data image 3 Data image 4 Data image 5 Reference image 1 0.7710 Reference image 2 0.7862 Reference image 3 0.7632 Reference image 4 0.7883 Reference image 5 0.7967

Table 1 - Correlation calculation results for the same phase at the same frequency

Data image 1 Data image 2 Data image 3 Data image 4 Data image 5 Reference image 1 -0.7849 Reference image 2 -0.7786 Reference image 3 -0.7509 Reference image 4 -0.7896 Reference image 5 -0.7647

Table 2 - Correlation calculation results for the opposite phase of the same frequency

As can be seen from the above table, when the data source and the reference source use the same frequency at the same frequency, the correlation calculation results can indicate that they are significantly positively correlated. When the data source and the reference source are in opposite phases of the same frequency, the correlation calculations can indicate that they are significantly negatively correlated.

Compared with the recognition distance of about 15 times of the two-dimensional code in the prior art, the identification distance of at least 200 times of the optical label of the present invention has obvious advantages. The long-distance recognition capability is especially suitable for outdoor recognition. Taking a recognition distance of 200 times as an example, for a light source with a length of 50 cm set on a street, a person within 100 meters from the light source can pass the mobile phone and the light source. Interact. In addition, the solution of the present invention does not require the CMOS imaging device to be located at a fixed distance from the optical tag, nor does it require time synchronization between the CMOS imaging device and the optical tag, and does not require accurate detection of the boundaries and widths of the individual stripes. Therefore, it has extremely strong stability and reliability in actual information transmission. In addition, the solution of the present invention also does not require that the CMOS imaging device must be substantially aligned with the optical tag for identification, especially for optical tags having strip, spherical sources. For example, for a strip or columnar light tag placed on a square, a CMOS imaging device within 360° of it can be identified. If the strip or columnar light label is placed on a wall, it can be identified by a CMOS imaging device that is approximately 180° around it. For a spherical optical tag placed on a square, it can be identified by a CMOS imaging device at any location in its three-dimensional space.

Due to the aforementioned advantages of the optical tag of the present invention, it can be used to achieve precise guidance within the field of view, for example, guidance of machines capable of autonomous movement. One embodiment of the present invention is directed to a system for guiding autonomously movable machine through an optical tag, comprising a machine capable of autonomous movement and a light tag as described in any of the above embodiments. The autonomously movable machine is equipped with a CMOS camera capable of collecting and identifying information transmitted by the optical tag.

The following describes an example of a drone delivery application for online shopping. Buyers can use their apartment as the shipping address and fill in the shipping address information in the online shopping platform, such as some of the following information: geographic location information, cell information, building number, floor, and so on. Buyers can place a light tag at the apartment (such as the balcony, exterior wall, etc. of the apartment) as the target light tag for the drone to deliver the goods. After the buyer completes the shopping through the network, the optical tag may be configured to deliver the predetermined information by continuously working in different modes, the predetermined information may be, for example, the ID information of the optical tag itself, the ID of the buyer on the online shopping platform. The information, the verification code received by the buyer from the platform after shopping on the online shopping platform, and the like, as long as the predetermined information is known to the online shopping platform and can be used to identify the buyer or the goods it purchases. The online shopping platform can transmit the predetermined information to the drone.

The method for guiding the drone through the optical tag of the present invention can be as shown in FIG. 19, which includes the following steps:

Step 101: Control the drone to travel to the vicinity of the target light tag.

After the drone picks up the goods to be sent to the buyer, it can first fly to the buyer's shipping address (ie, the buyer's apartment). In one embodiment, the delivery address may preferably be geographic location information of the target optical tag itself (eg, precise latitude and longitude, height information, etc. of the optical tag), and may also include other information, such as the target optical tag. Orientation information, etc.

Step 101 can be implemented in various possible existing ways in the art. For example, the drone can fly to the vicinity of the shipping address (ie, near the buyer's light tag) by means of GPS navigation or the like. The existing GPS navigation method can reach the precision range of several tens of meters, and the optical tag of the present invention can realize the recognition distance of at least 200 times, taking the recognition distance of 200 times as an example, for the light source with a length of 20 cm, the drone Identification can be achieved as long as it can fly to within 40 meters of the source. In step 101, the drone can also be guided to the vicinity of the target optical tag by using the relative positional relationship between the other optical tags and the target optical tag. The relative positional relationship between the individual optical tags can be stored, for example, in advance and can be obtained by the drone. When the drone is flying, it can identify other optical tags along its flight path and obtain the relative positional relationship between the other optical tags and the target optical tags. Then, the drone can be relatively positioned (also called reverse Positioning) to determine the relative positional relationship between the target optical tag and the other optical tag, so that the relative positional relationship between the target optical tag and the drone can be determined. Based on the relative positional relationship, the drone can be guided to the vicinity of the target light tag. Those skilled in the art will appreciate that it is also possible to direct the drone to the vicinity of the target light tag using a combination of the various means described above.

Various relative positioning methods known in the art can be used to determine the relative positional relationship of the drone to the optical tag. In one embodiment, the drone can use its imaging device to image the optical tag, and obtain the relative distance from the optical tag based on the acquired image (the larger the imaging, the closer the distance; the smaller the imaging, the farther the distance And the current orientation information of the drone can be obtained by the built-in sensor, and the relative direction of the drone and the optical tag is obtained based on the orientation information (preferably, the position of the optical tag in the image can be further combined to more accurately The relative direction of the drone and the optical tag is determined), so that the relative positional relationship between the drone and the optical tag can be obtained based on the relative distance and the relative direction of the drape. In addition, many imaging devices currently on the market are usually equipped with a binocular camera or a depth camera, and the image tag is collected by an imaging device equipped with a binocular camera or a depth camera, and the imaging device and the optical tag can also be easily obtained. The relative distance between them. In another embodiment, in order to determine the relative direction of the user and the optical label, the orientation information of the optical label may be stored in the server. After the user identifies the identification information of the optical label, the orientation information may be obtained from the server using the identification information. Then, based on the orientation information of the optical tag and the perspective distortion of the imaging of the optical tag on the user's mobile phone, the relative direction of the user and the optical tag can be calculated.

It should be noted that this step 101 is not a necessary step of the present invention, and may be omitted in some cases. For example, if the target light tag itself is already within the field of view of the drone.

Step 102: Collect information transmitted by a surrounding optical tag through a CMOS camera installed on the drone, and identify the transmitted information.

After flying to the buyer's optical tag, the drone can find the cursor in its field of view and collect the information transmitted by the discovered optical tag through the CMOS camera installed on it and identify the transmitted information. For example, a drone can obtain a continuous multi-frame image of a certain optical tag through its CMOS camera, and determine, for each frame image, whether a portion of the image corresponding to the position of the light source has streaks or which type of streak exists. And determining the information represented by each frame of image. In one embodiment, if the drone finds an optical tag within its field of view, but the distance is too far to recognize the information it transmits, the drone can properly access the optical tag to achieve Identification of information conveyed by optical tags.

Step 103: Determine whether the optical tag is a target optical tag based on the transmitted information.

The drone can determine whether the optical tag is a target optical tag based on information transmitted by the optical tag. For example, the drone can determine whether the predetermined information is explicitly or implicitly included in the transmitted information. If included, it can be determined that the optical tag is the target optical tag, otherwise, it can be determined that the optical tag is not the target optical tag. In one embodiment, it may be up to the drone itself to determine if the optical tag is a target optical tag. In another embodiment, the drone can transmit the information conveyed by the optical tag to a server capable of communicating with the drone, and the server determines whether the optical tag is the target optical tag based on the transmitted information, and The judgment result is sent to the drone. The information transmitted by the optical tag can be encrypted information.

Step 104: If the optical tag is a target optical tag, control the drone to travel to the optical tag.

Since the drone has determined that a certain optical tag in the field of view is a flight destination, the drone can fly to the optical tag without error, for example by visual guidance of the optical tag. In one embodiment, the drone can be stopped at a distance from the light tag using existing ranging techniques, such as a position tens of centimeters from the light tag, to avoid collisions with the light tag. In one embodiment, the drone can relatively position and adjust its flight path based on the perspective distortion of the image of the optical tag it captured, such that the drone can eventually stop in a certain direction relative to the light tag, such as , the front of the light label. A shelf for receiving goods may be disposed directly in front of the optical tag, and the drone can easily deliver the goods into the shelf.

If it is determined that the optical tag is not the target optical tag, the drone can identify other optical tags in the vicinity thereof, which is similar to the above process and will not be described again. Preferably, in one embodiment, if it is determined that the optical tag is not the target optical tag, the drone can determine the relative positional relationship between the optical tag and the target optical tag. For example, the drone can determine its relative positional relationship with the optical tag by relative positioning, and can identify the optical tag based on the information transmitted by the optical tag (eg, obtain identification information of the optical tag) and obtain the The relative positional relationship between the optical tag and the target optical tag (the relative positional relationship between the optical tags can be pre-stored and can be obtained by the drone, for example), thereby determining the relationship between the drone and the target optical tag Relative positional relationship. After obtaining the relative positional relationship, the drone can fly to the vicinity of the target light tag using the relative positional relationship and optionally other navigation information (eg, GPS information).

It will be understood by those skilled in the art that the drone delivery scheme of the present invention guided by the optical tag is not limited to the optical tag disposed at the buyer's apartment (for example, the balcony of the apartment, the outer wall, etc.), which obviously can also Suitable for light tags arranged in more open areas, for example, light tags placed in a courtyard.

In addition, if the buyer does not have his own light label, or if he wishes to deliver the goods to a location where other light labels are located (for example, public light labels located in squares, parks, etc., or light labels of friends' homes), they can The relevant information of the optical tag (ie, the target optical tag) at the delivery address (eg, the ID information of the target optical tag, geographic location information, etc.) is notified to the online shopping platform. The online shopping platform can inform the drone of the corresponding information, and the drone can recognize the information transmitted by the nearby optical tag (for example, the ID information transmitted by the optical tag) after flying to the vicinity of the target optical tag, and finally determine the target light. label.

In addition, the drone delivery scheme guided by the optical tag of the present invention can be applied not only to a light tag having a fixed position, but also to a non-fixed optical tag (for example, a light tag that can be carried by a person) . For example, if a buyer wants to make a online purchase while in a plaza event and wants to be able to deliver the goods to their current location, they can inform their online shopping platform of their current geographic location information and turn on the light tags they carry with them. The optical tag may be configured to transmit predetermined information, which may be, for example, ID information of the optical tag itself, ID information of the buyer on the online shopping platform, a verification code received by the buyer from the platform after the buyer purchases on the online shopping platform, And so on, as long as the predetermined information is known to the online shopping platform and can be used to identify the buyer or the goods it purchases. After the drone arrives near the buyer's location, it can identify the information transmitted by the nearby optical tag, and finally determine the target optical tag (that is, the light tag carried by the buyer) to complete the delivery of the goods. In one embodiment, the online shopping platform can inform the buyer of the estimated arrival time of the drone so that the buyer can move freely during this time, as long as the expected arrival time is returned to the vicinity of the previous location. In one embodiment, the buyer may not return to the previous location, but may send its new location to the online shopping platform, and the online shopping platform may notify the drone of the new location so that the drone can fly to the location. Near the new location. In one embodiment, the buyer may also set the goods delivery address to an address that is expected to arrive at a certain time and instruct the online shopping platform to ship the goods to the address at that time.

In the above, the drone delivery application of the online shopping is taken as an example, but it can be understood that the drone guidance through the optical label is not limited to the above application, but can be used for each of the precise positioning requiring the drone. Applications such as automatic charging of drones, automatic mooring of drones, navigation of drone lines, etc. In addition, those skilled in the art can understand that the optical tag-based guidance of the present invention is not only applicable to the drone, but can also be applied to other types of autonomously movable machines, such as driverless cars, robots, and the like. . A CMOS camera can be mounted on a driverless car or robot and can interact with optical tags in a similar manner to drones. In one embodiment, a portion of the autonomously movable machine is moveable, but another portion is fixed. For example, the autonomously movable machine may be a machine having a fixed position on a pipeline or in a warehouse, the body portion of the machine being fixed in most cases but having one or more movable machinery arm. A CMOS camera can be mounted on a fixed portion of the machine for determining the position of the optical tag so that the movable portion of the machine (eg, a robotic arm) can be directed to the position of the optical tag. For such a machine, it is apparent that step 101 described above is not required. Additionally, it will be appreciated that the CMOS camera can also be mounted on a movable portion of the machine, for example, on each robotic arm.

References herein to "individual embodiments," "some embodiments," "an embodiment," or "an embodiment" or "an" or "an" In at least one embodiment. Thus, appearances of the phrases "in the various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" are not necessarily referring to the same implementation. example. Furthermore, the particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, the particular features, structures, or properties shown or described in connection with one embodiment may be combined, in whole or in part, with the features, structures, or properties of one or more other embodiments without limitation, as long as the combination is not Logical or not working. A phrase similar to "according to A" or "based on A" as used herein means non-exclusive, that is, "according to A" may cover "based only on A" or "according to A and B" unless special The statement or the context clearly indicates that it means "based only on A". In the present application, some illustrative operational steps are described in a certain order for clarity of explanation, but those skilled in the art will appreciate that each of these operational steps is not essential, and some of the steps may be omitted. Or be replaced by other steps. These operational steps are not necessarily required to be performed sequentially in the manner shown. Instead, some of these operational steps may be performed in a different order depending on actual needs, or performed in parallel, as long as the new execution mode is not non-logical or inoperable.

Having thus described several aspects of at least one embodiment of the present invention, it is understood that various changes, modifications and improvements can be readily made by those skilled in the art. Such changes, modifications, and improvements are intended to be within the spirit and scope of the invention.

Claims (18)

A system for guiding autonomously movable machine, comprising: Autonomously movable machine having a rolling shutter camera mounted thereon; An optical communication device comprising a light source configured to be operable in at least two modes, the at least two modes comprising a first mode and a second mode, And wherein, in the first mode, controlling, by a light source control signal having a first frequency, an attribute of light emitted by the light source to continuously change at a first frequency to capture the light source by the rolling shutter camera a fringe is present on the image of the obtained light source, and in the second mode, no streaks are present or appear on the image of the light source obtained when the light source is photographed by the rolling shutter camera Stripes of different stripes in the first mode. The system of claim 1 wherein, in said second mode, the property of the light emitted by said source is controlled by a light source control signal having a second frequency different from said first frequency to continue at a second frequency Varying to not present streaks on the image of the light source obtained when the light source is photographed by the rolling shutter camera or to present a stripe different from the stripe in the first mode. The system of claim 2 wherein said second frequency is greater than said first frequency. The system according to claim 1, wherein in said second mode, an attribute of light emitted by said light source continuously changes at said first frequency, and when said light source is photographed by said rolling shutter camera The obtained image of the light source exhibits a stripe different from the stripe in the first mode. The system of claim 1 wherein the light source is a strip light source or a spherical light source. The system of claim 1 wherein said autonomously movable machine comprises a machine that is only partially movable. A method of guiding a machine capable of autonomous movement using the system of any one of claims 1-6, comprising: Collecting information transmitted by a surrounding optical communication device through a rolling shutter camera mounted on the autonomously movable machine, and identifying the transmitted information; Determining whether the optical communication device is a target optical communication device based on the transmitted information; If the optical communication device is a target optical communication device, the autonomously movable machine or a portion thereof is controlled to travel to the optical communication device. The method of claim 7 further comprising: if said optical communication device is not a target optical communication device: Identifying the optical communication device based on information transmitted by the optical communication device, and obtaining a relative positional relationship between the optical communication device and the target optical communication device; Determining a relative positional relationship between the autonomously movable machine or a portion thereof and the optical communication device; Determining a relative positional relationship between the target optical communication device and the autonomously movable machine or a portion thereof; The autonomously moveable machine or portion thereof is directed to the target optical communication device based at least in part on a relative positional relationship between the target optical communication device and the autonomously movable machine or portion thereof. The method of claim 8 wherein determining a relative positional relationship between said autonomously movable machine or portion thereof and said optical communication device comprises: The relative positional relationship between the autonomously movable machine or a portion thereof and the optical communication device is determined by relative positioning. The method of claim 7 wherein collecting, by the rolling shutter camera mounted on the autonomously movable machine, information conveyed by a surrounding optical communication device and identifying the communicated information comprises: Obtaining a continuous multi-frame image of the optical communication device by the rolling shutter camera; Determining, for each frame image, whether a portion of the image corresponding to the position of the light source has stripes or which type of stripes exist; The information represented by each frame of image is determined. The method of claim 7 further comprising first controlling said autonomously movable machine to travel to a vicinity of the target optical communication device. The method of claim 11 wherein first controlling the autonomously movable machine to travel to the vicinity of the target optical communication device comprises: Directing the autonomously movable machine to the vicinity of the target optical communication device, at least in part, by a satellite navigation system; and/or The autonomously movable machine is directed to the vicinity of the target optical communication device based at least in part on a relative positional relationship between the other optical communication device and the target optical communication device. The method of claim 12, wherein the at least partially utilizing a relative positional relationship between the other optical communication device and the target optical communication device directs the autonomously moveable machine to the vicinity of the target optical communication device comprises : The autonomously movable machine identifies other optical communication devices while traveling, and obtains a relative positional relationship between the other optical communication devices and the target optical communication device; Determining a relative positional relationship between the autonomously movable machine and the other optical communication device; Determining a relative positional relationship between the target optical communication device and the autonomously movable machine; The autonomously moveable machine is directed to the vicinity of the target optical communication device based at least in part on a relative positional relationship between the target optical communication device and the autonomously movable machine. The method of claim 7, wherein determining whether the optical communication device is a target optical communication device based on the communicated information comprises: It is judged whether or not the predetermined information is explicitly or implicitly included in the transmitted information. The method of claim 14, wherein the predetermined information is a predetermined identifier or a verification code. The method of claim 7, wherein determining whether the optical communication device is a target optical communication device based on the communicated information comprises: Determining, by the autonomously movable machine, whether the optical communication device is a target optical communication device; or The autonomously movable machine transmits the transmitted information to a server, and the server determines, based on the transmitted information, whether the optical communication device is a target optical communication device, and transmits the determination result to the autonomously movable machine. A machine capable of autonomous movement, comprising a rolling shutter camera, a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, can be used to implement any of claims 7-16 One of the methods described. A storage medium having stored therein a computer program, which when executed, can be used to implement the method of any of claims 7-16.

Images