WO2018032630A1 - Teaching toy kit and method for identifying programming module by using color and counter - Google Patents

Abstract

A teaching toy kit and a method for identifying a programming module by using color and counter. The teaching toy kit comprises a support (1), a helmet detector (2), and a programming module (3). The helmet detector (2) is mounted on the support (1). The programming module (3) is placed on a plane. A protrusion is provided at the bottom of the support (1), and a first recess (102) and a second recess (103) are provided on the top. The first recess (102) is used for holding a tablet computer. The helmet detector (2) is mounted in the second recess (103) and further comprises a body (201), a third recess (202), two fan-shaped blocks (203), and a convex mirror (204). The third recess (202) is positioned in the body (201). According to the solution, the type of a programming module can be identified on the basis of color and contour information of an icon on the programming module, and then is provided to upper-layer software for subsequent determination processing. The present invention makes a game funnier, enhances manipulative ability of a kid, improves interactivity, and helps a kid be familiar with and interested in programming.

Description

Educational toy kit and method for identifying programming module using color and contour Technical field

The present invention relates to the field of computer vision detection processing technology, and in particular to an educational toy kit and a method for recognizing a programming module using color and contour.

Background technique

There are many interesting preschool game apps or children's games on the tablet, but often just let the children point finger on the screen, the interaction is lacking, and watching the screen for a long time is easy to cause damage to the eyes; and now some interactive Traditional game toys have been separated from the development of the times, and the form cannot meet the needs of children to learn and play, and it is not convenient for children and parents to interact.

In order to solve the above problems, an educational toy kit has been successfully developed in the field of computer vision and image processing technology, including: a bracket, a helmet detector and a bottom plate, and a game program is installed in the tablet, and is collected on a flat surface by a tablet computer. The image of the bottom plate.

technical problem

Although the educational toy kit mentioned above solves the problem of lack of interaction of games in the tablet computer, the form is still single, and there is no game module corresponding to the game software, the game module cannot be identified, and the upper layer software cannot be provided for subsequent processing and Analysis, it is impossible to cultivate children's understanding and interest in programming from an early age.

Therefore, there is an urgent need in the field of computer vision detection processing technology for an educational toy kit and a method for recognizing a programming module by using a color and contour. The programming game module is placed on a plane, a game program is installed in the tablet computer, and the camera is captured by a tablet computer camera. Data, using the color and contour information of the icon on the programming module, identify the programming module in the camera data, and provide the upper layer software for subsequent judgment processing; enhance the fun of the game, children's hands-on ability and interactivity, and cultivate the child well. Programming knowledge and interest.

Technical solution

In order to solve the above problems, the present invention provides an educational toy kit and a method for recognizing a programming module by using a color and a contour. The technical solution is as follows:

An educational toy kit comprising: a bracket, a helmet detector and a programming module, and the helmet detector is mounted on the bracket, the programming module is placed on a plane; the bottom of the bracket has a protrusion, and the top has a first groove and a second groove The first groove is used for placing a tablet computer, and the tablet computer collects programming module information; the helmet detector is installed in the second groove;

The helmet detector further includes a body, a third groove, two segments and a convex mirror, and the third groove is located in the body for holding different types of tablets, and holding the tablet screen in the third groove The end of the convex mirror is provided with a convex mirror, the other end of the convex mirror is mounted on the edge of the helmet detector, the convex mirror is at an acute angle with the horizontal plane, and the third groove holds the end of the tablet screen higher than the camera position of the tablet, 2 The segments are located on the two edges of the convex mirror to hold the convex mirror and hold the tablet.

Preferably, in one of the educational toy kits described above, there are eight types of programming modules, and each type of programming module has a different color.

Preferably, in the above educational toy kit, the programming module comprises: a monster module, a jump module, a condition 1 module, a condition 2 module, a small q module, a forward module, a left turn module and a right turn module.

A method for programming module identification using color and contour information in an educational toy kit, comprising the following steps:

Step one, installing a game program in the tablet, the programming module is placed on the plane, the bottom end of the tablet is installed in the first groove, and the helmet detector is installed on the top of the tablet through the second groove;

Step 2: After the fixed installation, the color image is collected in real time through the front camera of the tablet computer;

Step 3: Detect the type of the programming module from the color image of step 2.

Preferably, in the above method for educational module identification using color and contour information in an educational toy kit, the color image acquired by the front camera in step 2 is I _xy , I _xy =f(x, y)=( R _xy , G _xy , B _xy ), where (x, y) represents the position coordinates of the color image pixel points, f(x, y) represents the pixel value of the image at the pixel point coordinate position, and R _xy represents the image pixel point. In the color value of the red channel, G _xy represents the color value of the image pixel in the green channel, and B _xy represents the color value of the image pixel in the blue channel.

Preferably, in the above method for educational module identification using color and contour information in an educational toy kit, the specific steps of detecting the type of the programming module in step three are:

1) using a perspective transformation principle, converting the color image I _xy into a front view image viewed from top to bottom;

2) extracting the image of the region of interest in the positive view image according to the prior knowledge, that is, the designated placement area of the programming module given by the upper layer software;

3) Because the monster and the jump module are red in color, they are more distinguishable from the colors of other modules. Therefore, the two modules are first distinguished from other modules by color; however, the color of the programming module is in RGB color. The space is not conducive to segmentation, and is sensitive to illumination changes. Therefore, the extracted region of interest image is converted from the RGB color space to the HSV color space that focuses on the color representation. The specific conversion formula is:

V=max{C(R'), C(G'), C(B')};

Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360;

4) In the HSV color space, the color image is binarized according to the a priori threshold of the red monster module and the jump module in the HSV space. The specific formula is as follows:

A corresponding binary image can be obtained by the formula B(x, y) = B_H(x, y) & B_S(x, y) & B_V(x, y).

Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise the value is 0; H _min and H _max respectively represent the hue of the red monster programming module and the skip programming module in the HSV color space. The a priori minimum and maximum values; S _min and S _max respectively represent the a priori minimum and maximum values of the saturation of the red monster programming module and the jump programming module in the HSV color space; V _min and V _max respectively represent the red monster programming The a priori minimum and maximum values of the brightness of the module and the skip programming module within the HSV color space;

5) removing the noise interference of the binarized image in step 4) by using the method of first etching and then expanding, and smoothing the boundary of the object;

6) Scan the binarized image to find all edge contours;

The binarized image can be regarded as a grayscale image with only two values. The edge of the image refers to the part of the grayscale image where the grayscale changes are more severe. The degree of change of the grayscale value is quantified by the gradient change between adjacent pixels. The gradient is a two-dimensional equivalent of the first-order two-dimensional derivative. The specific calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Further, according to the prior knowledge of the monster mode, the size of the edge contour of the jump module and the eccentricity, the unreasonable edge contour is filtered out, and then the remaining contour is determined, thereby determining whether the programming module is a monster model or Jump module, if yes, complete the programming module decision, if not, continue to step 7);

Among them, the rule of the monster model is: using the shape, size, eccentricity of the edge contour and the prior knowledge of the position and size relationship of the three small triangles unique to the monster module to determine whether the programming module is a monster module; The principle of the module is to divide the smallest outer envelope of the edge contour into four equal parts, using the shape between the upper left, upper right, lower left and lower right contours, the size ratio of the four contours, and the four contours occupying the entire edge contour. a priori knowledge of the scale and a priori knowledge of the size and eccentricity of the entire edge contour to determine if the programming module is a jump module;

7) Convert the image of the region of interest in step 2) to a grayscale image:

Gray(x,y)=0.2989×R _xy +0.5870×G _xy +0.1140×B _xy

Where Gray(x, y) represents a grayscale image;

8) using edge detection algorithm to detect strong edges in the image;

The edge of the image refers to the part of the gray image where the gray level changes sharply. The degree of change of the gray value is quantitatively represented by the gradient change between adjacent pixels. The gradient is the two-dimensional equivalent of the first-order two-dimensional derivative. The calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value of the image at the (x, y) point,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Due to the programming module target to be detected, after conversion to the grayscale image, there is a large contrast between the identification on the programming module and the white area on the programming module, so the edge detection can be used to extract the edge contour identified on the programming module. The edge extraction algorithm includes Sobel operator, Roberts operator, Prewitt operator and Canny operator. The specific formula is:

Where |G(x,y)| represents the gradient magnitude of the edge point;

9) expanding the edge contour of the programming module obtained in step 8);

10) Scanning the binarized image in step 9), since the current programming module belongs to a non-red system, it is determined which programming module belongs to the type other than the monster module and the skip module:

First, the binarized image obtained in step 9) is scanned to obtain a corresponding edge contour, and the unreasonable contour is filtered out by the prior knowledge of the size of the edge contour and the eccentricity;

Further, the minimum circumscribed rectangle of the edge contour remaining after filtering is divided into 4 equal parts, and the shape between the upper left, upper right, lower left, and lower right contours, the size ratio relationship of the four contours, and the four contours occupy the entire edge contour The a priori knowledge of the ratio and the prior knowledge of the size and eccentricity of the entire edge profile to determine the type of programming module.

Preferably, in the method for programming module identification using color and contour information in an educational toy kit as described above, the effective recognition area placed by the programming module in step 2) corresponds to only one identification module.

Preferably, in the method for using the color and contour information to perform programming module identification in the above educational toy kit, in step 10), if it is determined that the programming module is a conditional module, the centroid of the edge contour of the programming module is taken and judged. The color of the centroid is compared with the prior knowledge, and the programming module is the condition 1 module or the condition 2 module.

Preferably, in the method for identifying the programming module by using color and contour information in the above educational toy kit, in step 10), if it is determined that the programming module is a turning module, the turning point of the edge contour of the programming module is taken The midpoint of the diagonal point connection is judged by the color of the midpoint, and then compared with the prior knowledge, and the programming module is a left turn module or a right turn module.

Beneficial effect

1. The invention intelligently combines the application of computer vision graphic recognition technology with HSV color space, binarization processing, grayscale image and image cutting technology, can determine the type of programming module, has fast computing speed and accurate positioning. The hardware and software technology are well unified, the game interaction design is ingenious; the appearance is simple, the judgment is faster, and the fun and the intuitiveness are enhanced.

2. The detection algorithm of the invention is more scientific and mature, and the combination of image perspective transformation, color space conversion, contour detection, morphological processing, denoising and the like can quickly determine the type of the programming module.

3. The calculation speed of the invention is fast; each positioning detection takes about 100ms, which provides a smooth experience for the player.

4. The performance of the invention is stable. In the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test of 3,000 pictures are performed, and the false recognition rate and the missed detection rate are below 0.2%.

DRAWINGS

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.

2 is a schematic view showing the structure of a helmet detector in an educational toy kit of the present invention.

3 is a flow chart of a method for programming module identification using color and contour information in an educational toy kit of the present invention.

The correspondence between the reference numerals in Figure 1-3 and the part names is:

Bracket 1, first groove 102, second groove 103, helmet detector 2, body 201, third groove 202, sector block 203, mirror 204, programming module 3.

BEST MODE FOR CARRYING OUT THE INVENTION

As shown in FIG. 3, a method for programming module identification using color and contour information in an educational toy kit includes the following steps:

Step one, installing a game program in the tablet, the programming module is placed on the plane, the bottom end of the tablet is installed in the first groove, and the helmet detector is installed on the top of the tablet through the second groove;

Step 2: After the fixed installation, the color image is collected in real time through the front camera of the tablet computer;

The color image acquired by the front camera is I _xy , I _xy =f(x,y)=(R _xy , G _xy , B _xy ), where (x, y) represents the position coordinate of the pixel of the color image, f( x, y) represents the pixel value of the image at the pixel point coordinate position, R _xy represents the color value of the image pixel point in the red channel, G _xy represents the color value of the image pixel point in the green channel, and B _xy represents the image pixel point in the blue The color value of the color channel.

Step 3: detecting the type of the programming module from the color image in step two, the specific steps are:

1) using a perspective transformation principle, converting the color image I _xy into a front view image viewed from top to bottom;

2) According to the prior knowledge, the image of the region of interest is extracted in the positive view image, that is, the programming module specifies the placement area given by the upper layer software; the effective recognition region placed by the programming module corresponds to only one identification module;

3) Because the monster and the jump module are red in color, they are more distinguishable from the colors of other modules. Therefore, the two modules are first distinguished from other modules by color; however, the color of the programming module is in RGB color. The space is not conducive to segmentation, and is sensitive to illumination changes. Therefore, the extracted region of interest image is converted from the RGB color space to the HSV color space that focuses on the color representation. The specific conversion formula is:

V=max{C(R'), C(G'), C(B')};

Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360;

4) In the HSV color space, the color image is binarized according to the a priori threshold of the red monster module and the jump module in the HSV space. The specific formula is as follows:

Obtaining a corresponding binary image by the formula B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y);

Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, V area, if yes, the value is 1, otherwise, the value is 0; H _min , H _max respectively represent the red system programming module (monster programming module and jump programming module) in HSV color The a priori minimum and maximum values of the hue in the space; S _min and S _max respectively represent the minimum and maximum values of the prior saturation of the red monster programming module and the jump programming module in the HSV color space; V _min and V _max respectively represent The a priori minimum and maximum values of the brightness of the red monster programming module and the jump programming module within the HSV color space;

5) removing the noise interference of the binarized image in step 4) by using the method of first etching and then expanding, and smoothing the boundary of the object;

6) Scan the binarized image to find all edge contours;

The binarized image can be regarded as only two grayscale images. The edge of the image refers to the part of the grayscale image where the grayscale changes are more intense. The degree of change of the grayscale value is quantified by the gradient change between adjacent pixels. The gradient is a two-dimensional equivalent of the first-order two-dimensional derivative. The specific calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Further, according to the prior knowledge of the monster mode, the size of the edge contour of the jump module and the eccentricity, the unreasonable edge contour is filtered out, and then the remaining contour is determined, thereby determining whether the programming module is a monster model or Jump module, if yes, complete the programming module decision, if not, continue to step 7);

Among them, the rule of the monster model is: using the shape, size, eccentricity of the edge contour and the prior knowledge of the position and size relationship of the three small triangles unique to the monster module to determine whether the programming module is a monster module; The principle of the module is to divide the smallest outer envelope of the edge contour into four equal parts, using the shape between the upper left, upper right, lower left and lower right contours, the size ratio of the four contours, and the four contours occupying the entire edge contour. a priori knowledge of the scale and a priori knowledge of the size and eccentricity of the entire edge contour to determine if the programming module is a jump module;

7) Convert the image of the region of interest in step 2) to a grayscale image:

Gray(x,y)=0.2989×R _xy +0.5870×G _xy +0.1140×B _xy

Where Gray(x, y) represents a grayscale image;

8) using edge detection algorithm to detect strong edges in the image;

The edge of the image refers to the part of the gray image where the gray level changes sharply. The degree of change of the gray value is quantitatively represented by the gradient change between adjacent pixels. The gradient is the two-dimensional equivalent of the first-order two-dimensional derivative. The calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value of the image at the (x, y) point,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Due to the programming module target to be detected, after conversion to the grayscale image, there is a large contrast between the identification on the programming module and the white area on the programming module. Therefore, the edge detection method can be used to extract the edge represented on the programming module. Contour; edge extraction algorithm includes Sobel operator, Roberts operator, Prewitt operator and Canny operator. The specific formula is:

Where |G(x,y)| represents the gradient magnitude of the edge point;

9) expanding the edge contour of the programming module obtained in step 8);

10) Scanning the binarized image in step 9), since the current programming module belongs to a non-red system, it is determined that the programming module belongs to any type other than the monster module and the jump module;

First, the image of the binary image obtained in step 9) is scanned to obtain a corresponding edge contour, and the unreasonable contour is filtered out by the prior knowledge of the size of the edge contour and the eccentricity;

Further, the minimum circumscribed rectangle of the edge contour remaining after filtering is divided into 4 equal parts, and the shape between the upper left, upper right, lower left, and lower right contours, the size ratio relationship of the four contours, and the four contours occupy the entire edge contour The a priori knowledge of the ratio and the prior knowledge of the size and eccentricity of the entire edge profile to determine the type of programming module.

If it is determined that the programming module is a conditional module, the centroid of the edge contour of the programming module is taken, the color of the centroid is judged, and then compared with the prior knowledge, and the programming module is a conditional 1 module or a conditional 2 module.

If it is determined that the programming module is a turning module, the turning point of the edge contour of the programming module is taken from the midpoint of the line connecting the diagonal point thereof, and then compared with the prior knowledge, and the programming module is a left turn module or a right turn module. .

The invention skillfully combines the application of computer vision graphic recognition technology with HSV color space, binarization processing, grayscale image and image cutting technology, can determine the type of programming module, has fast computing speed, accurate positioning, and hardware It is well integrated with the software technology, the game interaction design is clever; the appearance is simple, the judgment is faster, and the fun and the intuitiveness are enhanced.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Example 1:

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.

2 is a schematic view showing the structure of a helmet detector in an educational toy kit of the present invention.

As shown in FIG. 1-2, a marine mold recognition method for an educational toy kit includes a bracket 1, a helmet detector 2, and a programming module 3, and the helmet detector 2 is mounted on the bracket 1; the programming module 3 is placed on a plane The top of the bracket 1 has a first recess 102 for placing a tablet computer, the tablet computer collects programming module information, and the helmet detector 2 is mounted in the second recess 103 The helmet detector 2 further includes: a body 201, a third groove 202, two sector blocks 203 and a convex mirror 204, and the third groove 202 is located in the body 201 for clamping different types of tablets, in the The end of the three-groove 202 holding the tablet screen is provided with a convex mirror 204. The other end of the convex mirror 204 is mounted on the edge of the helmet detector 2. The convex mirror 204 is at an acute angle with the horizontal plane, and the third recess 202 is clamped. The end of the tablet screen is higher than the camera position of the tablet, and two sectors 203 are located on the two edges of the convex mirror 204 for fixing the convex mirror 204 and holding the tablet.

3 is a flow chart of a method for programming module identification using color and contour information in an educational toy kit of the present invention.

As shown in FIG. 3, a method for programming module identification using color and contour information in an educational toy kit includes the following steps:

Step one, installing a game program in the tablet, the programming module is placed on the plane, the bottom end of the tablet is installed in the first groove, and the helmet detector is installed on the top of the tablet through the second groove;

Step 2: After the fixed installation, the color image is collected in real time through the front camera of the tablet computer;

The color image acquired by the front camera is I _xy , I _xy =f(x,y)=(R _xy , G _xy , B _xy ), where (x, y) represents the position coordinate of the pixel of the color image, f( x, y) represents the pixel value of the image at the pixel point coordinate position, R _xy represents the color value of the image pixel point in the red channel, G _xy represents the color value of the image pixel point in the green channel, and B _xy represents the image pixel point in the blue The color value of the color channel.

Step 3: detecting the type of the programming module from the color image in step two, the specific steps are:

1) using a perspective transformation principle, converting the color image I _xy into a front view image viewed from top to bottom;

2) According to the prior knowledge, the image of the region of interest is extracted in the positive view image, that is, the programming module specifies the placement area given by the upper layer software; the effective recognition region placed by the programming module corresponds to only one identification module;

3) Because the monster and the jump module are red in color, they are more distinguishable from the colors of other modules. Therefore, the two modules are first distinguished from other modules by color; however, the color of the programming module is in RGB color. The space is not conducive to segmentation, and is sensitive to illumination changes. Therefore, the extracted region of interest image is converted from the RGB color space to the HSV color space that focuses on the color representation. The specific conversion formula is:

V=max{C(R'), C(G'), C(B')};

Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. , blue three channel image The prime value, min{C(R'), C(G'), C(B')} represents the pixel minimum of one pixel in the original image in red, green and blue, and the H is taken The value range is between 0-360;

4) In the HSV color space, the color image is binarized according to the a priori threshold of the red monster module and the jump module in the HSV space. The specific formula is as follows:

Obtaining a corresponding binary image by the formula B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y);

Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, V area, if yes, the value is 1, otherwise, the value is 0; H _min , H _max respectively represent the red system programming module (monster programming module and jump programming module) in HSV color The a priori minimum and maximum values of the hue in the space; S _min and S _max respectively represent the minimum and maximum values of the prior saturation of the red monster programming module and the jump programming module in the HSV color space; V _min and V _max respectively represent The a priori minimum and maximum values of the brightness of the red monster programming module and the jump programming module within the HSV color space;

5) removing the noise interference of the binarized image in step 4) by using the method of first etching and then expanding, and smoothing the boundary of the object;

6) Scan the binarized image to find all edge contours;

The binarized image can be regarded as only two grayscale images. The edge of the image refers to the part of the grayscale image where the grayscale changes are more intense. The degree of change of the grayscale value is quantified by the gradient change between adjacent pixels. The gradient is a two-dimensional equivalent of the first-order two-dimensional derivative. The specific calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Further, according to the prior knowledge of the monster mode, the size of the edge contour of the jump module and the eccentricity, the unreasonable edge contour is filtered out, and then the remaining contour is determined, thereby determining whether the programming module is a monster model or Jump module, if yes, complete the programming module decision, if not, continue to step 7);

Among them, the rule of the monster model is: using the shape, size, eccentricity of the edge contour and the prior knowledge of the position and size relationship of the three small triangles unique to the monster module to determine whether the programming module is a monster module; The principle of the module is to divide the smallest outer envelope of the edge contour into four equal parts, using the shape between the upper left, upper right, lower left and lower right contours, the size ratio of the four contours, and the four contours occupying the entire edge contour. a priori knowledge of the scale and a priori knowledge of the size and eccentricity of the entire edge contour to determine if the programming module is a jump module;

7) Convert the image of the region of interest in step 2) to a grayscale image:

Gray(x,y)=0.2989×R _xy +0.5870×G _xy +0.1140×B _xy

Where Gray(x, y) represents a grayscale image;

8) using edge detection algorithm to detect strong edges in the image;

The edge of the image refers to the part of the gray image where the gray level changes sharply. The degree of change of the gray value is quantitatively represented by the gradient change between adjacent pixels. The gradient is the two-dimensional equivalent of the first-order two-dimensional derivative. The calculation process is:

First, calculate the difference between adjacent pixels. The specific formula is:

G _x =f[i,j+1]-f[i,j]

G _y =f[i,j]-f[i+1,j]

Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,

Further, the gradient between adjacent pixels is calculated, and the specific formula is:

表示像素值在x方向上求导，

表示像素值在y方向上求导；Where G(x, y) represents the gradient value of the image at the (x, y) point,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction;

Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;

Due to the programming module target to be detected, after conversion to the grayscale image, there is a large contrast between the identification on the programming module and the white area on the programming module. Therefore, the edge detection method can be used to extract the edge represented on the programming module. Contour; edge extraction algorithm includes Sobel operator, Roberts operator, Prewitt operator and Canny operator. The specific formula is:

Where |G(x,y)| represents the gradient magnitude of the edge point;

9) expanding the edge contour of the programming module obtained in step 8);

10) Scanning the binarized image in step 9), since the current programming module belongs to a non-red system, it is determined that the programming module belongs to any type other than the monster module and the jump module;

First, the image of the binary image obtained in step 9) is scanned to obtain a corresponding edge contour, and the unreasonable contour is filtered out by the prior knowledge of the size of the edge contour and the eccentricity;

Further, the minimum circumscribed rectangle of the edge contour remaining after filtering is divided into 4 equal parts, and the shape between the upper left, upper right, lower left, and lower right contours, the size ratio relationship of the four contours, and the four contours occupy the entire edge contour The a priori knowledge of the ratio and the prior knowledge of the size and eccentricity of the entire edge profile to determine the type of programming module.

If it is determined that the programming module is a conditional module, the centroid of the edge contour of the programming module is taken, the color of the centroid is judged, and then compared with the prior knowledge, and the programming module is a conditional 1 module or a conditional 2 module.

If it is determined that the programming module is a turning module, the turning point of the edge contour of the programming module is taken from the midpoint of the line connecting the diagonal point thereof, and then compared with the prior knowledge, and the programming module is a left turn module or a right turn module. .

The invention skillfully combines the application of computer vision graphic recognition technology with HSV color space, binarization processing, grayscale image and image cutting technology, can determine the type of programming module, has fast computing speed, accurate positioning, and hardware It is well integrated with the software technology, the game interaction design is clever; the appearance is simple, the judgment is faster, and the fun and the intuitiveness are enhanced.

The detection algorithm of the invention is more scientific and mature, and combines the perspective transformation of the image, the color space conversion, the contour detection, the morphological processing, the denoising and the like, and can quickly determine the type of the programming module.

The calculation speed of the invention is fast; each positioning detection takes about 100ms, which provides a smooth experience for the player.

The performance of the invention is stable, and in the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test are performed on 3,000 pictures, and the false recognition rate and the missed detection rate are below 0.2%.

The basic principles, main features, and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited by the foregoing embodiments, and that the present invention is described in the foregoing description and the description of the present invention. Such changes and modifications are intended to fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and their equivalents.

Industrial applicability

It is easily known to those skilled in the art from the above description that the technical solution of the present invention is suitable for industrial production and is used in production and life, and therefore the present invention has industrial applicability.

Claims (9)

An educational toy kit, comprising: a bracket, a helmet detector and a programming module, and the helmet detector is mounted on a bracket, the programming module is placed on a plane; the bracket has a protrusion at the bottom, the top The first groove and the second groove are used for placing a tablet computer, and the tablet computer collects programming module information; the helmet detector is installed in the second groove; The helmet detector further includes a body, a third groove, two segments and a convex mirror, and the third groove is located in the body for clamping different types of tablets in the third concave A convex mirror is disposed at an end of the slot holding tablet screen, and the other end of the convex mirror is mounted on an edge of the helmet detector, the convex mirror is at an acute angle with the horizontal plane, and the third recess holds the tablet The end of the screen is higher than the camera position of the tablet, and the two sectors are located on the two edges of the convex mirror for fixing the convex mirror and holding the tablet. The educational toy kit according to claim 1, wherein the programming modules are of a type of eight, and each of the types of programming modules is different in color. The educational toy kit according to claim 2, wherein the programming module comprises: a monster module, a jump module, a condition 1 module, a condition 2 module, a small q module, a forward module, a left turn module, and a right Transfer module. A method for programming module identification using color and contour information in an educational toy kit, comprising the steps of: Step one, installing a game program in the tablet, the programming module is placed on the plane, the bottom end of the tablet is installed in the first groove, and the helmet detector is installed on the top of the tablet through the second groove; Step 2: After the fixed installation, the color image is collected in real time through the front camera of the tablet computer; Step three, detecting the type of the programming module from the color image of the second step. The method for identifying a programming module by using color and contour information in an educational toy kit according to claim 4, wherein the color image acquired by the front camera in the second step is I _xy , I _xy = f ( x, y) = (R _xy , G _xy , B _xy ), where (x, y) represents the positional coordinates of the color image pixel points, and f(x, y) represents the pixel value of the image at the pixel point coordinate position, R _xy represents the color value of the image pixel in the red channel, G _xy represents the color value of the image pixel in the green channel, and B _xy represents the color value of the image pixel in the blue channel. The method for identifying a programming module by using color and contour information in an educational toy kit according to claim 5, wherein the specific steps of detecting the type of the programming module in the third step are: 1) using a perspective transformation principle, converting the color image I _xy into a front view image viewed from top to bottom; 2) extracting the image of the region of interest in the positive view image according to the prior knowledge, that is, the designated placement area of the programming module given by the upper layer software; 3) Because the monster and the jump module are red in color, they are distinguished from the colors of other modules. Therefore, the two modules are first distinguished from other modules by color; that is, the color of the programming module is in RGB. The color space is not conducive to segmentation, and is sensitive to illumination changes. Therefore, the extracted region of interest image is converted from the RGB color space to the HSV color space that focuses on the color representation. The specific conversion formula is: V=max{C(R'), C(G'), C(B')};

Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360; 4) In the HSV color space, the color image is binarized according to the a priori threshold of the red monster programming module and the jump programming module in the HSV space. The specific formula is as follows:

Obtaining a corresponding binary image by the formula B(x, y)=B-H(x,y)&B-S(x,y)&B-V(x,y); Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; BH(x, y), BS(x, y), BV(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, V area, if yes, the value is 1, otherwise, the value is 0; H _min , H _max respectively represent the hue of the red monster programming module and the jump programming module in the HSV color space The a priori minimum and maximum values; S _min and S _max respectively represent the a priori minimum and maximum values of the saturation of the red monster programming module and the jump programming module in the HSV color space; V _min and V _max respectively represent the red monsters The a priori minimum and maximum values of the brightness of the programming module and the jump programming module within the HSV color space; 5) removing the noise interference of the binarized image in the step 4) by using the method of first etching and then expanding, and smoothing the boundary of the object; 6) Scan the binarized image to find all edge contours; The binarized image can be regarded as a grayscale image with only two values. The edge of the image refers to the part of the grayscale image where the grayscale changes are more severe. The degree of change of the grayscale value is quantified by the gradient change between adjacent pixels. The gradient is a two-dimensional equivalent of the first-order two-dimensional derivative. The specific calculation process is: First, calculate the difference between adjacent pixels. The specific formula is: G _x =f[i,j+1]-f[i,j] G _y =f[i,j]-f[i+1,j] Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column; Further, the gradient between adjacent pixels is calculated, and the specific formula is:

Where G(x, y) represents the gradient value at the (x, y) point of the image,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction; Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour; Further, according to the prior knowledge of the monster mode, the size of the edge contour of the jump module and the eccentricity, the unreasonable edge contour is filtered out, and then the remaining contour is determined, thereby determining whether the programming module is a monster model or Jump module, if yes, complete the programming module decision, if not, continue to step 7); Among them, the rule of the monster model is: using the shape, size, eccentricity of the edge contour and the prior knowledge of the position and size relationship of the three small triangles unique to the monster module to determine whether the programming module is a monster module; The principle of the module is to divide the smallest outer envelope of the edge contour into four equal parts, using the shape between the upper left, upper right, lower left and lower right contours, the size ratio of the four contours, and the four contours occupying the entire edge contour. a priori knowledge of the scale and a priori knowledge of the size and eccentricity of the entire edge contour to determine if the programming module is a jump module; 7) converting the image of the region of interest in step 2) into a grayscale image: Gray(x,y)=0.2989×R _xy +0.5870×G _xy +0.1140×B _xy Where Gray(x, y) represents a grayscale image; 8) using edge detection algorithm to detect strong edges in the image; The edge of the image refers to the part of the gray image where the gray level changes sharply. The degree of change of the gray value is quantitatively represented by the gradient change between adjacent pixels. The gradient is the two-dimensional equivalent of the first-order two-dimensional derivative. The calculation process is: First, calculate the difference between adjacent pixels. The specific formula is: G _x =f[i,j+1]-f[i,j] G _y =f[i,j]-f[i+1,j] Where G _x represents the difference of adjacent pixels in the x direction, G _y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column; Further, the gradient between adjacent pixels is calculated, and the specific formula is:

Where G(x, y) represents the gradient value of the image at the (x, y) point,

Indicates that the pixel value is derived in the x direction.

Indicates that the pixel value is derived in the y direction; Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour; Due to the target of the programming module to be detected, after conversion to the grayscale image, there is a large contrast between the identification on the programming module and the white area on the programming module. Therefore, the edge detection method can be used to extract the identification on the programming module. Edge contouring; edge extraction algorithms include Sobel operator, Roberts operator, Prewitt operator, and Canny operator. The specific formula is:

Where |G(x,y)| represents the gradient magnitude of the edge point; 9) performing an expansion process on the edge contour of the programming module obtained in the step 8); 10) Scanning the binarized image in the step 9), since the current programming module belongs to a non-red system, it is determined which type of the programming module belongs to the monster module and the skip module; First, the binarized image obtained in the step 9) is scanned to obtain a corresponding edge contour, and the unreasonable contour is filtered out by the prior knowledge of the size of the edge contour and the eccentricity; Further, the minimum circumscribed rectangle of the edge contour remaining after filtering is divided into 4 equal parts, and the shape between the upper left, upper right, lower left, and lower right contours, the size ratio relationship of the four contours, and the four contours occupy the entire edge contour The a priori knowledge of the ratio and the prior knowledge of the size and eccentricity of the entire edge profile to determine the type of programming module. The method for identifying a programming module using color and contour information in an educational toy condition according to claim 6, wherein the effective identification area placed by the programming module in the step 2) corresponds to only one identification module. The method for identifying a programming module by using color and contour information in an educational toy condition according to claim 6, wherein in the step 10), if it is determined that the programming module is a conditional module, the programming module is taken. The centroid of the edge contour, judging the color of the centroid, and then comparing with the prior knowledge, the programming module is the condition 1 module or the condition 2 module. The method for identifying a programming module by using color and contour information in an educational toy condition according to claim 6, wherein in the step 10), if it is determined that the programming module is a turning module, the programming module is taken. The turning point of the edge contour and the midpoint of the line connecting the diagonal points are compared with the prior knowledge, and the programming module is a left-turning module or a right-turning module.

Images